mirror of
https://github.com/block/buzz.git
synced 2026-10-31 06:21:34 +00:00
docs(desktop): define performance cutover contracts
Add the milestone-zero architecture contract and a fail-closed packaged WKWebView acceptance foundation with deterministic fixtures, provenance-bound artifacts, environment capture, and validation tooling. Co-authored-by: Carl <32a2e2c9d428ee08902cab75d956da2c1d235a22d4766b0dd4138bf6e2e5db1d@buzz.block.builderlab.xyz> Signed-off-by: Wes <wesbillman@users.noreply.github.com>
This commit is contained in:
Wes
82
desktop/tests/wkwebview/README.md
Normal file
82
desktop/tests/wkwebview/README.md
Normal file
@@ -0,0 +1,82 @@
|
||||
# Packaged WKWebView acceptance foundation
|
||||
|
||||
This directory is the additive, non-production contract for measuring Buzz in its
|
||||
shipped macOS WKWebView. It intentionally does **not** use Playwright or Chrome as
|
||||
a fallback. A run without an explicit native input driver is written as
|
||||
`status: "blocked"` and exits with status 2.
|
||||
|
||||
## Artifacts
|
||||
|
||||
- `fixtures/acceptance-v1.json` is the deterministic fixture and scenario
|
||||
taxonomy. Its pinned generator version, seed, reference clock, event-ID
|
||||
derivation, ordering contract, output count/bytes, and full JSONL SHA make
|
||||
relay materialization reproducible. `tooling/materialize-fixture.mjs` is the
|
||||
executable materializer. `fixture-manifest.json` separately pins the recipe’s
|
||||
byte count and SHA-256 digest.
|
||||
- `schemas/` defines the immutable fixture, captured environment, run envelope,
|
||||
and raw result contracts.
|
||||
- `tooling/capture-environment.mjs` captures required package/host evidence
|
||||
directly from the caller-supplied packaged `.app` and current macOS host:
|
||||
Info.plist identity/version, code-signing identity and entitlements, hardware,
|
||||
macOS/WebKit, main display, power/low-power/thermal state, and exact native
|
||||
driver bytes. Missing evidence is fatal; callers do not author release
|
||||
environment metadata by hand.
|
||||
- `tooling/launch.mjs` verifies that captured environment, fixture and driver
|
||||
provenance, then hands a run ID plus the exact environment-file digest to the
|
||||
external native driver. It verifies only the supplied packaged `.app`; it does
|
||||
not claim that the bundle came from a particular release pipeline. Completion
|
||||
rejects result substitution.
|
||||
- `tooling/report.mjs` rejects malformed or incomplete raw samples and emits
|
||||
nearest-rank p50/p95/p99 summaries while retaining package, hardware, OS,
|
||||
WebKit, display, runtime-state, relay, fixture, driver, and raw-file provenance.
|
||||
|
||||
The native driver interface is intentionally narrow: the executable receives
|
||||
`--wkwebview-run <absolute-run.json>` and the same path in
|
||||
`BUZZ_WKWEBVIEW_RUN`. It must drive native wheel/trackpad/keyboard input against
|
||||
the `.app` in the run envelope and write the declared raw result path. A browser
|
||||
driver is not contract-compatible.
|
||||
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
# Materialize the immutable relay fixture (exclusive output).
|
||||
node tests/wkwebview/tooling/materialize-fixture.mjs \
|
||||
/path/to/events.jsonl
|
||||
|
||||
# Capture package and host evidence. Failure to capture a required field blocks
|
||||
# acceptance; do not hand-author environment.json.
|
||||
node tests/wkwebview/tooling/capture-environment.mjs \
|
||||
--app /path/to/Buzz.app \
|
||||
--git-sha <40-hex-commit> \
|
||||
--relay-version <relay-version> \
|
||||
--driver /path/to/native-driver \
|
||||
--driver-name <driver-name> \
|
||||
--driver-version <driver-version> \
|
||||
--output /path/to/environment.json
|
||||
|
||||
# Explicitly blocked when --driver is omitted; still writes run.json.
|
||||
node tests/wkwebview/tooling/launch.mjs \
|
||||
--app /path/to/Buzz.app \
|
||||
--environment /path/to/environment.json \
|
||||
--run /path/to/run.json \
|
||||
--result /path/to/raw-result.json
|
||||
|
||||
# Native run.
|
||||
node tests/wkwebview/tooling/launch.mjs \
|
||||
--app /path/to/Buzz.app \
|
||||
--environment /path/to/environment.json \
|
||||
--run /path/to/run.json \
|
||||
--result /path/to/raw-result.json \
|
||||
--driver /path/to/native-driver
|
||||
|
||||
# Validate raw data and produce provenance-preserving percentiles.
|
||||
node tests/wkwebview/tooling/report.mjs \
|
||||
/path/to/raw-result.json /path/to/run.json /path/to/report.json
|
||||
|
||||
node --test tests/wkwebview/tooling/validation.test.mjs
|
||||
```
|
||||
|
||||
Raw result and report outputs use exclusive creation so evidence cannot be silently
|
||||
overwritten. The run envelope alone is updated in place from `running` to a
|
||||
terminal `completed` or `failed` outcome. Preserve raw samples; reports are
|
||||
derived and must never replace them.
|
||||
12
desktop/tests/wkwebview/fixture-manifest.json
Normal file
12
desktop/tests/wkwebview/fixture-manifest.json
Normal file
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"manifestVersion": 1,
|
||||
"fixtures": [
|
||||
{
|
||||
"fixtureId": "buzz-wkwebview-acceptance",
|
||||
"version": 1,
|
||||
"path": "fixtures/acceptance-v1.json",
|
||||
"bytes": 3041,
|
||||
"sha256": "efdb8dc8ef0624be8ba86dc3fc0bb3374bd266b63f5a0f661df02797bc107000"
|
||||
}
|
||||
]
|
||||
}
|
||||
103
desktop/tests/wkwebview/fixtures/acceptance-v1.json
Normal file
103
desktop/tests/wkwebview/fixtures/acceptance-v1.json
Normal file
@@ -0,0 +1,103 @@
|
||||
{
|
||||
"fixtureId": "buzz-wkwebview-acceptance",
|
||||
"version": 1,
|
||||
"relayFixtureVersion": "wkwebview-relay-v1",
|
||||
"materialization": {
|
||||
"generator": "buzz-wkwebview-fixture",
|
||||
"generatorVersion": "1.0.0",
|
||||
"algorithm": "nostr-canonical-sha256-v1",
|
||||
"seedHex": "62757a7a2d776b776965772d616363657074616e63652d7631",
|
||||
"referenceTime": "2026-08-20T00:00:00.000Z",
|
||||
"signerPrivateKeyHex": "0000000000000000000000000000000000000000000000000000000000000001",
|
||||
"signatureContract": "BIP340 with 32 zero aux-rand bytes",
|
||||
"eventKind": 9,
|
||||
"tagContract": "[[\"h\",channelId],[\"fixture\",fixtureId],[\"ordinal\",zeroBasedOrdinal decimal]]",
|
||||
"contentContract": "UTF-8 fixture:{contentProfile}:{seedHex}:{zeroBasedOrdinal decimal}",
|
||||
"eventIdContract": "NIP-01 sha256 canonical [0,pubkey,created_at,kind,tags,content]",
|
||||
"orderingContract": "created_at-ascending-then-event-id-ascending",
|
||||
"createdAtStepSeconds": 1,
|
||||
"outputEventCount": 8740,
|
||||
"outputBytes": 4255320,
|
||||
"outputSha256": "ac948604f0100b93ce9d6f143b7160b6cd7f1389e1be407ed15fc45dca72edca"
|
||||
},
|
||||
"inputSemantics": "native-wheel-trackpad-keyboard",
|
||||
"channels": [
|
||||
{
|
||||
"id": "shallow",
|
||||
"messageCount": 40,
|
||||
"contentProfile": "plain"
|
||||
},
|
||||
{
|
||||
"id": "deep",
|
||||
"messageCount": 7500,
|
||||
"contentProfile": "mixed"
|
||||
},
|
||||
{
|
||||
"id": "rich",
|
||||
"messageCount": 1200,
|
||||
"contentProfile": "media-code-embeds-reactions-threads"
|
||||
}
|
||||
],
|
||||
"scenarioTaxonomy": {
|
||||
"switch": [
|
||||
"cold-switch",
|
||||
"warm-switch",
|
||||
"repeated-a-b-switch",
|
||||
"shallow-deep-rich-switch"
|
||||
],
|
||||
"input": [
|
||||
"composer-quiet",
|
||||
"composer-agent-busy",
|
||||
"composer-ipc-storm",
|
||||
"composer-combined-load",
|
||||
"wheel-momentum",
|
||||
"keyboard-navigation"
|
||||
],
|
||||
"timeline": [
|
||||
"cascading-prepend-15-pages",
|
||||
"commit-during-momentum",
|
||||
"positioned-insert-above",
|
||||
"positioned-insert-in-viewport",
|
||||
"positioned-insert-below",
|
||||
"append-at-floor",
|
||||
"append-while-reading",
|
||||
"late-geometry-growth",
|
||||
"deep-target-with-interruption"
|
||||
],
|
||||
"plateau": [
|
||||
"full-traversal-steady-state",
|
||||
"pagination-steady-state",
|
||||
"channel-switch-steady-state"
|
||||
],
|
||||
"ipc": [
|
||||
"live-agent-storm",
|
||||
"shadow-delta-coalescing",
|
||||
"invalidate-overflow-backpressure"
|
||||
],
|
||||
"recovery": [
|
||||
"offline-reconnect",
|
||||
"relay-restart",
|
||||
"app-restart",
|
||||
"compatible-migration",
|
||||
"unknown-newer-replica",
|
||||
"rebuild-downgrade"
|
||||
],
|
||||
"crash": [
|
||||
"web-process-death",
|
||||
"app-process-death",
|
||||
"termination-before-native-commit",
|
||||
"termination-after-native-commit-before-delta",
|
||||
"termination-after-renderer-delta"
|
||||
]
|
||||
},
|
||||
"correctnessCeilings": {
|
||||
"physicalBottomPx": 1,
|
||||
"prependAnchorDriftPx": 5,
|
||||
"richResizeAnchorDriftPx": 2,
|
||||
"mountedRows": 400,
|
||||
"queueOverflows": 0,
|
||||
"blankOrStaleFrames": 0,
|
||||
"droppedInputs": 0,
|
||||
"unexplainedDataGaps": 0
|
||||
}
|
||||
}
|
||||
135
desktop/tests/wkwebview/schemas/environment.schema.json
Normal file
135
desktop/tests/wkwebview/schemas/environment.schema.json
Normal file
@@ -0,0 +1,135 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "https://buzz.local/schemas/wkwebview/environment-v1.json",
|
||||
"title": "Buzz packaged WKWebView acceptance environment",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"schemaVersion",
|
||||
"capture",
|
||||
"build",
|
||||
"package",
|
||||
"hardware",
|
||||
"platform",
|
||||
"display",
|
||||
"runtimeState",
|
||||
"fixture",
|
||||
"relay",
|
||||
"nativeDriver"
|
||||
],
|
||||
"properties": {
|
||||
"schemaVersion": { "type": "integer", "const": 1 },
|
||||
"capture": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["tool", "version", "capturedAt", "infoPlistSha256"],
|
||||
"properties": {
|
||||
"tool": { "const": "buzz-wkwebview-environment-capture" },
|
||||
"version": { "const": "1.0.0" },
|
||||
"capturedAt": { "type": "string", "format": "date-time" },
|
||||
"infoPlistSha256": { "$ref": "#/$defs/sha256" }
|
||||
}
|
||||
},
|
||||
"build": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["gitSha"],
|
||||
"properties": {
|
||||
"gitSha": { "type": "string", "pattern": "^[0-9a-f]{40}$" }
|
||||
}
|
||||
},
|
||||
"package": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"appPath",
|
||||
"bundleIdentifier",
|
||||
"version",
|
||||
"identity",
|
||||
"entitlementsSha256"
|
||||
],
|
||||
"properties": {
|
||||
"appPath": { "type": "string", "pattern": "\\.app$" },
|
||||
"bundleIdentifier": { "type": "string", "minLength": 1 },
|
||||
"version": { "type": "string", "minLength": 1 },
|
||||
"identity": { "type": "string", "minLength": 1 },
|
||||
"entitlementsSha256": { "$ref": "#/$defs/sha256" }
|
||||
}
|
||||
},
|
||||
"hardware": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["model", "cpu", "memoryBytes"],
|
||||
"properties": {
|
||||
"model": { "type": "string", "minLength": 1 },
|
||||
"cpu": { "type": "string", "minLength": 1 },
|
||||
"memoryBytes": { "type": "integer", "minimum": 1 }
|
||||
}
|
||||
},
|
||||
"platform": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["macosVersion", "webkitVersion"],
|
||||
"properties": {
|
||||
"macosVersion": { "type": "string", "minLength": 1 },
|
||||
"webkitVersion": { "type": "string", "minLength": 1 }
|
||||
}
|
||||
},
|
||||
"display": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["refreshRateHz", "widthPx", "heightPx", "scaleFactor"],
|
||||
"properties": {
|
||||
"refreshRateHz": { "type": "number", "exclusiveMinimum": 0 },
|
||||
"widthPx": { "type": "integer", "minimum": 1 },
|
||||
"heightPx": { "type": "integer", "minimum": 1 },
|
||||
"scaleFactor": { "type": "number", "exclusiveMinimum": 0 }
|
||||
}
|
||||
},
|
||||
"runtimeState": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["powerSource", "lowPowerMode", "thermalState"],
|
||||
"properties": {
|
||||
"powerSource": { "enum": ["ac", "battery"] },
|
||||
"lowPowerMode": { "type": "boolean" },
|
||||
"thermalState": { "enum": ["nominal", "fair", "serious", "critical"] }
|
||||
}
|
||||
},
|
||||
"fixture": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["fixtureId", "version", "sha256"],
|
||||
"properties": {
|
||||
"fixtureId": { "type": "string", "minLength": 1 },
|
||||
"version": { "type": "integer", "minimum": 1 },
|
||||
"sha256": { "$ref": "#/$defs/sha256" }
|
||||
}
|
||||
},
|
||||
"relay": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["version", "fixtureVersion"],
|
||||
"properties": {
|
||||
"version": { "type": "string", "minLength": 1 },
|
||||
"fixtureVersion": { "type": "string", "minLength": 1 }
|
||||
}
|
||||
},
|
||||
"nativeDriver": {
|
||||
"oneOf": [
|
||||
{ "type": "null" },
|
||||
{
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["name", "version", "sha256"],
|
||||
"properties": {
|
||||
"name": { "type": "string", "minLength": 1 },
|
||||
"version": { "type": "string", "minLength": 1 },
|
||||
"sha256": { "$ref": "#/$defs/sha256" }
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"$defs": { "sha256": { "type": "string", "pattern": "^[0-9a-f]{64}$" } }
|
||||
}
|
||||
191
desktop/tests/wkwebview/schemas/fixture.schema.json
Normal file
191
desktop/tests/wkwebview/schemas/fixture.schema.json
Normal file
@@ -0,0 +1,191 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "https://buzz.local/schemas/wkwebview/fixture-v1.json",
|
||||
"title": "Buzz packaged WKWebView acceptance fixture",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"fixtureId",
|
||||
"version",
|
||||
"relayFixtureVersion",
|
||||
"materialization",
|
||||
"inputSemantics",
|
||||
"channels",
|
||||
"scenarioTaxonomy",
|
||||
"correctnessCeilings"
|
||||
],
|
||||
"properties": {
|
||||
"fixtureId": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"version": {
|
||||
"type": "integer",
|
||||
"const": 1
|
||||
},
|
||||
"relayFixtureVersion": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"materialization": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"generator",
|
||||
"generatorVersion",
|
||||
"algorithm",
|
||||
"seedHex",
|
||||
"referenceTime",
|
||||
"signerPrivateKeyHex",
|
||||
"signatureContract",
|
||||
"eventKind",
|
||||
"tagContract",
|
||||
"contentContract",
|
||||
"eventIdContract",
|
||||
"orderingContract",
|
||||
"createdAtStepSeconds",
|
||||
"outputEventCount",
|
||||
"outputBytes",
|
||||
"outputSha256"
|
||||
],
|
||||
"properties": {
|
||||
"generator": {
|
||||
"const": "buzz-wkwebview-fixture"
|
||||
},
|
||||
"generatorVersion": {
|
||||
"const": "1.0.0"
|
||||
},
|
||||
"algorithm": {
|
||||
"const": "nostr-canonical-sha256-v1"
|
||||
},
|
||||
"seedHex": {
|
||||
"type": "string",
|
||||
"pattern": "^[0-9a-f]+$"
|
||||
},
|
||||
"referenceTime": {
|
||||
"type": "string",
|
||||
"format": "date-time"
|
||||
},
|
||||
"signerPrivateKeyHex": {
|
||||
"type": "string",
|
||||
"pattern": "^[0-9a-f]{64}$"
|
||||
},
|
||||
"signatureContract": {
|
||||
"const": "BIP340 with 32 zero aux-rand bytes"
|
||||
},
|
||||
"eventKind": {
|
||||
"const": 9
|
||||
},
|
||||
"tagContract": {
|
||||
"const": "[[\"h\",channelId],[\"fixture\",fixtureId],[\"ordinal\",zeroBasedOrdinal decimal]]"
|
||||
},
|
||||
"contentContract": {
|
||||
"const": "UTF-8 fixture:{contentProfile}:{seedHex}:{zeroBasedOrdinal decimal}"
|
||||
},
|
||||
"eventIdContract": {
|
||||
"const": "NIP-01 sha256 canonical [0,pubkey,created_at,kind,tags,content]"
|
||||
},
|
||||
"orderingContract": {
|
||||
"const": "created_at-ascending-then-event-id-ascending"
|
||||
},
|
||||
"createdAtStepSeconds": {
|
||||
"type": "integer",
|
||||
"minimum": 1
|
||||
},
|
||||
"outputEventCount": {
|
||||
"type": "integer",
|
||||
"minimum": 1
|
||||
},
|
||||
"outputBytes": {
|
||||
"type": "integer",
|
||||
"minimum": 1
|
||||
},
|
||||
"outputSha256": {
|
||||
"type": "string",
|
||||
"pattern": "^[0-9a-f]{64}$"
|
||||
}
|
||||
}
|
||||
},
|
||||
"inputSemantics": {
|
||||
"const": "native-wheel-trackpad-keyboard"
|
||||
},
|
||||
"channels": {
|
||||
"type": "array",
|
||||
"minItems": 3,
|
||||
"items": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["id", "messageCount", "contentProfile"],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"messageCount": {
|
||||
"type": "integer",
|
||||
"minimum": 1
|
||||
},
|
||||
"contentProfile": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"scenarioTaxonomy": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"switch",
|
||||
"input",
|
||||
"timeline",
|
||||
"plateau",
|
||||
"ipc",
|
||||
"recovery",
|
||||
"crash"
|
||||
],
|
||||
"properties": {
|
||||
"switch": {
|
||||
"$ref": "#/$defs/scenarios"
|
||||
},
|
||||
"input": {
|
||||
"$ref": "#/$defs/scenarios"
|
||||
},
|
||||
"timeline": {
|
||||
"$ref": "#/$defs/scenarios"
|
||||
},
|
||||
"plateau": {
|
||||
"$ref": "#/$defs/scenarios"
|
||||
},
|
||||
"ipc": {
|
||||
"$ref": "#/$defs/scenarios"
|
||||
},
|
||||
"recovery": {
|
||||
"$ref": "#/$defs/scenarios"
|
||||
},
|
||||
"crash": {
|
||||
"$ref": "#/$defs/scenarios"
|
||||
}
|
||||
}
|
||||
},
|
||||
"correctnessCeilings": {
|
||||
"type": "object",
|
||||
"additionalProperties": {
|
||||
"type": "number",
|
||||
"minimum": 0
|
||||
},
|
||||
"minProperties": 1
|
||||
}
|
||||
},
|
||||
"$defs": {
|
||||
"scenarios": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"uniqueItems": true,
|
||||
"items": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
67
desktop/tests/wkwebview/schemas/result.schema.json
Normal file
67
desktop/tests/wkwebview/schemas/result.schema.json
Normal file
@@ -0,0 +1,67 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "https://buzz.local/schemas/wkwebview/result-v1.json",
|
||||
"title": "Buzz packaged WKWebView acceptance result",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"schemaVersion",
|
||||
"runId",
|
||||
"environmentSha256",
|
||||
"environment",
|
||||
"fixture",
|
||||
"samples"
|
||||
],
|
||||
"properties": {
|
||||
"schemaVersion": { "type": "integer", "const": 1 },
|
||||
"runId": { "type": "string", "minLength": 1 },
|
||||
"environmentSha256": { "type": "string", "pattern": "^[0-9a-f]{64}$" },
|
||||
"environment": { "$ref": "environment.schema.json" },
|
||||
"fixture": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["fixtureId", "version", "sha256"],
|
||||
"properties": {
|
||||
"fixtureId": { "type": "string", "minLength": 1 },
|
||||
"version": { "type": "integer", "minimum": 1 },
|
||||
"sha256": { "type": "string", "pattern": "^[0-9a-f]{64}$" }
|
||||
}
|
||||
},
|
||||
"samples": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"items": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"sampleId",
|
||||
"category",
|
||||
"scenario",
|
||||
"metric",
|
||||
"unit",
|
||||
"value",
|
||||
"observedAt"
|
||||
],
|
||||
"properties": {
|
||||
"sampleId": { "type": "string", "minLength": 1 },
|
||||
"category": {
|
||||
"enum": [
|
||||
"switch",
|
||||
"input",
|
||||
"timeline",
|
||||
"plateau",
|
||||
"ipc",
|
||||
"recovery",
|
||||
"crash"
|
||||
]
|
||||
},
|
||||
"scenario": { "type": "string", "minLength": 1 },
|
||||
"metric": { "type": "string", "minLength": 1 },
|
||||
"unit": { "enum": ["ms", "px", "bytes", "count", "ratio"] },
|
||||
"value": { "type": "number" },
|
||||
"observedAt": { "type": "string", "format": "date-time" }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
50
desktop/tests/wkwebview/schemas/run.schema.json
Normal file
50
desktop/tests/wkwebview/schemas/run.schema.json
Normal file
@@ -0,0 +1,50 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "https://buzz.local/schemas/wkwebview/run-v1.json",
|
||||
"title": "Buzz packaged WKWebView acceptance run",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"schemaVersion",
|
||||
"runId",
|
||||
"createdAt",
|
||||
"status",
|
||||
"appPath",
|
||||
"fixtureManifestPath",
|
||||
"environmentPath",
|
||||
"environmentSha256",
|
||||
"resultPath"
|
||||
],
|
||||
"properties": {
|
||||
"schemaVersion": { "type": "integer", "const": 1 },
|
||||
"runId": { "type": "string", "minLength": 1 },
|
||||
"createdAt": { "type": "string", "format": "date-time" },
|
||||
"status": { "enum": ["blocked", "running", "completed", "failed"] },
|
||||
"blockedReason": { "enum": ["native-driver-not-supplied"] },
|
||||
"failureReason": { "type": "string", "minLength": 1 },
|
||||
"appPath": { "type": "string", "pattern": "\\.app$" },
|
||||
"fixtureManifestPath": { "type": "string", "minLength": 1 },
|
||||
"environmentPath": { "type": "string", "minLength": 1 },
|
||||
"environmentSha256": { "type": "string", "pattern": "^[0-9a-f]{64}$" },
|
||||
"resultPath": { "type": "string", "minLength": 1 },
|
||||
"nativeDriverPath": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"allOf": [
|
||||
{
|
||||
"if": { "properties": { "status": { "const": "blocked" } } },
|
||||
"then": { "required": ["blockedReason"] }
|
||||
},
|
||||
{
|
||||
"if": {
|
||||
"properties": {
|
||||
"status": { "enum": ["running", "completed", "failed"] }
|
||||
}
|
||||
},
|
||||
"then": { "required": ["nativeDriverPath"] }
|
||||
},
|
||||
{
|
||||
"if": { "properties": { "status": { "const": "failed" } } },
|
||||
"then": { "required": ["failureReason"] }
|
||||
}
|
||||
]
|
||||
}
|
||||
286
desktop/tests/wkwebview/tooling/capture-environment.mjs
Normal file
286
desktop/tests/wkwebview/tooling/capture-environment.mjs
Normal file
@@ -0,0 +1,286 @@
|
||||
#!/usr/bin/env node
|
||||
import { createHash } from "node:crypto";
|
||||
import { execFile } from "node:child_process";
|
||||
import { readFile, writeFile } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { loadVerifiedFixture, validateEnvironment } from "./validation.mjs";
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
const TOOL_ROOT = path.resolve(
|
||||
path.dirname(fileURLToPath(import.meta.url)),
|
||||
"..",
|
||||
);
|
||||
|
||||
function sha256(bytes) {
|
||||
return createHash("sha256").update(bytes).digest("hex");
|
||||
}
|
||||
|
||||
async function runCommand(command, args) {
|
||||
const { stdout, stderr } = await execFileAsync(command, args, {
|
||||
encoding: "utf8",
|
||||
maxBuffer: 10 * 1024 * 1024,
|
||||
});
|
||||
return { stdout, stderr };
|
||||
}
|
||||
|
||||
function requiredLine(value, label) {
|
||||
const result = value.trim();
|
||||
if (!result) throw new Error(`Capture unavailable: ${label}`);
|
||||
return result;
|
||||
}
|
||||
|
||||
function parseDisplay(profileJson) {
|
||||
const profile = JSON.parse(profileJson);
|
||||
const displays = profile.SPDisplaysDataType?.flatMap(
|
||||
(gpu) => gpu.spdisplays_ndrvs ?? [],
|
||||
);
|
||||
const display = displays?.find(
|
||||
(candidate) => candidate.spdisplays_main === "spdisplays_yes",
|
||||
);
|
||||
if (!display) throw new Error("Capture unavailable: main display");
|
||||
const pixels = display._spdisplays_pixels?.match(/^(\d+) x (\d+)$/);
|
||||
const resolution = display._spdisplays_resolution?.match(
|
||||
/^(\d+) x (\d+) @ ([0-9.]+)Hz$/,
|
||||
);
|
||||
if (!pixels || !resolution) {
|
||||
throw new Error("Capture unavailable: main display geometry/refresh rate");
|
||||
}
|
||||
const scaleFactor = Number(pixels[1]) / Number(resolution[1]);
|
||||
if (!Number.isFinite(scaleFactor) || scaleFactor <= 0) {
|
||||
throw new Error("Capture unavailable: main display scale factor");
|
||||
}
|
||||
return {
|
||||
refreshRateHz: Number(resolution[3]),
|
||||
widthPx: Number(pixels[1]),
|
||||
heightPx: Number(pixels[2]),
|
||||
scaleFactor,
|
||||
};
|
||||
}
|
||||
|
||||
function parseSigningIdentity(stderr) {
|
||||
const lines = stderr.split("\n");
|
||||
const authorities = lines
|
||||
.filter((line) => line.startsWith("Authority="))
|
||||
.map((line) => line.slice("Authority=".length));
|
||||
if (authorities.length > 0) return authorities.join(" > ");
|
||||
const signature = lines.find((line) => line.startsWith("Signature="));
|
||||
if (signature) return signature.slice("Signature=".length);
|
||||
throw new Error("Capture unavailable: package signing identity");
|
||||
}
|
||||
|
||||
function parsePowerState(batteryOutput, customOutput) {
|
||||
const powerSource = batteryOutput.includes("AC Power") ? "ac" : "battery";
|
||||
const section = powerSource === "ac" ? "AC Power:" : "Battery Power:";
|
||||
const sectionMatch = customOutput.match(
|
||||
new RegExp(
|
||||
`(?:^|\\n)${section.replace(":", "\\:")}\\n([\\s\\S]*?)(?=\\n\\S[^\\n]*:\\n|$)`,
|
||||
),
|
||||
);
|
||||
const selected = sectionMatch?.[1] ?? "";
|
||||
const powerMode = selected.match(/^\s*powermode\s+(\d+)\s*$/m)?.[1];
|
||||
if (powerMode === undefined) {
|
||||
throw new Error("Capture unavailable: low-power mode");
|
||||
}
|
||||
return { powerSource, lowPowerMode: powerMode !== "0" };
|
||||
}
|
||||
|
||||
function parseThermalState(output) {
|
||||
if (output.includes("No thermal warning level has been recorded")) {
|
||||
return "nominal";
|
||||
}
|
||||
for (const state of ["critical", "serious", "fair", "nominal"]) {
|
||||
if (output.toLowerCase().includes(state)) return state;
|
||||
}
|
||||
throw new Error("Capture unavailable: thermal state");
|
||||
}
|
||||
|
||||
export async function captureEnvironment({
|
||||
appPath,
|
||||
gitSha,
|
||||
relayVersion,
|
||||
nativeDriverPath,
|
||||
nativeDriverName,
|
||||
nativeDriverVersion,
|
||||
manifestPath = path.join(TOOL_ROOT, "fixture-manifest.json"),
|
||||
commandRunner = runCommand,
|
||||
capturedAt = new Date().toISOString(),
|
||||
}) {
|
||||
const resolvedAppPath = path.resolve(appPath);
|
||||
const infoPlistPath = path.join(resolvedAppPath, "Contents", "Info.plist");
|
||||
const infoPlistBytes = await readFile(infoPlistPath);
|
||||
const driverBytes = nativeDriverPath
|
||||
? await readFile(nativeDriverPath)
|
||||
: null;
|
||||
if (
|
||||
[nativeDriverPath, nativeDriverName, nativeDriverVersion].filter(Boolean)
|
||||
.length !== 0 &&
|
||||
[nativeDriverPath, nativeDriverName, nativeDriverVersion].some(
|
||||
(value) => !value,
|
||||
)
|
||||
) {
|
||||
throw new Error("Capture requires driver path, name, and version together");
|
||||
}
|
||||
const command = async (program, args, label, stream = "stdout") =>
|
||||
requiredLine((await commandRunner(program, args))[stream], label);
|
||||
const bundleIdentifier = await command(
|
||||
"plutil",
|
||||
["-extract", "CFBundleIdentifier", "raw", "-o", "-", infoPlistPath],
|
||||
"bundle identifier",
|
||||
);
|
||||
const version = await command(
|
||||
"plutil",
|
||||
["-extract", "CFBundleShortVersionString", "raw", "-o", "-", infoPlistPath],
|
||||
"bundle version",
|
||||
);
|
||||
const signing = await commandRunner("codesign", [
|
||||
"-dv",
|
||||
"--verbose=4",
|
||||
resolvedAppPath,
|
||||
]);
|
||||
const entitlements = await commandRunner("codesign", [
|
||||
"-d",
|
||||
"--entitlements",
|
||||
":-",
|
||||
resolvedAppPath,
|
||||
]);
|
||||
const entitlementsBytes = Buffer.from(
|
||||
requiredLine(entitlements.stdout, "package entitlements"),
|
||||
);
|
||||
const model = await command("sysctl", ["-n", "hw.model"], "hardware model");
|
||||
const cpu = await command(
|
||||
"sysctl",
|
||||
["-n", "machdep.cpu.brand_string"],
|
||||
"CPU model",
|
||||
);
|
||||
const memoryBytes = Number(
|
||||
await command("sysctl", ["-n", "hw.memsize"], "physical memory"),
|
||||
);
|
||||
const macosVersion = await command(
|
||||
"sw_vers",
|
||||
["-productVersion"],
|
||||
"macOS version",
|
||||
);
|
||||
const webkitVersion = await command(
|
||||
"defaults",
|
||||
[
|
||||
"read",
|
||||
"/System/Library/Frameworks/WebKit.framework/Resources/Info",
|
||||
"CFBundleShortVersionString",
|
||||
],
|
||||
"WebKit version",
|
||||
);
|
||||
const display = parseDisplay(
|
||||
await command(
|
||||
"system_profiler",
|
||||
["SPDisplaysDataType", "-json"],
|
||||
"display profile",
|
||||
),
|
||||
);
|
||||
const battery = await commandRunner("pmset", ["-g", "batt"]);
|
||||
const custom = await commandRunner("pmset", ["-g", "custom"]);
|
||||
const thermal = await commandRunner("pmset", ["-g", "therm"]);
|
||||
const power = parsePowerState(battery.stdout, custom.stdout);
|
||||
const { fixture, entry } = await loadVerifiedFixture(
|
||||
manifestPath,
|
||||
"buzz-wkwebview-acceptance",
|
||||
);
|
||||
if (entry.version !== 1) {
|
||||
throw new Error("Captured fixture version is not supported");
|
||||
}
|
||||
|
||||
const environment = {
|
||||
schemaVersion: 1,
|
||||
capture: {
|
||||
tool: "buzz-wkwebview-environment-capture",
|
||||
version: "1.0.0",
|
||||
capturedAt,
|
||||
infoPlistSha256: sha256(infoPlistBytes),
|
||||
},
|
||||
build: { gitSha },
|
||||
package: {
|
||||
appPath: resolvedAppPath,
|
||||
bundleIdentifier,
|
||||
version,
|
||||
identity: parseSigningIdentity(signing.stderr),
|
||||
entitlementsSha256: sha256(entitlementsBytes),
|
||||
},
|
||||
hardware: { model, cpu, memoryBytes },
|
||||
platform: { macosVersion, webkitVersion },
|
||||
display,
|
||||
runtimeState: {
|
||||
...power,
|
||||
thermalState: parseThermalState(thermal.stdout),
|
||||
},
|
||||
fixture: {
|
||||
fixtureId: entry.fixtureId,
|
||||
version: entry.version,
|
||||
sha256: entry.sha256,
|
||||
},
|
||||
relay: {
|
||||
version: relayVersion,
|
||||
fixtureVersion: fixture.relayFixtureVersion,
|
||||
},
|
||||
nativeDriver:
|
||||
driverBytes === null
|
||||
? null
|
||||
: {
|
||||
name: nativeDriverName,
|
||||
version: nativeDriverVersion,
|
||||
sha256: sha256(driverBytes),
|
||||
},
|
||||
};
|
||||
const errors = validateEnvironment(environment);
|
||||
if (errors.length > 0) {
|
||||
throw new Error(
|
||||
`Captured environment is invalid:\n- ${errors.join("\n- ")}`,
|
||||
);
|
||||
}
|
||||
return environment;
|
||||
}
|
||||
|
||||
function parseArgs(args) {
|
||||
const options = {};
|
||||
for (let index = 0; index < args.length; index += 2) {
|
||||
const name = args[index];
|
||||
const value = args[index + 1];
|
||||
if (!name?.startsWith("--") || value === undefined) {
|
||||
throw new Error(`Invalid argument ${name}`);
|
||||
}
|
||||
options[name.slice(2)] = value;
|
||||
}
|
||||
return options;
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const options = parseArgs(process.argv.slice(2));
|
||||
const required = ["app", "git-sha", "relay-version", "output"];
|
||||
if (required.some((key) => !options[key])) {
|
||||
console.error(
|
||||
"Usage: node tests/wkwebview/tooling/capture-environment.mjs --app <Buzz.app> --git-sha <sha> --relay-version <version> --output <environment.json> [--driver <path> --driver-name <name> --driver-version <version>] [--manifest <manifest.json>]",
|
||||
);
|
||||
process.exitCode = 1;
|
||||
return;
|
||||
}
|
||||
const environment = await captureEnvironment({
|
||||
appPath: options.app,
|
||||
gitSha: options["git-sha"],
|
||||
relayVersion: options["relay-version"],
|
||||
nativeDriverPath: options.driver,
|
||||
nativeDriverName: options["driver-name"],
|
||||
nativeDriverVersion: options["driver-version"],
|
||||
manifestPath: options.manifest,
|
||||
});
|
||||
await writeFile(options.output, `${JSON.stringify(environment, null, 2)}\n`, {
|
||||
flag: "wx",
|
||||
});
|
||||
console.log(path.resolve(options.output));
|
||||
}
|
||||
|
||||
if (
|
||||
process.argv[1] &&
|
||||
path.resolve(process.argv[1]) === fileURLToPath(import.meta.url)
|
||||
) {
|
||||
await main();
|
||||
}
|
||||
247
desktop/tests/wkwebview/tooling/launch.mjs
Executable file
247
desktop/tests/wkwebview/tooling/launch.mjs
Executable file
@@ -0,0 +1,247 @@
|
||||
#!/usr/bin/env node
|
||||
import { createHash } from "node:crypto";
|
||||
import { access, mkdir, readFile, writeFile } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
import { spawn } from "node:child_process";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import {
|
||||
loadVerifiedFixture,
|
||||
validateEnvironment,
|
||||
validateResult,
|
||||
} from "./validation.mjs";
|
||||
|
||||
const TOOL_ROOT = path.resolve(
|
||||
path.dirname(fileURLToPath(import.meta.url)),
|
||||
"..",
|
||||
);
|
||||
|
||||
function makeRun({
|
||||
appPath,
|
||||
environmentPath,
|
||||
manifestPath,
|
||||
resultPath,
|
||||
nativeDriverPath,
|
||||
environmentSha256,
|
||||
}) {
|
||||
const now = new Date();
|
||||
return {
|
||||
schemaVersion: 1,
|
||||
runId: `wkwebview-${now.toISOString().replaceAll(/[:.]/g, "-")}`,
|
||||
createdAt: now.toISOString(),
|
||||
status: nativeDriverPath ? "running" : "blocked",
|
||||
...(!nativeDriverPath && { blockedReason: "native-driver-not-supplied" }),
|
||||
appPath: path.resolve(appPath),
|
||||
fixtureManifestPath: path.resolve(manifestPath),
|
||||
environmentPath: path.resolve(environmentPath),
|
||||
environmentSha256,
|
||||
resultPath: path.resolve(resultPath),
|
||||
...(nativeDriverPath && {
|
||||
nativeDriverPath: path.resolve(nativeDriverPath),
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
||||
async function writeRun(runPath, run, exclusive = false) {
|
||||
await mkdir(path.dirname(runPath), { recursive: true });
|
||||
await writeFile(runPath, `${JSON.stringify(run, null, 2)}\n`, {
|
||||
...(exclusive && { flag: "wx" }),
|
||||
});
|
||||
}
|
||||
|
||||
async function sha256File(filePath) {
|
||||
const bytes = await readFile(filePath);
|
||||
return createHash("sha256").update(bytes).digest("hex");
|
||||
}
|
||||
|
||||
async function assertPathDoesNotExist(filePath, label) {
|
||||
try {
|
||||
await access(filePath);
|
||||
} catch (error) {
|
||||
if (error?.code === "ENOENT") return;
|
||||
throw error;
|
||||
}
|
||||
throw new Error(`${label} already exists: ${path.resolve(filePath)}`);
|
||||
}
|
||||
|
||||
function invokeNativeDriver(driverPath, runPath) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const child = spawn(driverPath, ["--wkwebview-run", runPath], {
|
||||
stdio: "inherit",
|
||||
env: { ...process.env, BUZZ_WKWEBVIEW_RUN: runPath },
|
||||
});
|
||||
child.once("error", reject);
|
||||
child.once("exit", (code, signal) => {
|
||||
if (signal) reject(new Error(`Native driver terminated by ${signal}`));
|
||||
else resolve(code ?? 1);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
export async function launchHarness({
|
||||
appPath,
|
||||
environmentPath,
|
||||
runPath,
|
||||
resultPath,
|
||||
nativeDriverPath,
|
||||
manifestPath = path.join(TOOL_ROOT, "fixture-manifest.json"),
|
||||
}) {
|
||||
if (!appPath.endsWith(".app"))
|
||||
throw new Error("--app must identify a packaged .app bundle");
|
||||
await access(path.join(appPath, "Contents", "MacOS"));
|
||||
const environmentBytes = await readFile(environmentPath);
|
||||
const environmentSha256 = createHash("sha256")
|
||||
.update(environmentBytes)
|
||||
.digest("hex");
|
||||
let environment;
|
||||
try {
|
||||
environment = JSON.parse(environmentBytes.toString("utf8"));
|
||||
} catch (error) {
|
||||
throw new Error(
|
||||
`Cannot read JSON from ${environmentPath}: ${error.message}`,
|
||||
);
|
||||
}
|
||||
const environmentErrors = validateEnvironment(environment);
|
||||
const infoPlistSha256 = await sha256File(
|
||||
path.join(appPath, "Contents", "Info.plist"),
|
||||
);
|
||||
if (environment.capture?.infoPlistSha256 !== infoPlistSha256) {
|
||||
environmentErrors.push(
|
||||
"environment.capture.infoPlistSha256 does not match the supplied .app",
|
||||
);
|
||||
}
|
||||
if (nativeDriverPath && environment.nativeDriver === null) {
|
||||
environmentErrors.push(
|
||||
"environment.nativeDriver provenance is required when --driver is supplied",
|
||||
);
|
||||
}
|
||||
if (!nativeDriverPath && environment.nativeDriver !== null) {
|
||||
environmentErrors.push(
|
||||
"environment.nativeDriver must be null when no --driver is supplied",
|
||||
);
|
||||
}
|
||||
if (environmentErrors.length > 0) {
|
||||
throw new Error(
|
||||
`Invalid WKWebView environment:\n- ${environmentErrors.join("\n- ")}`,
|
||||
);
|
||||
}
|
||||
if (path.resolve(environment.package.appPath) !== path.resolve(appPath)) {
|
||||
throw new Error("Environment package.appPath does not match --app");
|
||||
}
|
||||
if (nativeDriverPath) {
|
||||
const nativeDriverSha256 = await sha256File(nativeDriverPath);
|
||||
if (nativeDriverSha256 !== environment.nativeDriver.sha256) {
|
||||
throw new Error(
|
||||
"Native driver SHA-256 does not match environment.nativeDriver.sha256",
|
||||
);
|
||||
}
|
||||
await assertPathDoesNotExist(resultPath, "Raw result path");
|
||||
}
|
||||
const { fixture, entry } = await loadVerifiedFixture(
|
||||
manifestPath,
|
||||
environment.fixture.fixtureId,
|
||||
);
|
||||
if (
|
||||
environment.fixture.version !== entry.version ||
|
||||
environment.fixture.sha256 !== entry.sha256
|
||||
) {
|
||||
throw new Error(
|
||||
"Environment fixture provenance does not match the immutable manifest",
|
||||
);
|
||||
}
|
||||
|
||||
const run = makeRun({
|
||||
appPath,
|
||||
environmentPath,
|
||||
manifestPath,
|
||||
resultPath,
|
||||
nativeDriverPath,
|
||||
environmentSha256,
|
||||
});
|
||||
await writeRun(runPath, run, true);
|
||||
if (!nativeDriverPath) return run;
|
||||
|
||||
try {
|
||||
const exitCode = await invokeNativeDriver(
|
||||
nativeDriverPath,
|
||||
path.resolve(runPath),
|
||||
);
|
||||
if (exitCode !== 0) {
|
||||
throw new Error(`Native driver exited with status ${exitCode}`);
|
||||
}
|
||||
await access(resultPath);
|
||||
const resultBytes = await readFile(resultPath, "utf8");
|
||||
let rawResult;
|
||||
try {
|
||||
rawResult = JSON.parse(resultBytes);
|
||||
} catch (error) {
|
||||
throw new Error(`Cannot read JSON from ${resultPath}: ${error.message}`);
|
||||
}
|
||||
if (rawResult.runId !== run.runId) {
|
||||
throw new Error("Raw result runId does not match the launch run");
|
||||
}
|
||||
if (rawResult.environmentSha256 !== run.environmentSha256) {
|
||||
throw new Error(
|
||||
"Raw result environment digest does not match the launch environment",
|
||||
);
|
||||
}
|
||||
if (JSON.stringify(rawResult.environment) !== JSON.stringify(environment)) {
|
||||
throw new Error(
|
||||
"Raw result environment does not match the launch environment",
|
||||
);
|
||||
}
|
||||
const resultErrors = validateResult(rawResult, fixture, entry);
|
||||
if (resultErrors.length > 0) {
|
||||
throw new Error(
|
||||
`Invalid WKWebView result:\n- ${resultErrors.join("\n- ")}`,
|
||||
);
|
||||
}
|
||||
run.status = "completed";
|
||||
await writeRun(runPath, run);
|
||||
return run;
|
||||
} catch (error) {
|
||||
run.status = "failed";
|
||||
run.failureReason = error instanceof Error ? error.message : String(error);
|
||||
await writeRun(runPath, run);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
function parseArgs(args) {
|
||||
const options = {};
|
||||
for (let index = 0; index < args.length; index += 2) {
|
||||
const name = args[index];
|
||||
const value = args[index + 1];
|
||||
if (!name?.startsWith("--") || value === undefined)
|
||||
throw new Error(`Invalid argument ${name}`);
|
||||
options[name.slice(2)] = value;
|
||||
}
|
||||
return options;
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const options = parseArgs(process.argv.slice(2));
|
||||
if (!options.app || !options.environment || !options.run || !options.result) {
|
||||
console.error(
|
||||
"Usage: node tests/wkwebview/tooling/launch.mjs --app <Buzz.app> --environment <environment.json> --run <run.json> --result <raw-result.json> [--driver <native-driver>] [--manifest <fixture-manifest.json>]",
|
||||
);
|
||||
process.exitCode = 1;
|
||||
return;
|
||||
}
|
||||
const run = await launchHarness({
|
||||
appPath: options.app,
|
||||
environmentPath: options.environment,
|
||||
runPath: options.run,
|
||||
resultPath: options.result,
|
||||
nativeDriverPath: options.driver,
|
||||
manifestPath: options.manifest,
|
||||
});
|
||||
console.log(JSON.stringify(run));
|
||||
if (run.status === "blocked") process.exitCode = 2;
|
||||
}
|
||||
|
||||
if (
|
||||
process.argv[1] &&
|
||||
path.resolve(process.argv[1]) === fileURLToPath(import.meta.url)
|
||||
) {
|
||||
await main();
|
||||
}
|
||||
124
desktop/tests/wkwebview/tooling/materialize-fixture.mjs
Normal file
124
desktop/tests/wkwebview/tooling/materialize-fixture.mjs
Normal file
@@ -0,0 +1,124 @@
|
||||
#!/usr/bin/env node
|
||||
import { createHash } from "node:crypto";
|
||||
import { createRequire } from "node:module";
|
||||
import { writeFile } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { getPublicKey, verifyEvent } from "nostr-tools/pure";
|
||||
import { loadVerifiedFixture } from "./validation.mjs";
|
||||
|
||||
const requireFromNostrTools = createRequire(
|
||||
import.meta.resolve("nostr-tools/pure"),
|
||||
);
|
||||
const { schnorr } = requireFromNostrTools("@noble/curves/secp256k1.js");
|
||||
const TOOL_ROOT = path.resolve(
|
||||
path.dirname(fileURLToPath(import.meta.url)),
|
||||
"..",
|
||||
);
|
||||
|
||||
export function materializeFixture(fixture) {
|
||||
const contract = fixture.materialization;
|
||||
const secretKey = Buffer.from(contract.signerPrivateKeyHex, "hex");
|
||||
const publicKey = getPublicKey(secretKey);
|
||||
const referenceTime = Date.parse(contract.referenceTime) / 1000;
|
||||
if (!Number.isInteger(referenceTime)) {
|
||||
throw new Error(
|
||||
"Fixture referenceTime must resolve to whole epoch seconds",
|
||||
);
|
||||
}
|
||||
|
||||
const events = [];
|
||||
let ordinal = 0;
|
||||
for (const channel of fixture.channels) {
|
||||
for (
|
||||
let channelOrdinal = 0;
|
||||
channelOrdinal < channel.messageCount;
|
||||
channelOrdinal += 1
|
||||
) {
|
||||
const createdAt = referenceTime + ordinal * contract.createdAtStepSeconds;
|
||||
const tags = [
|
||||
["h", channel.id],
|
||||
["fixture", fixture.fixtureId],
|
||||
["ordinal", String(ordinal)],
|
||||
];
|
||||
const content = `fixture:${channel.contentProfile}:${contract.seedHex}:${ordinal}`;
|
||||
const id = createHash("sha256")
|
||||
.update(
|
||||
JSON.stringify([
|
||||
0,
|
||||
publicKey,
|
||||
createdAt,
|
||||
contract.eventKind,
|
||||
tags,
|
||||
content,
|
||||
]),
|
||||
)
|
||||
.digest("hex");
|
||||
const sig = Buffer.from(
|
||||
schnorr.sign(Buffer.from(id, "hex"), secretKey, new Uint8Array(32)),
|
||||
).toString("hex");
|
||||
const event = {
|
||||
id,
|
||||
pubkey: publicKey,
|
||||
created_at: createdAt,
|
||||
kind: contract.eventKind,
|
||||
tags,
|
||||
content,
|
||||
sig,
|
||||
};
|
||||
if (!verifyEvent(event)) {
|
||||
throw new Error(
|
||||
`Materialized event ${ordinal} failed signature verification`,
|
||||
);
|
||||
}
|
||||
events.push(event);
|
||||
ordinal += 1;
|
||||
}
|
||||
}
|
||||
return events.sort(
|
||||
(left, right) =>
|
||||
left.created_at - right.created_at || left.id.localeCompare(right.id),
|
||||
);
|
||||
}
|
||||
|
||||
export function serializeMaterializedEvents(events) {
|
||||
return `${events.map((event) => JSON.stringify(event)).join("\n")}\n`;
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const [
|
||||
outputPath,
|
||||
manifestPath = path.join(TOOL_ROOT, "fixture-manifest.json"),
|
||||
] = process.argv.slice(2);
|
||||
if (!outputPath) {
|
||||
console.error(
|
||||
"Usage: node tests/wkwebview/tooling/materialize-fixture.mjs <events.jsonl> [fixture-manifest.json]",
|
||||
);
|
||||
process.exitCode = 1;
|
||||
return;
|
||||
}
|
||||
const { fixture } = await loadVerifiedFixture(
|
||||
manifestPath,
|
||||
"buzz-wkwebview-acceptance",
|
||||
);
|
||||
const bytes = serializeMaterializedEvents(materializeFixture(fixture));
|
||||
await writeFile(outputPath, bytes, { flag: "wx" });
|
||||
console.log(
|
||||
JSON.stringify({
|
||||
path: path.resolve(outputPath),
|
||||
events: fixture.channels.reduce(
|
||||
(total, channel) => total + channel.messageCount,
|
||||
0,
|
||||
),
|
||||
bytes: Buffer.byteLength(bytes),
|
||||
sha256: createHash("sha256").update(bytes).digest("hex"),
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
if (
|
||||
process.argv[1] &&
|
||||
path.resolve(process.argv[1]) === fileURLToPath(import.meta.url)
|
||||
) {
|
||||
await main();
|
||||
}
|
||||
114
desktop/tests/wkwebview/tooling/report.mjs
Executable file
114
desktop/tests/wkwebview/tooling/report.mjs
Executable file
@@ -0,0 +1,114 @@
|
||||
#!/usr/bin/env node
|
||||
import { createHash } from "node:crypto";
|
||||
import { readFile, writeFile } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import {
|
||||
loadVerifiedFixture,
|
||||
readJson,
|
||||
summarizeSamples,
|
||||
validateRun,
|
||||
validateResult,
|
||||
} from "./validation.mjs";
|
||||
|
||||
const TOOL_ROOT = path.resolve(
|
||||
path.dirname(fileURLToPath(import.meta.url)),
|
||||
"..",
|
||||
);
|
||||
|
||||
export async function buildReport({
|
||||
resultPath,
|
||||
runPath,
|
||||
manifestPath = path.join(TOOL_ROOT, "fixture-manifest.json"),
|
||||
}) {
|
||||
const resultBytes = await readFile(resultPath);
|
||||
let result;
|
||||
try {
|
||||
result = JSON.parse(resultBytes.toString("utf8"));
|
||||
} catch (error) {
|
||||
throw new Error(`Cannot read JSON from ${resultPath}: ${error.message}`);
|
||||
}
|
||||
if (!runPath) throw new Error("A persisted run artifact is required");
|
||||
const run = await readJson(runPath);
|
||||
const runErrors = validateRun(run);
|
||||
if (runErrors.length > 0) {
|
||||
throw new Error(`Invalid WKWebView run:\n- ${runErrors.join("\n- ")}`);
|
||||
}
|
||||
const environmentBytes = await readFile(run.environmentPath);
|
||||
const environmentSha256 = createHash("sha256")
|
||||
.update(environmentBytes)
|
||||
.digest("hex");
|
||||
if (run.status !== "completed")
|
||||
throw new Error("Run artifact is not completed");
|
||||
if (run.resultPath !== path.resolve(resultPath)) {
|
||||
throw new Error("Run artifact does not reference this raw result");
|
||||
}
|
||||
if (run.runId !== result.runId)
|
||||
throw new Error("Result runId does not match run artifact");
|
||||
if (
|
||||
run.environmentSha256 !== environmentSha256 ||
|
||||
result.environmentSha256 !== environmentSha256
|
||||
) {
|
||||
throw new Error("Environment digest does not match launch-time bytes");
|
||||
}
|
||||
const launchEnvironment = JSON.parse(environmentBytes.toString("utf8"));
|
||||
if (
|
||||
JSON.stringify(result.environment) !== JSON.stringify(launchEnvironment)
|
||||
) {
|
||||
throw new Error(
|
||||
"Embedded result environment does not match launch environment",
|
||||
);
|
||||
}
|
||||
const fixtureId = result.fixture?.fixtureId;
|
||||
if (typeof fixtureId !== "string" || fixtureId.length === 0) {
|
||||
throw new Error("Result does not declare fixture.fixtureId");
|
||||
}
|
||||
const { fixture, entry } = await loadVerifiedFixture(manifestPath, fixtureId);
|
||||
const errors = validateResult(result, fixture, entry);
|
||||
if (errors.length > 0) {
|
||||
throw new Error(`Invalid WKWebView result:\n- ${errors.join("\n- ")}`);
|
||||
}
|
||||
return {
|
||||
schemaVersion: 1,
|
||||
runId: result.runId,
|
||||
generatedAt: new Date().toISOString(),
|
||||
provenance: {
|
||||
rawResultPath: path.resolve(resultPath),
|
||||
rawResultSha256: createHash("sha256").update(resultBytes).digest("hex"),
|
||||
buildGitSha: result.environment.build.gitSha,
|
||||
package: result.environment.package,
|
||||
hardware: result.environment.hardware,
|
||||
platform: result.environment.platform,
|
||||
display: result.environment.display,
|
||||
runtimeState: result.environment.runtimeState,
|
||||
relay: result.environment.relay,
|
||||
nativeDriver: result.environment.nativeDriver,
|
||||
fixture: entry,
|
||||
},
|
||||
percentileMethod: "nearest-rank",
|
||||
summaries: summarizeSamples(result.samples),
|
||||
};
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const [resultPath, runPath, outputPath, manifestPath] = process.argv.slice(2);
|
||||
if (!resultPath || !runPath || !outputPath) {
|
||||
console.error(
|
||||
"Usage: node tests/wkwebview/tooling/report.mjs <raw-result.json> <run.json> <report.json> [fixture-manifest.json]",
|
||||
);
|
||||
process.exitCode = 1;
|
||||
return;
|
||||
}
|
||||
const report = await buildReport({ resultPath, runPath, manifestPath });
|
||||
await writeFile(outputPath, `${JSON.stringify(report, null, 2)}\n`, {
|
||||
flag: "wx",
|
||||
});
|
||||
console.log(outputPath);
|
||||
}
|
||||
|
||||
if (
|
||||
process.argv[1] &&
|
||||
path.resolve(process.argv[1]) === fileURLToPath(import.meta.url)
|
||||
) {
|
||||
await main();
|
||||
}
|
||||
746
desktop/tests/wkwebview/tooling/validation.mjs
Normal file
746
desktop/tests/wkwebview/tooling/validation.mjs
Normal file
@@ -0,0 +1,746 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { readFile } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
|
||||
export const SCENARIO_CATEGORIES = [
|
||||
"switch",
|
||||
"input",
|
||||
"timeline",
|
||||
"plateau",
|
||||
"ipc",
|
||||
"recovery",
|
||||
"crash",
|
||||
];
|
||||
|
||||
const SAMPLE_UNITS = new Set(["ms", "px", "bytes", "count", "ratio"]);
|
||||
const SHA_256 = /^[0-9a-f]{64}$/;
|
||||
const GIT_SHA = /^[0-9a-f]{40}$/;
|
||||
|
||||
function isRecord(value) {
|
||||
return value !== null && typeof value === "object" && !Array.isArray(value);
|
||||
}
|
||||
|
||||
function requireRecord(value, label, errors) {
|
||||
if (!isRecord(value)) {
|
||||
errors.push(`${label} must be an object`);
|
||||
return {};
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
function requireNonEmptyString(value, label, errors) {
|
||||
if (typeof value !== "string" || value.length === 0) {
|
||||
errors.push(`${label} must be a non-empty string`);
|
||||
}
|
||||
}
|
||||
|
||||
function requirePositiveNumber(value, label, errors) {
|
||||
if (typeof value !== "number" || !Number.isFinite(value) || value <= 0) {
|
||||
errors.push(`${label} must be a positive finite number`);
|
||||
}
|
||||
}
|
||||
|
||||
function requireInteger(value, label, errors, minimum = 0) {
|
||||
if (!Number.isInteger(value) || value < minimum) {
|
||||
errors.push(`${label} must be an integer >= ${minimum}`);
|
||||
}
|
||||
}
|
||||
|
||||
function requireTimestamp(value, label, errors) {
|
||||
requireNonEmptyString(value, label, errors);
|
||||
if (typeof value === "string" && Number.isNaN(Date.parse(value))) {
|
||||
errors.push(`${label} must be an ISO-8601 timestamp`);
|
||||
}
|
||||
}
|
||||
|
||||
function requireSha(value, label, errors, pattern = SHA_256) {
|
||||
if (typeof value !== "string" || !pattern.test(value)) {
|
||||
errors.push(`${label} has an invalid digest`);
|
||||
}
|
||||
}
|
||||
|
||||
function rejectUnknownKeys(value, allowed, label, errors) {
|
||||
if (!isRecord(value)) return;
|
||||
for (const key of Object.keys(value)) {
|
||||
if (!allowed.has(key)) errors.push(`${label}.${key} is not allowed`);
|
||||
}
|
||||
}
|
||||
|
||||
export function validateFixture(fixture) {
|
||||
const errors = [];
|
||||
const root = requireRecord(fixture, "fixture", errors);
|
||||
rejectUnknownKeys(
|
||||
root,
|
||||
new Set([
|
||||
"fixtureId",
|
||||
"version",
|
||||
"relayFixtureVersion",
|
||||
"materialization",
|
||||
"inputSemantics",
|
||||
"channels",
|
||||
"scenarioTaxonomy",
|
||||
"correctnessCeilings",
|
||||
]),
|
||||
"fixture",
|
||||
errors,
|
||||
);
|
||||
requireNonEmptyString(root.fixtureId, "fixture.fixtureId", errors);
|
||||
if (root.version !== 1) errors.push("fixture.version must equal 1");
|
||||
requireNonEmptyString(
|
||||
root.relayFixtureVersion,
|
||||
"fixture.relayFixtureVersion",
|
||||
errors,
|
||||
);
|
||||
const materialization = requireRecord(
|
||||
root.materialization,
|
||||
"fixture.materialization",
|
||||
errors,
|
||||
);
|
||||
const materializationKeys = [
|
||||
"generator",
|
||||
"generatorVersion",
|
||||
"algorithm",
|
||||
"seedHex",
|
||||
"referenceTime",
|
||||
"signerPrivateKeyHex",
|
||||
"signatureContract",
|
||||
"eventKind",
|
||||
"tagContract",
|
||||
"contentContract",
|
||||
"eventIdContract",
|
||||
"orderingContract",
|
||||
"createdAtStepSeconds",
|
||||
"outputEventCount",
|
||||
"outputBytes",
|
||||
"outputSha256",
|
||||
];
|
||||
rejectUnknownKeys(
|
||||
materialization,
|
||||
new Set(materializationKeys),
|
||||
"fixture.materialization",
|
||||
errors,
|
||||
);
|
||||
const expectedMaterialization = {
|
||||
generator: "buzz-wkwebview-fixture",
|
||||
generatorVersion: "1.0.0",
|
||||
algorithm: "nostr-canonical-sha256-v1",
|
||||
signatureContract: "BIP340 with 32 zero aux-rand bytes",
|
||||
eventKind: 9,
|
||||
tagContract:
|
||||
'[["h",channelId],["fixture",fixtureId],["ordinal",zeroBasedOrdinal decimal]]',
|
||||
contentContract:
|
||||
"UTF-8 fixture:{contentProfile}:{seedHex}:{zeroBasedOrdinal decimal}",
|
||||
eventIdContract:
|
||||
"NIP-01 sha256 canonical [0,pubkey,created_at,kind,tags,content]",
|
||||
orderingContract: "created_at-ascending-then-event-id-ascending",
|
||||
};
|
||||
for (const [key, value] of Object.entries(expectedMaterialization)) {
|
||||
if (materialization[key] !== value) {
|
||||
errors.push(`fixture.materialization.${key} has an unsupported contract`);
|
||||
}
|
||||
}
|
||||
if (
|
||||
typeof materialization.seedHex !== "string" ||
|
||||
!/^[0-9a-f]+$/.test(materialization.seedHex) ||
|
||||
materialization.seedHex.length % 2 !== 0
|
||||
) {
|
||||
errors.push(
|
||||
"fixture.materialization.seedHex must be even-length lowercase hex",
|
||||
);
|
||||
}
|
||||
if (
|
||||
typeof materialization.signerPrivateKeyHex !== "string" ||
|
||||
!/^[0-9a-f]{64}$/.test(materialization.signerPrivateKeyHex)
|
||||
) {
|
||||
errors.push(
|
||||
"fixture.materialization.signerPrivateKeyHex must be 64 lowercase hex characters",
|
||||
);
|
||||
}
|
||||
requireTimestamp(
|
||||
materialization.referenceTime,
|
||||
"fixture.materialization.referenceTime",
|
||||
errors,
|
||||
);
|
||||
requireInteger(
|
||||
materialization.createdAtStepSeconds,
|
||||
"fixture.materialization.createdAtStepSeconds",
|
||||
errors,
|
||||
1,
|
||||
);
|
||||
requireInteger(
|
||||
materialization.outputEventCount,
|
||||
"fixture.materialization.outputEventCount",
|
||||
errors,
|
||||
1,
|
||||
);
|
||||
requireInteger(
|
||||
materialization.outputBytes,
|
||||
"fixture.materialization.outputBytes",
|
||||
errors,
|
||||
1,
|
||||
);
|
||||
requireSha(
|
||||
materialization.outputSha256,
|
||||
"fixture.materialization.outputSha256",
|
||||
errors,
|
||||
);
|
||||
if (root.inputSemantics !== "native-wheel-trackpad-keyboard") {
|
||||
errors.push("fixture.inputSemantics must require native input");
|
||||
}
|
||||
if (!Array.isArray(root.channels) || root.channels.length < 3) {
|
||||
errors.push("fixture.channels must contain at least three channels");
|
||||
} else {
|
||||
for (const [index, channelValue] of root.channels.entries()) {
|
||||
const label = `fixture.channels[${index}]`;
|
||||
const channel = requireRecord(channelValue, label, errors);
|
||||
rejectUnknownKeys(
|
||||
channel,
|
||||
new Set(["id", "messageCount", "contentProfile"]),
|
||||
label,
|
||||
errors,
|
||||
);
|
||||
requireNonEmptyString(channel.id, `${label}.id`, errors);
|
||||
requireInteger(channel.messageCount, `${label}.messageCount`, errors, 1);
|
||||
requireNonEmptyString(
|
||||
channel.contentProfile,
|
||||
`${label}.contentProfile`,
|
||||
errors,
|
||||
);
|
||||
}
|
||||
}
|
||||
const taxonomy = requireRecord(
|
||||
root.scenarioTaxonomy,
|
||||
"fixture.scenarioTaxonomy",
|
||||
errors,
|
||||
);
|
||||
rejectUnknownKeys(
|
||||
taxonomy,
|
||||
new Set(SCENARIO_CATEGORIES),
|
||||
"fixture.scenarioTaxonomy",
|
||||
errors,
|
||||
);
|
||||
for (const category of SCENARIO_CATEGORIES) {
|
||||
const scenarios = taxonomy[category];
|
||||
if (
|
||||
!Array.isArray(scenarios) ||
|
||||
scenarios.length === 0 ||
|
||||
scenarios.some(
|
||||
(scenario) => typeof scenario !== "string" || scenario.length === 0,
|
||||
) ||
|
||||
new Set(scenarios).size !== scenarios.length
|
||||
) {
|
||||
errors.push(
|
||||
`fixture.scenarioTaxonomy.${category} must contain unique scenarios`,
|
||||
);
|
||||
}
|
||||
}
|
||||
const ceilings = requireRecord(
|
||||
root.correctnessCeilings,
|
||||
"fixture.correctnessCeilings",
|
||||
errors,
|
||||
);
|
||||
if (Object.keys(ceilings).length === 0) {
|
||||
errors.push("fixture.correctnessCeilings must not be empty");
|
||||
}
|
||||
for (const [key, value] of Object.entries(ceilings)) {
|
||||
if (typeof value !== "number" || !Number.isFinite(value) || value < 0) {
|
||||
errors.push(
|
||||
`fixture.correctnessCeilings.${key} must be a non-negative number`,
|
||||
);
|
||||
}
|
||||
}
|
||||
return errors;
|
||||
}
|
||||
|
||||
export function validateEnvironment(environment) {
|
||||
const errors = [];
|
||||
const root = requireRecord(environment, "environment", errors);
|
||||
rejectUnknownKeys(
|
||||
root,
|
||||
new Set([
|
||||
"schemaVersion",
|
||||
"capture",
|
||||
"build",
|
||||
"package",
|
||||
"hardware",
|
||||
"platform",
|
||||
"display",
|
||||
"runtimeState",
|
||||
"fixture",
|
||||
"relay",
|
||||
"nativeDriver",
|
||||
]),
|
||||
"environment",
|
||||
errors,
|
||||
);
|
||||
if (root.schemaVersion !== 1)
|
||||
errors.push("environment.schemaVersion must equal 1");
|
||||
|
||||
const capture = requireRecord(root.capture, "environment.capture", errors);
|
||||
rejectUnknownKeys(
|
||||
capture,
|
||||
new Set(["tool", "version", "capturedAt", "infoPlistSha256"]),
|
||||
"environment.capture",
|
||||
errors,
|
||||
);
|
||||
if (capture.tool !== "buzz-wkwebview-environment-capture") {
|
||||
errors.push("environment.capture.tool is invalid");
|
||||
}
|
||||
if (capture.version !== "1.0.0") {
|
||||
errors.push("environment.capture.version is invalid");
|
||||
}
|
||||
requireTimestamp(
|
||||
capture.capturedAt,
|
||||
"environment.capture.capturedAt",
|
||||
errors,
|
||||
);
|
||||
requireSha(
|
||||
capture.infoPlistSha256,
|
||||
"environment.capture.infoPlistSha256",
|
||||
errors,
|
||||
);
|
||||
|
||||
const build = requireRecord(root.build, "environment.build", errors);
|
||||
rejectUnknownKeys(build, new Set(["gitSha"]), "environment.build", errors);
|
||||
requireSha(build.gitSha, "environment.build.gitSha", errors, GIT_SHA);
|
||||
|
||||
const pkg = requireRecord(root.package, "environment.package", errors);
|
||||
rejectUnknownKeys(
|
||||
pkg,
|
||||
new Set([
|
||||
"appPath",
|
||||
"bundleIdentifier",
|
||||
"version",
|
||||
"identity",
|
||||
"entitlementsSha256",
|
||||
]),
|
||||
"environment.package",
|
||||
errors,
|
||||
);
|
||||
for (const key of ["appPath", "bundleIdentifier", "version", "identity"]) {
|
||||
requireNonEmptyString(pkg[key], `environment.package.${key}`, errors);
|
||||
}
|
||||
if (typeof pkg.appPath === "string" && !pkg.appPath.endsWith(".app")) {
|
||||
errors.push("environment.package.appPath must identify a .app bundle");
|
||||
}
|
||||
requireSha(
|
||||
pkg.entitlementsSha256,
|
||||
"environment.package.entitlementsSha256",
|
||||
errors,
|
||||
);
|
||||
|
||||
const hardware = requireRecord(root.hardware, "environment.hardware", errors);
|
||||
rejectUnknownKeys(
|
||||
hardware,
|
||||
new Set(["model", "cpu", "memoryBytes"]),
|
||||
"environment.hardware",
|
||||
errors,
|
||||
);
|
||||
requireNonEmptyString(hardware.model, "environment.hardware.model", errors);
|
||||
requireNonEmptyString(hardware.cpu, "environment.hardware.cpu", errors);
|
||||
requireInteger(
|
||||
hardware.memoryBytes,
|
||||
"environment.hardware.memoryBytes",
|
||||
errors,
|
||||
1,
|
||||
);
|
||||
|
||||
const platform = requireRecord(root.platform, "environment.platform", errors);
|
||||
rejectUnknownKeys(
|
||||
platform,
|
||||
new Set(["macosVersion", "webkitVersion"]),
|
||||
"environment.platform",
|
||||
errors,
|
||||
);
|
||||
requireNonEmptyString(
|
||||
platform.macosVersion,
|
||||
"environment.platform.macosVersion",
|
||||
errors,
|
||||
);
|
||||
requireNonEmptyString(
|
||||
platform.webkitVersion,
|
||||
"environment.platform.webkitVersion",
|
||||
errors,
|
||||
);
|
||||
|
||||
const display = requireRecord(root.display, "environment.display", errors);
|
||||
rejectUnknownKeys(
|
||||
display,
|
||||
new Set(["refreshRateHz", "widthPx", "heightPx", "scaleFactor"]),
|
||||
"environment.display",
|
||||
errors,
|
||||
);
|
||||
requirePositiveNumber(
|
||||
display.refreshRateHz,
|
||||
"environment.display.refreshRateHz",
|
||||
errors,
|
||||
);
|
||||
requireInteger(display.widthPx, "environment.display.widthPx", errors, 1);
|
||||
requireInteger(display.heightPx, "environment.display.heightPx", errors, 1);
|
||||
requirePositiveNumber(
|
||||
display.scaleFactor,
|
||||
"environment.display.scaleFactor",
|
||||
errors,
|
||||
);
|
||||
|
||||
const runtime = requireRecord(
|
||||
root.runtimeState,
|
||||
"environment.runtimeState",
|
||||
errors,
|
||||
);
|
||||
rejectUnknownKeys(
|
||||
runtime,
|
||||
new Set(["powerSource", "lowPowerMode", "thermalState"]),
|
||||
"environment.runtimeState",
|
||||
errors,
|
||||
);
|
||||
if (!new Set(["ac", "battery"]).has(runtime.powerSource)) {
|
||||
errors.push("environment.runtimeState.powerSource must be ac or battery");
|
||||
}
|
||||
if (typeof runtime.lowPowerMode !== "boolean") {
|
||||
errors.push("environment.runtimeState.lowPowerMode must be boolean");
|
||||
}
|
||||
if (
|
||||
!new Set(["nominal", "fair", "serious", "critical"]).has(
|
||||
runtime.thermalState,
|
||||
)
|
||||
) {
|
||||
errors.push("environment.runtimeState.thermalState is invalid");
|
||||
}
|
||||
|
||||
const fixture = requireRecord(root.fixture, "environment.fixture", errors);
|
||||
rejectUnknownKeys(
|
||||
fixture,
|
||||
new Set(["fixtureId", "version", "sha256"]),
|
||||
"environment.fixture",
|
||||
errors,
|
||||
);
|
||||
requireNonEmptyString(
|
||||
fixture.fixtureId,
|
||||
"environment.fixture.fixtureId",
|
||||
errors,
|
||||
);
|
||||
requireInteger(fixture.version, "environment.fixture.version", errors, 1);
|
||||
requireSha(fixture.sha256, "environment.fixture.sha256", errors);
|
||||
|
||||
const relay = requireRecord(root.relay, "environment.relay", errors);
|
||||
rejectUnknownKeys(
|
||||
relay,
|
||||
new Set(["version", "fixtureVersion"]),
|
||||
"environment.relay",
|
||||
errors,
|
||||
);
|
||||
requireNonEmptyString(relay.version, "environment.relay.version", errors);
|
||||
requireNonEmptyString(
|
||||
relay.fixtureVersion,
|
||||
"environment.relay.fixtureVersion",
|
||||
errors,
|
||||
);
|
||||
|
||||
if (!Object.hasOwn(root, "nativeDriver")) {
|
||||
errors.push(
|
||||
"environment.nativeDriver is required (use null when no native driver is supplied)",
|
||||
);
|
||||
}
|
||||
const driver = root.nativeDriver;
|
||||
if (driver !== null) {
|
||||
const driverRecord = requireRecord(
|
||||
driver,
|
||||
"environment.nativeDriver",
|
||||
errors,
|
||||
);
|
||||
rejectUnknownKeys(
|
||||
driverRecord,
|
||||
new Set(["name", "version", "sha256"]),
|
||||
"environment.nativeDriver",
|
||||
errors,
|
||||
);
|
||||
requireNonEmptyString(
|
||||
driverRecord.name,
|
||||
"environment.nativeDriver.name",
|
||||
errors,
|
||||
);
|
||||
requireNonEmptyString(
|
||||
driverRecord.version,
|
||||
"environment.nativeDriver.version",
|
||||
errors,
|
||||
);
|
||||
requireSha(driverRecord.sha256, "environment.nativeDriver.sha256", errors);
|
||||
}
|
||||
|
||||
return errors;
|
||||
}
|
||||
|
||||
export function validateRun(run) {
|
||||
const errors = [];
|
||||
const root = requireRecord(run, "run", errors);
|
||||
rejectUnknownKeys(
|
||||
root,
|
||||
new Set([
|
||||
"schemaVersion",
|
||||
"runId",
|
||||
"createdAt",
|
||||
"status",
|
||||
"blockedReason",
|
||||
"failureReason",
|
||||
"appPath",
|
||||
"fixtureManifestPath",
|
||||
"environmentPath",
|
||||
"environmentSha256",
|
||||
"resultPath",
|
||||
"nativeDriverPath",
|
||||
]),
|
||||
"run",
|
||||
errors,
|
||||
);
|
||||
if (root.schemaVersion !== 1) errors.push("run.schemaVersion must equal 1");
|
||||
requireNonEmptyString(root.runId, "run.runId", errors);
|
||||
requireTimestamp(root.createdAt, "run.createdAt", errors);
|
||||
if (
|
||||
!new Set(["blocked", "running", "completed", "failed"]).has(root.status)
|
||||
) {
|
||||
errors.push("run.status is invalid");
|
||||
}
|
||||
requireNonEmptyString(root.appPath, "run.appPath", errors);
|
||||
if (typeof root.appPath === "string" && !root.appPath.endsWith(".app")) {
|
||||
errors.push("run.appPath must identify a .app bundle");
|
||||
}
|
||||
for (const key of ["fixtureManifestPath", "environmentPath", "resultPath"]) {
|
||||
requireNonEmptyString(root[key], `run.${key}`, errors);
|
||||
}
|
||||
requireSha(root.environmentSha256, "run.environmentSha256", errors);
|
||||
|
||||
if (root.status === "blocked") {
|
||||
if (root.blockedReason !== "native-driver-not-supplied") {
|
||||
errors.push(
|
||||
"run.blockedReason must equal native-driver-not-supplied for a blocked run",
|
||||
);
|
||||
}
|
||||
} else if (Object.hasOwn(root, "blockedReason")) {
|
||||
errors.push("run.blockedReason is only allowed for a blocked run");
|
||||
}
|
||||
if (
|
||||
root.status === "running" ||
|
||||
root.status === "completed" ||
|
||||
root.status === "failed"
|
||||
) {
|
||||
requireNonEmptyString(
|
||||
root.nativeDriverPath,
|
||||
"run.nativeDriverPath",
|
||||
errors,
|
||||
);
|
||||
} else if (Object.hasOwn(root, "nativeDriverPath")) {
|
||||
errors.push("run.nativeDriverPath is not allowed for a blocked run");
|
||||
}
|
||||
if (root.status === "failed") {
|
||||
requireNonEmptyString(root.failureReason, "run.failureReason", errors);
|
||||
} else if (Object.hasOwn(root, "failureReason")) {
|
||||
errors.push("run.failureReason is only allowed for a failed run");
|
||||
}
|
||||
return errors;
|
||||
}
|
||||
|
||||
export function validateResult(result, fixture, fixtureEntry) {
|
||||
const errors = [];
|
||||
const root = requireRecord(result, "result", errors);
|
||||
rejectUnknownKeys(
|
||||
root,
|
||||
new Set([
|
||||
"schemaVersion",
|
||||
"runId",
|
||||
"environmentSha256",
|
||||
"environment",
|
||||
"fixture",
|
||||
"samples",
|
||||
]),
|
||||
"result",
|
||||
errors,
|
||||
);
|
||||
if (root.schemaVersion !== 1)
|
||||
errors.push("result.schemaVersion must equal 1");
|
||||
requireNonEmptyString(root.runId, "result.runId", errors);
|
||||
requireSha(root.environmentSha256, "result.environmentSha256", errors);
|
||||
errors.push(...validateEnvironment(root.environment));
|
||||
if (root.environment?.nativeDriver === null) {
|
||||
errors.push("result.environment.nativeDriver provenance is required");
|
||||
}
|
||||
|
||||
const fixtureRef = requireRecord(root.fixture, "result.fixture", errors);
|
||||
rejectUnknownKeys(
|
||||
fixtureRef,
|
||||
new Set(["fixtureId", "version", "sha256"]),
|
||||
"result.fixture",
|
||||
errors,
|
||||
);
|
||||
requireNonEmptyString(
|
||||
fixtureRef.fixtureId,
|
||||
"result.fixture.fixtureId",
|
||||
errors,
|
||||
);
|
||||
requireInteger(fixtureRef.version, "result.fixture.version", errors, 1);
|
||||
requireSha(fixtureRef.sha256, "result.fixture.sha256", errors);
|
||||
|
||||
if (isRecord(root.environment?.fixture)) {
|
||||
for (const key of ["fixtureId", "version", "sha256"]) {
|
||||
if (fixtureRef[key] !== root.environment.fixture[key]) {
|
||||
errors.push(
|
||||
`result.fixture.${key} must match environment.fixture.${key}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
if (isRecord(fixture)) {
|
||||
if (
|
||||
fixtureRef.fixtureId !== fixture.fixtureId ||
|
||||
fixtureRef.version !== fixture.version
|
||||
) {
|
||||
errors.push(
|
||||
"result fixture identity does not match the checked-in fixture",
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
if (isRecord(fixtureEntry)) {
|
||||
for (const key of ["fixtureId", "version", "sha256"]) {
|
||||
if (fixtureRef[key] !== fixtureEntry[key]) {
|
||||
errors.push(`result.fixture.${key} must match the checked-in manifest`);
|
||||
}
|
||||
}
|
||||
}
|
||||
if (
|
||||
typeof fixtureEntry?.fixtureId === "string" &&
|
||||
root.environment?.fixture?.fixtureId === fixtureEntry.fixtureId
|
||||
) {
|
||||
for (const key of ["version", "sha256"]) {
|
||||
if (root.environment.fixture[key] !== fixtureEntry[key]) {
|
||||
errors.push(
|
||||
`result.environment.fixture.${key} must match the checked-in manifest`,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (!Array.isArray(root.samples) || root.samples.length === 0) {
|
||||
errors.push("result.samples must contain raw samples");
|
||||
return errors;
|
||||
}
|
||||
|
||||
const sampleIds = new Set();
|
||||
for (const [index, sampleValue] of root.samples.entries()) {
|
||||
const label = `result.samples[${index}]`;
|
||||
const sample = requireRecord(sampleValue, label, errors);
|
||||
rejectUnknownKeys(
|
||||
sample,
|
||||
new Set([
|
||||
"sampleId",
|
||||
"category",
|
||||
"scenario",
|
||||
"metric",
|
||||
"unit",
|
||||
"value",
|
||||
"observedAt",
|
||||
]),
|
||||
label,
|
||||
errors,
|
||||
);
|
||||
requireNonEmptyString(sample.sampleId, `${label}.sampleId`, errors);
|
||||
if (sampleIds.has(sample.sampleId))
|
||||
errors.push(`${label}.sampleId must be unique`);
|
||||
sampleIds.add(sample.sampleId);
|
||||
if (!SCENARIO_CATEGORIES.includes(sample.category)) {
|
||||
errors.push(`${label}.category is invalid`);
|
||||
}
|
||||
requireNonEmptyString(sample.scenario, `${label}.scenario`, errors);
|
||||
requireNonEmptyString(sample.metric, `${label}.metric`, errors);
|
||||
if (!SAMPLE_UNITS.has(sample.unit)) errors.push(`${label}.unit is invalid`);
|
||||
if (typeof sample.value !== "number" || !Number.isFinite(sample.value)) {
|
||||
errors.push(`${label}.value must be finite`);
|
||||
}
|
||||
requireTimestamp(sample.observedAt, `${label}.observedAt`, errors);
|
||||
const allowedScenarios = fixture?.scenarioTaxonomy?.[sample.category];
|
||||
if (
|
||||
Array.isArray(allowedScenarios) &&
|
||||
!allowedScenarios.includes(sample.scenario)
|
||||
) {
|
||||
errors.push(`${label}.scenario is not declared by the immutable fixture`);
|
||||
}
|
||||
}
|
||||
return errors;
|
||||
}
|
||||
|
||||
export function nearestRank(values, percentile) {
|
||||
if (!Array.isArray(values) || values.length === 0) {
|
||||
throw new Error("nearestRank requires at least one sample");
|
||||
}
|
||||
if (!(percentile > 0 && percentile <= 1)) {
|
||||
throw new Error("percentile must be in the interval (0, 1]");
|
||||
}
|
||||
if (
|
||||
values.some((value) => typeof value !== "number" || !Number.isFinite(value))
|
||||
) {
|
||||
throw new Error("nearestRank requires finite numeric samples");
|
||||
}
|
||||
const sorted = [...values].sort((left, right) => left - right);
|
||||
return sorted[Math.ceil(percentile * sorted.length) - 1];
|
||||
}
|
||||
|
||||
export function summarizeSamples(samples) {
|
||||
const groups = new Map();
|
||||
for (const sample of samples) {
|
||||
const key = JSON.stringify([
|
||||
sample.category,
|
||||
sample.scenario,
|
||||
sample.metric,
|
||||
sample.unit,
|
||||
]);
|
||||
const group = groups.get(key) ?? { ...sample, values: [] };
|
||||
group.values.push(sample.value);
|
||||
groups.set(key, group);
|
||||
}
|
||||
return [...groups.values()]
|
||||
.map(({ category, scenario, metric, unit, values }) => ({
|
||||
category,
|
||||
scenario,
|
||||
metric,
|
||||
unit,
|
||||
count: values.length,
|
||||
p50: nearestRank(values, 0.5),
|
||||
p95: nearestRank(values, 0.95),
|
||||
p99: nearestRank(values, 0.99),
|
||||
}))
|
||||
.sort((left, right) =>
|
||||
[left.category, left.scenario, left.metric, left.unit]
|
||||
.join("\0")
|
||||
.localeCompare(
|
||||
[right.category, right.scenario, right.metric, right.unit].join("\0"),
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
export async function readJson(filePath) {
|
||||
try {
|
||||
return JSON.parse(await readFile(filePath, "utf8"));
|
||||
} catch (error) {
|
||||
throw new Error(`Cannot read JSON from ${filePath}: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
export async function loadVerifiedFixture(manifestPath, fixtureId) {
|
||||
const manifest = await readJson(manifestPath);
|
||||
const entry = manifest.fixtures?.find(
|
||||
(candidate) => candidate.fixtureId === fixtureId,
|
||||
);
|
||||
if (!entry) throw new Error(`Fixture ${fixtureId} is not in ${manifestPath}`);
|
||||
const fixturePath = path.resolve(path.dirname(manifestPath), entry.path);
|
||||
const bytes = await readFile(fixturePath);
|
||||
const digest = createHash("sha256").update(bytes).digest("hex");
|
||||
if (bytes.byteLength !== entry.bytes || digest !== entry.sha256) {
|
||||
throw new Error(`Fixture integrity check failed for ${fixturePath}`);
|
||||
}
|
||||
const fixture = JSON.parse(bytes.toString("utf8"));
|
||||
const errors = validateFixture(fixture);
|
||||
if (errors.length > 0) {
|
||||
throw new Error(`Invalid immutable fixture:\n- ${errors.join("\n- ")}`);
|
||||
}
|
||||
return { fixture, entry, fixturePath };
|
||||
}
|
||||
755
desktop/tests/wkwebview/tooling/validation.test.mjs
Normal file
755
desktop/tests/wkwebview/tooling/validation.test.mjs
Normal file
@@ -0,0 +1,755 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { createHash } from "node:crypto";
|
||||
import { mkdtemp, mkdir, readFile, rm, writeFile } from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import test from "node:test";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { captureEnvironment } from "./capture-environment.mjs";
|
||||
import { launchHarness } from "./launch.mjs";
|
||||
import {
|
||||
materializeFixture,
|
||||
serializeMaterializedEvents,
|
||||
} from "./materialize-fixture.mjs";
|
||||
import { buildReport } from "./report.mjs";
|
||||
import {
|
||||
loadVerifiedFixture,
|
||||
nearestRank,
|
||||
summarizeSamples,
|
||||
validateEnvironment,
|
||||
validateFixture,
|
||||
validateRun,
|
||||
validateResult,
|
||||
} from "./validation.mjs";
|
||||
|
||||
const WKWEBVIEW_ROOT = path.resolve(
|
||||
path.dirname(fileURLToPath(import.meta.url)),
|
||||
"..",
|
||||
);
|
||||
const MANIFEST_PATH = path.join(WKWEBVIEW_ROOT, "fixture-manifest.json");
|
||||
const FIXTURE_SHA =
|
||||
"efdb8dc8ef0624be8ba86dc3fc0bb3374bd266b63f5a0f661df02797bc107000";
|
||||
const DIGEST = "a".repeat(64);
|
||||
|
||||
function sha256(value) {
|
||||
return createHash("sha256").update(value).digest("hex");
|
||||
}
|
||||
|
||||
function environment(
|
||||
appPath = "/Applications/Buzz.app",
|
||||
nativeDriverSha256 = DIGEST,
|
||||
) {
|
||||
return {
|
||||
schemaVersion: 1,
|
||||
capture: {
|
||||
tool: "buzz-wkwebview-environment-capture",
|
||||
version: "1.0.0",
|
||||
capturedAt: "2026-08-20T16:00:00.000Z",
|
||||
infoPlistSha256: sha256("fixture app plist"),
|
||||
},
|
||||
build: { gitSha: "b".repeat(40) },
|
||||
package: {
|
||||
appPath,
|
||||
bundleIdentifier: "com.block.buzz",
|
||||
version: "1.2.3",
|
||||
identity: "package-equivalent-test",
|
||||
entitlementsSha256: DIGEST,
|
||||
},
|
||||
hardware: {
|
||||
model: "Mac15,6",
|
||||
cpu: "Apple M3 Pro",
|
||||
memoryBytes: 18_000_000_000,
|
||||
},
|
||||
platform: { macosVersion: "15.6", webkitVersion: "620.3.4" },
|
||||
display: {
|
||||
refreshRateHz: 120,
|
||||
widthPx: 3024,
|
||||
heightPx: 1964,
|
||||
scaleFactor: 2,
|
||||
},
|
||||
runtimeState: {
|
||||
powerSource: "ac",
|
||||
lowPowerMode: false,
|
||||
thermalState: "nominal",
|
||||
},
|
||||
fixture: {
|
||||
fixtureId: "buzz-wkwebview-acceptance",
|
||||
version: 1,
|
||||
sha256: FIXTURE_SHA,
|
||||
},
|
||||
relay: { version: "buzz-relay-test", fixtureVersion: "wkwebview-relay-v1" },
|
||||
nativeDriver: {
|
||||
name: "native-test-driver",
|
||||
version: "1",
|
||||
sha256: nativeDriverSha256,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function result(
|
||||
appPath,
|
||||
runId = "run-1",
|
||||
environmentSha256 = DIGEST,
|
||||
nativeDriverSha256 = DIGEST,
|
||||
) {
|
||||
return {
|
||||
schemaVersion: 1,
|
||||
runId,
|
||||
environmentSha256,
|
||||
environment: environment(appPath, nativeDriverSha256),
|
||||
fixture: {
|
||||
fixtureId: "buzz-wkwebview-acceptance",
|
||||
version: 1,
|
||||
sha256: FIXTURE_SHA,
|
||||
},
|
||||
samples: [
|
||||
{
|
||||
sampleId: "sample-1",
|
||||
category: "switch",
|
||||
scenario: "cold-switch",
|
||||
metric: "stable-painted-frame",
|
||||
unit: "ms",
|
||||
value: 11,
|
||||
observedAt: "2026-08-20T16:00:00.000Z",
|
||||
},
|
||||
{
|
||||
sampleId: "sample-2",
|
||||
category: "switch",
|
||||
scenario: "cold-switch",
|
||||
metric: "stable-painted-frame",
|
||||
unit: "ms",
|
||||
value: 19,
|
||||
observedAt: "2026-08-20T16:00:01.000Z",
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
test("checked-in fixture matches its immutable manifest", async () => {
|
||||
const { fixture, entry } = await loadVerifiedFixture(
|
||||
MANIFEST_PATH,
|
||||
"buzz-wkwebview-acceptance",
|
||||
);
|
||||
assert.equal(entry.sha256, FIXTURE_SHA);
|
||||
assert.deepEqual(validateFixture(fixture), []);
|
||||
assert.equal(fixture.inputSemantics, "native-wheel-trackpad-keyboard");
|
||||
assert.deepEqual(Object.keys(fixture.scenarioTaxonomy), [
|
||||
"switch",
|
||||
"input",
|
||||
"timeline",
|
||||
"plateau",
|
||||
"ipc",
|
||||
"recovery",
|
||||
"crash",
|
||||
]);
|
||||
});
|
||||
|
||||
test("fixture materializer matches the golden count, boundaries, and full JSONL digest", async () => {
|
||||
const { fixture } = await loadVerifiedFixture(
|
||||
MANIFEST_PATH,
|
||||
"buzz-wkwebview-acceptance",
|
||||
);
|
||||
const events = materializeFixture(fixture);
|
||||
const bytes = serializeMaterializedEvents(events);
|
||||
assert.equal(events.length, 8740);
|
||||
assert.equal(Buffer.byteLength(bytes), 4255320);
|
||||
assert.equal(
|
||||
sha256(bytes),
|
||||
"ac948604f0100b93ce9d6f143b7160b6cd7f1389e1be407ed15fc45dca72edca",
|
||||
);
|
||||
assert.deepEqual(
|
||||
{ id: events[0].id, sig: events[0].sig },
|
||||
{
|
||||
id: "e4800994921ad9f45a226d672a7af9cd588bf343eecaed5cedadaf297f8886e5",
|
||||
sig: "4eaff818e1b8efa64c18d194ffb15f1b1fbd001bfd521889de65babbefb38047943a0ef03c0ad0d2fc425d5355368f56e39d6f24ee4b05847074ff1488e9d9f7",
|
||||
},
|
||||
);
|
||||
assert.deepEqual(
|
||||
{ id: events.at(-1).id, sig: events.at(-1).sig },
|
||||
{
|
||||
id: "de52203f953aca0df0a76d42fb1ef3c9f58140b15fe646d0077114be996a8135",
|
||||
sig: "35c8a50c1392bc5fd3358dfd6ccca8cbed3169443031ce37b8f9a15789b4fa014e1d012256b615d1d54f04ee16d4be26e2509e8e4ccb1814931b9465d6c7ac6e",
|
||||
},
|
||||
);
|
||||
for (let index = 1; index < events.length; index += 1) {
|
||||
const previous = events[index - 1];
|
||||
const current = events[index];
|
||||
assert(
|
||||
previous.created_at < current.created_at ||
|
||||
(previous.created_at === current.created_at &&
|
||||
previous.id.localeCompare(current.id) <= 0),
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test("environment capture derives package and host provenance at the command boundary", async (t) => {
|
||||
const directory = await mkdtemp(
|
||||
path.join(os.tmpdir(), "buzz-wkwebview-capture-"),
|
||||
);
|
||||
t.after(() => rm(directory, { recursive: true, force: true }));
|
||||
const appPath = path.join(directory, "Buzz.app");
|
||||
const infoPlistPath = path.join(appPath, "Contents", "Info.plist");
|
||||
const driverPath = path.join(directory, "native-driver");
|
||||
await mkdir(path.dirname(infoPlistPath), { recursive: true });
|
||||
await writeFile(infoPlistPath, "captured plist bytes");
|
||||
await writeFile(driverPath, "captured driver bytes");
|
||||
const responses = new Map([
|
||||
[
|
||||
`plutil -extract CFBundleIdentifier raw -o - ${infoPlistPath}`,
|
||||
{ stdout: "com.block.buzz\n", stderr: "" },
|
||||
],
|
||||
[
|
||||
`plutil -extract CFBundleShortVersionString raw -o - ${infoPlistPath}`,
|
||||
{ stdout: "1.2.3\n", stderr: "" },
|
||||
],
|
||||
[
|
||||
`codesign -dv --verbose=4 ${appPath}`,
|
||||
{
|
||||
stdout: "",
|
||||
stderr: "Authority=Developer ID Application: Block, Inc.\n",
|
||||
},
|
||||
],
|
||||
[
|
||||
`codesign -d --entitlements :- ${appPath}`,
|
||||
{ stdout: "<plist>entitlements</plist>\n", stderr: "" },
|
||||
],
|
||||
["sysctl -n hw.model", { stdout: "Mac15,6\n", stderr: "" }],
|
||||
[
|
||||
"sysctl -n machdep.cpu.brand_string",
|
||||
{ stdout: "Apple M3 Pro\n", stderr: "" },
|
||||
],
|
||||
["sysctl -n hw.memsize", { stdout: "18000000000\n", stderr: "" }],
|
||||
["sw_vers -productVersion", { stdout: "15.6\n", stderr: "" }],
|
||||
[
|
||||
"defaults read /System/Library/Frameworks/WebKit.framework/Resources/Info CFBundleShortVersionString",
|
||||
{ stdout: "620.3.4\n", stderr: "" },
|
||||
],
|
||||
[
|
||||
"system_profiler SPDisplaysDataType -json",
|
||||
{
|
||||
stdout: JSON.stringify({
|
||||
SPDisplaysDataType: [
|
||||
{
|
||||
spdisplays_ndrvs: [
|
||||
{
|
||||
spdisplays_main: "spdisplays_yes",
|
||||
_spdisplays_pixels: "3024 x 1964",
|
||||
_spdisplays_resolution: "1512 x 982 @ 120.00Hz",
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
}),
|
||||
stderr: "",
|
||||
},
|
||||
],
|
||||
["pmset -g batt", { stdout: "Now drawing from 'AC Power'\n", stderr: "" }],
|
||||
[
|
||||
"pmset -g custom",
|
||||
{
|
||||
stdout: "Battery Power:\n powermode 1\nAC Power:\n powermode 0\n",
|
||||
stderr: "",
|
||||
},
|
||||
],
|
||||
[
|
||||
"pmset -g therm",
|
||||
{
|
||||
stdout: "Note: No thermal warning level has been recorded\n",
|
||||
stderr: "",
|
||||
},
|
||||
],
|
||||
]);
|
||||
const commandRunner = async (command, args) => {
|
||||
const response = responses.get([command, ...args].join(" "));
|
||||
if (!response)
|
||||
throw new Error(`unexpected command: ${command} ${args.join(" ")}`);
|
||||
return response;
|
||||
};
|
||||
|
||||
const captured = await captureEnvironment({
|
||||
appPath,
|
||||
gitSha: "b".repeat(40),
|
||||
relayVersion: "relay-1",
|
||||
nativeDriverPath: driverPath,
|
||||
nativeDriverName: "driver",
|
||||
nativeDriverVersion: "1",
|
||||
manifestPath: MANIFEST_PATH,
|
||||
commandRunner,
|
||||
capturedAt: "2026-08-20T16:00:00.000Z",
|
||||
});
|
||||
assert.deepEqual(validateEnvironment(captured), []);
|
||||
assert.equal(captured.package.bundleIdentifier, "com.block.buzz");
|
||||
assert.equal(
|
||||
captured.package.identity,
|
||||
"Developer ID Application: Block, Inc.",
|
||||
);
|
||||
assert.equal(captured.display.refreshRateHz, 120);
|
||||
assert.equal(captured.display.scaleFactor, 2);
|
||||
assert.deepEqual(captured.runtimeState, {
|
||||
powerSource: "ac",
|
||||
lowPowerMode: false,
|
||||
thermalState: "nominal",
|
||||
});
|
||||
assert.equal(captured.nativeDriver.sha256, sha256("captured driver bytes"));
|
||||
});
|
||||
|
||||
test("environment capture fails closed when required host evidence is unavailable", async (t) => {
|
||||
const directory = await mkdtemp(
|
||||
path.join(os.tmpdir(), "buzz-wkwebview-capture-fail-"),
|
||||
);
|
||||
t.after(() => rm(directory, { recursive: true, force: true }));
|
||||
const appPath = path.join(directory, "Buzz.app");
|
||||
await mkdir(path.join(appPath, "Contents"), { recursive: true });
|
||||
await writeFile(path.join(appPath, "Contents", "Info.plist"), "plist");
|
||||
const driverPath = path.join(directory, "driver");
|
||||
await writeFile(driverPath, "driver");
|
||||
await assert.rejects(
|
||||
() =>
|
||||
captureEnvironment({
|
||||
appPath,
|
||||
gitSha: "b".repeat(40),
|
||||
relayVersion: "relay-1",
|
||||
nativeDriverPath: driverPath,
|
||||
nativeDriverName: "driver",
|
||||
nativeDriverVersion: "1",
|
||||
manifestPath: MANIFEST_PATH,
|
||||
commandRunner: async () => ({ stdout: "", stderr: "" }),
|
||||
}),
|
||||
/Capture unavailable: bundle identifier/,
|
||||
);
|
||||
});
|
||||
|
||||
test("valid environment and raw result pass provenance validation", async () => {
|
||||
const { fixture, entry } = await loadVerifiedFixture(
|
||||
MANIFEST_PATH,
|
||||
"buzz-wkwebview-acceptance",
|
||||
);
|
||||
const rawResult = result("/Applications/Buzz.app");
|
||||
assert.deepEqual(validateEnvironment(rawResult.environment), []);
|
||||
assert.deepEqual(validateResult(rawResult, fixture, entry), []);
|
||||
});
|
||||
|
||||
test("missing provenance, unknown keys, malformed samples, and fixture drift fail", async () => {
|
||||
const { fixture, entry } = await loadVerifiedFixture(
|
||||
MANIFEST_PATH,
|
||||
"buzz-wkwebview-acceptance",
|
||||
);
|
||||
const rawResult = result("/Applications/Buzz.app");
|
||||
delete rawResult.environment.platform.webkitVersion;
|
||||
rawResult.environment.package.unexpected = true;
|
||||
rawResult.samples[0].value = Number.NaN;
|
||||
rawResult.samples[0].unexpected = true;
|
||||
rawResult.samples[1].sampleId = "sample-1";
|
||||
rawResult.samples.push({
|
||||
...rawResult.samples[1],
|
||||
sampleId: "sample-3",
|
||||
category: "switch",
|
||||
scenario: "not-in-fixture",
|
||||
});
|
||||
rawResult.fixture.sha256 = "c".repeat(64);
|
||||
rawResult.environment.fixture.sha256 = rawResult.fixture.sha256;
|
||||
const errors = validateResult(rawResult, fixture, entry);
|
||||
assert(errors.some((error) => error.includes("webkitVersion")));
|
||||
assert(
|
||||
errors.some((error) => error.includes("package.unexpected is not allowed")),
|
||||
);
|
||||
assert(
|
||||
errors.some((error) =>
|
||||
error.includes("samples[0].unexpected is not allowed"),
|
||||
),
|
||||
);
|
||||
assert(errors.some((error) => error.includes("value must be finite")));
|
||||
assert(errors.some((error) => error.includes("sampleId must be unique")));
|
||||
assert(errors.some((error) => error.includes("immutable fixture")));
|
||||
assert(
|
||||
errors.some((error) =>
|
||||
error.includes("sha256 must match the checked-in manifest"),
|
||||
),
|
||||
);
|
||||
});
|
||||
|
||||
test("nearest-rank summaries use the ceiling rank without interpolation", () => {
|
||||
assert.equal(nearestRank([4, 1, 3, 2], 0.5), 2);
|
||||
assert.equal(nearestRank([4, 1, 3, 2], 0.95), 4);
|
||||
const summaries = summarizeSamples(result("/Applications/Buzz.app").samples);
|
||||
assert.deepEqual(summaries, [
|
||||
{
|
||||
category: "switch",
|
||||
scenario: "cold-switch",
|
||||
metric: "stable-painted-frame",
|
||||
unit: "ms",
|
||||
count: 2,
|
||||
p50: 11,
|
||||
p95: 19,
|
||||
p99: 19,
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
test("report retains provenance and rejects launch-environment substitution", async (t) => {
|
||||
const directory = await mkdtemp(
|
||||
path.join(os.tmpdir(), "buzz-wkwebview-report-"),
|
||||
);
|
||||
t.after(() => rm(directory, { recursive: true, force: true }));
|
||||
const appPath = "/Applications/Buzz.app";
|
||||
const environmentPath = path.join(directory, "environment.json");
|
||||
const environmentBytes = Buffer.from(
|
||||
`${JSON.stringify(environment(appPath))}\n`,
|
||||
);
|
||||
const environmentSha256 = createHash("sha256")
|
||||
.update(environmentBytes)
|
||||
.digest("hex");
|
||||
await writeFile(environmentPath, environmentBytes);
|
||||
const resultPath = path.join(directory, "raw-result.json");
|
||||
const runPath = path.join(directory, "run.json");
|
||||
const rawResult = result(appPath, "run-1", environmentSha256);
|
||||
const completeRun = {
|
||||
schemaVersion: 1,
|
||||
runId: "run-1",
|
||||
createdAt: "2026-08-20T16:00:00.000Z",
|
||||
status: "completed",
|
||||
appPath,
|
||||
fixtureManifestPath: MANIFEST_PATH,
|
||||
environmentPath,
|
||||
environmentSha256,
|
||||
resultPath,
|
||||
nativeDriverPath: "/tmp/native-test-driver",
|
||||
};
|
||||
await writeFile(resultPath, `${JSON.stringify(rawResult)}\n`);
|
||||
await writeFile(runPath, `${JSON.stringify(completeRun)}\n`);
|
||||
const report = await buildReport({
|
||||
resultPath,
|
||||
runPath,
|
||||
manifestPath: MANIFEST_PATH,
|
||||
});
|
||||
assert.equal(report.percentileMethod, "nearest-rank");
|
||||
assert.match(report.provenance.rawResultSha256, /^[0-9a-f]{64}$/);
|
||||
assert.equal(report.provenance.fixture.sha256, FIXTURE_SHA);
|
||||
assert.equal(report.provenance.nativeDriver.name, "native-test-driver");
|
||||
assert.equal(report.provenance.platform.webkitVersion, "620.3.4");
|
||||
assert.deepEqual(report.summaries[0], {
|
||||
category: "switch",
|
||||
scenario: "cold-switch",
|
||||
metric: "stable-painted-frame",
|
||||
unit: "ms",
|
||||
count: 2,
|
||||
p50: 11,
|
||||
p95: 19,
|
||||
p99: 19,
|
||||
});
|
||||
|
||||
rawResult.environment.package.version = "substituted";
|
||||
await writeFile(resultPath, `${JSON.stringify(rawResult)}\n`);
|
||||
await assert.rejects(
|
||||
() => buildReport({ resultPath, runPath, manifestPath: MANIFEST_PATH }),
|
||||
/Embedded result environment does not match launch environment/,
|
||||
);
|
||||
|
||||
rawResult.environment.package.version = "1.2.3";
|
||||
await writeFile(resultPath, `${JSON.stringify(rawResult)}\n`);
|
||||
delete completeRun.createdAt;
|
||||
completeRun.unexpected = true;
|
||||
await writeFile(runPath, `${JSON.stringify(completeRun)}\n`);
|
||||
await assert.rejects(
|
||||
() => buildReport({ resultPath, runPath, manifestPath: MANIFEST_PATH }),
|
||||
/run\.unexpected is not allowed[\s\S]*run\.createdAt/,
|
||||
);
|
||||
});
|
||||
|
||||
test("launch records blocked instead of substituting a browser without a native driver", async (t) => {
|
||||
const directory = await mkdtemp(
|
||||
path.join(os.tmpdir(), "buzz-wkwebview-launch-"),
|
||||
);
|
||||
t.after(() => rm(directory, { recursive: true, force: true }));
|
||||
const appPath = path.join(directory, "Buzz.app");
|
||||
await mkdir(path.join(appPath, "Contents", "MacOS"), { recursive: true });
|
||||
await writeFile(
|
||||
path.join(appPath, "Contents", "Info.plist"),
|
||||
"fixture app plist",
|
||||
);
|
||||
const environmentPath = path.join(directory, "environment.json");
|
||||
const runPath = path.join(directory, "run.json");
|
||||
const resultPath = path.join(directory, "raw-result.json");
|
||||
const blockedEnvironment = environment(appPath);
|
||||
blockedEnvironment.nativeDriver = null;
|
||||
await writeFile(environmentPath, `${JSON.stringify(blockedEnvironment)}\n`);
|
||||
|
||||
const run = await launchHarness({
|
||||
appPath,
|
||||
environmentPath,
|
||||
runPath,
|
||||
resultPath,
|
||||
manifestPath: MANIFEST_PATH,
|
||||
});
|
||||
assert.equal(run.status, "blocked");
|
||||
assert.equal(run.blockedReason, "native-driver-not-supplied");
|
||||
assert.equal(run.nativeDriverPath, undefined);
|
||||
assert.deepEqual(JSON.parse(await readFile(runPath, "utf8")), run);
|
||||
await assert.rejects(() => readFile(resultPath), { code: "ENOENT" });
|
||||
});
|
||||
|
||||
test("tampered fixture bytes fail manifest verification", async (t) => {
|
||||
const directory = await mkdtemp(
|
||||
path.join(os.tmpdir(), "buzz-wkwebview-fixture-"),
|
||||
);
|
||||
t.after(() => rm(directory, { recursive: true, force: true }));
|
||||
await mkdir(path.join(directory, "fixtures"));
|
||||
await writeFile(
|
||||
path.join(directory, "fixtures", "fixture.json"),
|
||||
'{"tampered":true}\n',
|
||||
);
|
||||
await writeFile(
|
||||
path.join(directory, "manifest.json"),
|
||||
JSON.stringify({
|
||||
manifestVersion: 1,
|
||||
fixtures: [
|
||||
{
|
||||
fixtureId: "fixture",
|
||||
version: 1,
|
||||
path: "fixtures/fixture.json",
|
||||
bytes: 1,
|
||||
sha256: DIGEST,
|
||||
},
|
||||
],
|
||||
}),
|
||||
);
|
||||
await assert.rejects(
|
||||
() => loadVerifiedFixture(path.join(directory, "manifest.json"), "fixture"),
|
||||
/integrity check failed/,
|
||||
);
|
||||
});
|
||||
|
||||
test("native driver success persists a completed terminal run", async (t) => {
|
||||
const directory = await mkdtemp(
|
||||
path.join(os.tmpdir(), "buzz-wkwebview-driver-success-"),
|
||||
);
|
||||
t.after(() => rm(directory, { recursive: true, force: true }));
|
||||
const appPath = path.join(directory, "Buzz.app");
|
||||
await mkdir(path.join(appPath, "Contents", "MacOS"), { recursive: true });
|
||||
await writeFile(
|
||||
path.join(appPath, "Contents", "Info.plist"),
|
||||
"fixture app plist",
|
||||
);
|
||||
const environmentPath = path.join(directory, "environment.json");
|
||||
const runPath = path.join(directory, "run.json");
|
||||
const resultPath = path.join(directory, "raw-result.json");
|
||||
const driverPath = path.join(directory, "driver.sh");
|
||||
const driverSource = `#!/bin/sh\nnode - <<'NODE'\nconst fs = require("node:fs");\nconst run = require(process.env.BUZZ_WKWEBVIEW_RUN);\nconst environment = JSON.parse(fs.readFileSync(run.environmentPath, "utf8"));\nconst result = ${JSON.stringify(
|
||||
{
|
||||
schemaVersion: 1,
|
||||
fixture: {
|
||||
fixtureId: "buzz-wkwebview-acceptance",
|
||||
version: 1,
|
||||
sha256: FIXTURE_SHA,
|
||||
},
|
||||
samples: result(appPath).samples,
|
||||
},
|
||||
)};\nresult.runId = run.runId;\nresult.environmentSha256 = run.environmentSha256;\nresult.environment = environment;\nfs.writeFileSync(run.resultPath, JSON.stringify(result) + "\\n", { flag: "wx" });\nNODE\n`;
|
||||
const driverSha256 = sha256(driverSource);
|
||||
const capturedEnvironment = environment(appPath, driverSha256);
|
||||
await writeFile(environmentPath, `${JSON.stringify(capturedEnvironment)}\n`);
|
||||
await writeFile(driverPath, driverSource, { mode: 0o755 });
|
||||
|
||||
const run = await launchHarness({
|
||||
appPath,
|
||||
environmentPath,
|
||||
runPath,
|
||||
resultPath,
|
||||
nativeDriverPath: driverPath,
|
||||
manifestPath: MANIFEST_PATH,
|
||||
});
|
||||
assert.equal(run.status, "completed");
|
||||
assert.equal(run.failureReason, undefined);
|
||||
assert.deepEqual(JSON.parse(await readFile(runPath, "utf8")), run);
|
||||
const writtenResult = JSON.parse(await readFile(resultPath, "utf8"));
|
||||
assert.equal(writtenResult.runId, run.runId);
|
||||
assert.equal(writtenResult.environmentSha256, run.environmentSha256);
|
||||
});
|
||||
|
||||
test("launch rejects a substituted native driver before creating a run", async (t) => {
|
||||
const directory = await mkdtemp(
|
||||
path.join(os.tmpdir(), "buzz-wkwebview-driver-digest-"),
|
||||
);
|
||||
t.after(() => rm(directory, { recursive: true, force: true }));
|
||||
const appPath = path.join(directory, "Buzz.app");
|
||||
await mkdir(path.join(appPath, "Contents", "MacOS"), { recursive: true });
|
||||
await writeFile(
|
||||
path.join(appPath, "Contents", "Info.plist"),
|
||||
"fixture app plist",
|
||||
);
|
||||
const environmentPath = path.join(directory, "environment.json");
|
||||
const runPath = path.join(directory, "run.json");
|
||||
const resultPath = path.join(directory, "raw-result.json");
|
||||
const driverPath = path.join(directory, "driver.sh");
|
||||
await writeFile(driverPath, "#!/bin/sh\nexit 0\n", { mode: 0o755 });
|
||||
await writeFile(environmentPath, `${JSON.stringify(environment(appPath))}\n`);
|
||||
|
||||
await assert.rejects(
|
||||
() =>
|
||||
launchHarness({
|
||||
appPath,
|
||||
environmentPath,
|
||||
runPath,
|
||||
resultPath,
|
||||
nativeDriverPath: driverPath,
|
||||
manifestPath: MANIFEST_PATH,
|
||||
}),
|
||||
/Native driver SHA-256 does not match/,
|
||||
);
|
||||
await assert.rejects(() => readFile(runPath), { code: "ENOENT" });
|
||||
});
|
||||
|
||||
test("launch rejects a stale raw-result path before invoking the driver", async (t) => {
|
||||
const directory = await mkdtemp(
|
||||
path.join(os.tmpdir(), "buzz-wkwebview-stale-result-"),
|
||||
);
|
||||
t.after(() => rm(directory, { recursive: true, force: true }));
|
||||
const appPath = path.join(directory, "Buzz.app");
|
||||
await mkdir(path.join(appPath, "Contents", "MacOS"), { recursive: true });
|
||||
await writeFile(
|
||||
path.join(appPath, "Contents", "Info.plist"),
|
||||
"fixture app plist",
|
||||
);
|
||||
const environmentPath = path.join(directory, "environment.json");
|
||||
const runPath = path.join(directory, "run.json");
|
||||
const resultPath = path.join(directory, "raw-result.json");
|
||||
const driverPath = path.join(directory, "driver.sh");
|
||||
const markerPath = path.join(directory, "driver-invoked");
|
||||
const driverSource = `#!/bin/sh\ntouch ${JSON.stringify(markerPath)}\nexit 0\n`;
|
||||
await writeFile(driverPath, driverSource, { mode: 0o755 });
|
||||
await writeFile(
|
||||
environmentPath,
|
||||
`${JSON.stringify(environment(appPath, sha256(driverSource)))}\n`,
|
||||
);
|
||||
await writeFile(resultPath, `${JSON.stringify(result(appPath))}\n`);
|
||||
|
||||
await assert.rejects(
|
||||
() =>
|
||||
launchHarness({
|
||||
appPath,
|
||||
environmentPath,
|
||||
runPath,
|
||||
resultPath,
|
||||
nativeDriverPath: driverPath,
|
||||
manifestPath: MANIFEST_PATH,
|
||||
}),
|
||||
/Raw result path already exists/,
|
||||
);
|
||||
await assert.rejects(() => readFile(markerPath), { code: "ENOENT" });
|
||||
await assert.rejects(() => readFile(runPath), { code: "ENOENT" });
|
||||
});
|
||||
|
||||
test("native driver failure persists a failed terminal run with reason", async (t) => {
|
||||
const directory = await mkdtemp(
|
||||
path.join(os.tmpdir(), "buzz-wkwebview-driver-failure-"),
|
||||
);
|
||||
t.after(() => rm(directory, { recursive: true, force: true }));
|
||||
const appPath = path.join(directory, "Buzz.app");
|
||||
await mkdir(path.join(appPath, "Contents", "MacOS"), { recursive: true });
|
||||
await writeFile(
|
||||
path.join(appPath, "Contents", "Info.plist"),
|
||||
"fixture app plist",
|
||||
);
|
||||
const environmentPath = path.join(directory, "environment.json");
|
||||
const runPath = path.join(directory, "run.json");
|
||||
const resultPath = path.join(directory, "raw-result.json");
|
||||
const driverPath = path.join(directory, "driver.sh");
|
||||
const driverSource = "#!/bin/sh\nexit 7\n";
|
||||
await writeFile(
|
||||
environmentPath,
|
||||
`${JSON.stringify(environment(appPath, sha256(driverSource)))}\n`,
|
||||
);
|
||||
await writeFile(driverPath, driverSource, { mode: 0o755 });
|
||||
|
||||
await assert.rejects(
|
||||
() =>
|
||||
launchHarness({
|
||||
appPath,
|
||||
environmentPath,
|
||||
runPath,
|
||||
resultPath,
|
||||
nativeDriverPath: driverPath,
|
||||
manifestPath: MANIFEST_PATH,
|
||||
}),
|
||||
/status 7/,
|
||||
);
|
||||
const run = JSON.parse(await readFile(runPath, "utf8"));
|
||||
assert.equal(run.status, "failed");
|
||||
assert.equal(run.failureReason, "Native driver exited with status 7");
|
||||
assert.equal(run.nativeDriverPath, driverPath);
|
||||
});
|
||||
|
||||
test("run validator enforces status-dependent fields and unknown-key rejection", () => {
|
||||
const completed = {
|
||||
schemaVersion: 1,
|
||||
runId: "run-1",
|
||||
createdAt: "2026-08-20T16:00:00.000Z",
|
||||
status: "completed",
|
||||
appPath: "/Applications/Buzz.app",
|
||||
fixtureManifestPath: "/tmp/manifest.json",
|
||||
environmentPath: "/tmp/environment.json",
|
||||
environmentSha256: DIGEST,
|
||||
resultPath: "/tmp/result.json",
|
||||
unexpected: true,
|
||||
};
|
||||
const errors = validateRun(completed);
|
||||
assert(
|
||||
errors.some((error) => error.includes("run.unexpected is not allowed")),
|
||||
);
|
||||
assert(
|
||||
errors.some((error) =>
|
||||
error.includes("run.nativeDriverPath must be a non-empty string"),
|
||||
),
|
||||
);
|
||||
|
||||
assert(
|
||||
validateRun({
|
||||
...completed,
|
||||
status: "running",
|
||||
unexpected: undefined,
|
||||
}).some((error) =>
|
||||
error.includes("run.nativeDriverPath must be a non-empty string"),
|
||||
),
|
||||
);
|
||||
assert(
|
||||
validateRun({
|
||||
...completed,
|
||||
nativeDriverPath: "/tmp/driver",
|
||||
blockedReason: "native-driver-not-supplied",
|
||||
unexpected: undefined,
|
||||
}).some((error) =>
|
||||
error.includes("run.blockedReason is only allowed for a blocked run"),
|
||||
),
|
||||
);
|
||||
assert(
|
||||
validateRun({
|
||||
...completed,
|
||||
nativeDriverPath: "/tmp/driver",
|
||||
failureReason: "stale failure",
|
||||
unexpected: undefined,
|
||||
}).some((error) =>
|
||||
error.includes("run.failureReason is only allowed for a failed run"),
|
||||
),
|
||||
);
|
||||
});
|
||||
|
||||
test("fixture validator rejects nested unknown keys", async () => {
|
||||
const { fixture } = await loadVerifiedFixture(
|
||||
MANIFEST_PATH,
|
||||
"buzz-wkwebview-acceptance",
|
||||
);
|
||||
const malformed = structuredClone(fixture);
|
||||
malformed.channels[0].unexpected = true;
|
||||
malformed.scenarioTaxonomy.unexpected = ["scenario"];
|
||||
const errors = validateFixture(malformed);
|
||||
assert(
|
||||
errors.some((error) =>
|
||||
error.includes("channels[0].unexpected is not allowed"),
|
||||
),
|
||||
);
|
||||
assert(
|
||||
errors.some((error) =>
|
||||
error.includes("scenarioTaxonomy.unexpected is not allowed"),
|
||||
),
|
||||
);
|
||||
});
|
||||
421
docs/desktop-performance-rearchitecture.md
Normal file
421
docs/desktop-performance-rearchitecture.md
Normal file
@@ -0,0 +1,421 @@
|
||||
# Desktop performance rearchitecture
|
||||
|
||||
Status: implementation contract, milestone 0
|
||||
|
||||
Buzz Desktop will move from renderer-owned relay state and network-backed message
|
||||
queries to a native Rust data plane with a durable local replica. The migration
|
||||
is a strangler: the current path remains authoritative while additive native
|
||||
components run dormant, then in shadow, before each read surface cuts over.
|
||||
|
||||
This document is normative for implementation sequencing and release safety. It
|
||||
complements the existing channel-window wire contract in
|
||||
[`docs/bridge-channel-window.md`](bridge-channel-window.md); it does not change
|
||||
NIP-CW ordering, cursor, summary, or auxiliary-closure semantics.
|
||||
|
||||
## Success criteria
|
||||
|
||||
The shipped desktop app should:
|
||||
|
||||
- paint the last-known sidebar and channel state without waiting for the relay;
|
||||
- reach stable first paint for a warm channel switch below 100 ms at p95 on the
|
||||
named reference device and build as a product objective, promoting that number
|
||||
to a release gate only after milestone 0 proves it against packaged current
|
||||
main;
|
||||
- remain responsive during cold channel entry, live event storms, pagination,
|
||||
and reconnect;
|
||||
- preserve timeline position continuously through prepend, backdated insert,
|
||||
late row growth, target navigation, and channel restoration;
|
||||
- bound loaded rows, mounted DOM, retained nodes, native and webview memory,
|
||||
replica disk growth, and IPC queue pressure; and
|
||||
- use one native authenticated relay session after all feature cutovers.
|
||||
|
||||
Absolute budgets beyond the warm-switch target are set from milestone 0's
|
||||
packaged-build baseline. A budget is not valid unless it names hardware, macOS,
|
||||
app commit/build, data fixture, run count, and percentile method.
|
||||
|
||||
## Release contract
|
||||
|
||||
Every slice has three independent gates:
|
||||
|
||||
1. **Merge-safe:** focused tests pass; additive code is dormant by default;
|
||||
migrations are transactional and compatible with existing data; failure
|
||||
cannot perturb the legacy path.
|
||||
2. **Release-safe:** the shipped binary is correct with the feature disabled;
|
||||
shadow work has explicit CPU, memory, disk, and IPC limits; an unavailable,
|
||||
corrupt, or newer replica fails open to legacy reads and never blocks app
|
||||
startup.
|
||||
3. **Cutover-safe:** shadow comparison and packaged-app evidence pass; the
|
||||
surface has a runtime kill switch and automatic legacy fallback; rollback
|
||||
cannot lose or duplicate user state. Any comparison lane that reports queue
|
||||
overflow, version gaps, or missing evidence is invalid, not a pass.
|
||||
|
||||
A red cutover gate leaves the new path off and does not prevent unrelated safe
|
||||
work from shipping. A legacy implementation is not deleted in the same release
|
||||
that first makes its replacement authoritative. Keep fallback for at least one
|
||||
proven release cycle, then remove it in a later, separately reversible slice.
|
||||
|
||||
`replica.db` is rebuildable relay-derived state. An older app ignores an unknown
|
||||
or newer replica and may recreate its own compatible database. The durable
|
||||
outbox is the exception: once introduced, it is non-rebuildable user state and
|
||||
gets an independent cutover with one authoritative sender, stable event IDs,
|
||||
idempotent retry, restart proof, and duplicate-free rollback.
|
||||
|
||||
## Target boundaries
|
||||
|
||||
### Native relay fan-out
|
||||
|
||||
`RelaySession` becomes the one socket owner for a `(relay, identity)` scope.
|
||||
Finite `fetch_events` requests remain separately multiplexed and are not routed
|
||||
through persistent fan-out.
|
||||
|
||||
Persistent consumers register through an opaque lease. Registration is
|
||||
non-destructive; lease drop/unregister is idempotent; scope replacement and
|
||||
session shutdown close every consumer exactly once. Each consumer has its own
|
||||
bounded queue and declared delivery policy:
|
||||
|
||||
- **durable ingest/archive:** ordered, lossless delivery; backpressure is
|
||||
explicit and cannot be converted into silent drops;
|
||||
- **shadow comparison:** bounded; lag or overflow is observable, invalidates the
|
||||
comparison window, and never stalls or damages durable delivery; and
|
||||
- **renderer projection deltas:** bounded and coalescing/invalidate-capable;
|
||||
renderer slowness never stalls durable storage.
|
||||
|
||||
The socket loop must not await independent consumers sequentially. Tests cover
|
||||
ordered dual delivery, a slow/overflowing shadow beside lossless archive
|
||||
traffic, unregister during a blocked send, same-scope remount, scope teardown,
|
||||
finite-fetch coexistence, reconnect, and CLOSED/rate-limit recovery.
|
||||
|
||||
### Replica ownership and transactions
|
||||
|
||||
Use a standalone SQLite WAL database, `replica.db`, scoped by the canonical
|
||||
relay/community identity (not raw URL spelling) and signer identity. Alias URLs
|
||||
for the same community resolve to one store; different communities or signers
|
||||
never share one. Do not reuse `archive.db`: archive authorization, retention,
|
||||
and ownership are different contracts. The replica has one writer and a
|
||||
versioned, marker-guarded migration ledger. Each migration and marker commit in
|
||||
one transaction and is safe to rerun after a crash.
|
||||
|
||||
A verified event is applied in one transaction containing:
|
||||
|
||||
1. event-ID dedupe;
|
||||
2. canonical event and tag indexes;
|
||||
3. deletions/tombstones and auxiliary closure inputs;
|
||||
4. affected sidebar, channel-window, and thread projection rows; and
|
||||
5. only those coverage intervals proved by the same fetch/ingest operation.
|
||||
|
||||
Deletion facts are order-independent. A deletion whose target is not yet stored
|
||||
creates a durable pending tombstone keyed by target event ID, retaining the
|
||||
signed deletion event and every reference needed to validate and resolve it.
|
||||
When the target later arrives, the writer resolves its canonical channel and
|
||||
applies the target, tombstone, affected projections, and pending-record
|
||||
resolution atomically, so delete-before-target cannot resurrect a row. A
|
||||
target-before-delete follows the same final-state transaction contract.
|
||||
Channel-less deletion events inherit scope only from the resolved target; the
|
||||
absence of an `h` tag is never treated as evidence that the target is global.
|
||||
Pending tombstones survive crash/restart and replica reopen. Coverage cannot
|
||||
advance across a fetch window while any deletion fact required by its predicate
|
||||
remains unresolved; it may lag until the target is fetched or the relay page
|
||||
proves resolution under an explicit protocol rule. Replica acceptance tests
|
||||
cover target-before-delete and delete-before-target, including a channel-less
|
||||
deletion, with process termination before and after each transaction boundary
|
||||
and successful resolution after restart.
|
||||
|
||||
A renderer delta is emitted only after that transaction commits. Commit without
|
||||
delta is repaired by the next projection read or version invalidation; delta
|
||||
without commit is forbidden.
|
||||
|
||||
### Coverage
|
||||
|
||||
Coverage is a set of proved composite-key ranges, not "highest created_at
|
||||
seen." Channel-window order remains `(created_at DESC, id ASC)` and pagination
|
||||
remains authoritative to relay-signed kind `39006` bounds. Coverage identity
|
||||
also includes every request predicate that changes the answer: community,
|
||||
signer/authorization scope, channel, kind set, auxiliary-closure policy, and
|
||||
wire/projection schema version. A range advances only in the same transaction as
|
||||
every row and auxiliary/tombstone fact needed to answer that exact predicate.
|
||||
On interruption or ambiguity, coverage may lag stored events but must never lead
|
||||
them. Garbage collection removes the oldest complete ranges and invalidates
|
||||
their coverage atomically.
|
||||
|
||||
Reconnect repair is separate from historical page coverage. Until negentropy or
|
||||
a relay ingest cursor exists, the legacy TypeScript path derives replay from
|
||||
`lastSeenCreatedAt`, the maximum event timestamp it has observed. Because the
|
||||
relay accepts events as much as 900 seconds in the future, that cursor can lead
|
||||
relay/commit time by 900 seconds. Channel-bearing live subscriptions must
|
||||
therefore replay by:
|
||||
|
||||
```text
|
||||
relay_ingest_future_tolerance
|
||||
+ CREATED_AT_FLOOR_SECS
|
||||
+ FENCE_CLOCK_MARGIN_SECS
|
||||
= 900 + 960 + 5
|
||||
= 1,865 seconds
|
||||
```
|
||||
|
||||
The first history request has no client-clock-derived upper bound; it pages from
|
||||
the relay's newest matching event down to that floor. Every page transition is
|
||||
lossless under the canonical `(created_at DESC, id ASC)` order. The current
|
||||
TypeScript repair orchestrator uses the existing authenticated Tauri
|
||||
`POST /query` bridge for history pages and supplies the relay's composite
|
||||
`(until, before_id)` continuation. Its next-page predicate is `created_at < until
|
||||
OR (created_at = until AND id > before_id)`. Standard WebSocket Nostr REQ does
|
||||
not expose `before_id`, so timestamp-only WS paging is not an acceptable
|
||||
fallback. A reconnect generation creates one bounded repair dedupe set before
|
||||
restoring the live REQ. Both restored-live delivery and bridge history dispatch
|
||||
consult it before invoking the consumer, and the set remains until live EOSE and
|
||||
the repair pass have both completed or aborted; this proves one callback for
|
||||
either live-before-history or history-before-live overlap rather than relying on
|
||||
downstream cache normalization. A future transport without a composite cursor
|
||||
would need an inclusive, tie-complete boundary drain with explicit event-ID
|
||||
progress; it must neither subtract one second after a full page nor repeatedly
|
||||
accept the same limited page without progress. This rule applies to the
|
||||
unbounded first page too: more than one full page may legally share its oldest
|
||||
`created_at`, and every matching event in that second must be delivered before
|
||||
paging below it. For this paged repair request, the derived or previously pinned
|
||||
replay floor overrides the live subscription's original mount-time `since`;
|
||||
otherwise a backdated event older than the mount timestamp remains invisible
|
||||
despite the wider window. The restored live REQ keeps its original filter.
|
||||
Event IDs are deduped across restored-live and history deliveries before
|
||||
consumer dispatch. The lookback applies only to subscriptions with exactly one
|
||||
`#h` and the full channel event set; it must not widen profile, read-state, or
|
||||
other global live subscriptions. A source-coupling test reads the Rust
|
||||
ingest-envelope, DB-floor, and margin definitions and fails if the TypeScript
|
||||
derived value diverges.
|
||||
|
||||
The future Rust engine may use the narrower
|
||||
`CREATED_AT_FLOOR_SECS + FENCE_CLOCK_MARGIN_SECS` window (currently 965 seconds)
|
||||
only after it records an authoritative sync-time watermark rather than an
|
||||
observed event timestamp. Both forms cover backdated edits and kinds `5`,
|
||||
`9005`, and `40003`; reconnect proof therefore includes a backdated tombstone
|
||||
that would otherwise resurrect a ghost row. The compatibility fix lands now in
|
||||
the existing TypeScript replay loop because the first Rust slice is fan-out
|
||||
only and does not own replay. Parity tests keep the invariant portable when
|
||||
replay later migrates to the native engine.
|
||||
|
||||
The replica hot-zone rule is channel-scoped. The DB fence exempts
|
||||
`channel_id IS NULL`; profiles and other channel-less coordinates use
|
||||
coordinate supersession plus explicit refetch. A channel-less deletion targeting
|
||||
a channel event is applied under the target event's resolved channel, never the
|
||||
deletion's absent `h` tag. Once the native engine uses an authoritative
|
||||
sync-time watermark, a local coverage claim is trustworthy only for reads older
|
||||
than `now - 965s`; the hot zone continues to consult live state. The legacy TS
|
||||
replay cursor remains subject to the wider 1,865-second compatibility bound
|
||||
above. The current five-second replay skew is unsound for accepted backdated
|
||||
events. A monotonic ingest cursor would require explicit relay schema, indexing,
|
||||
and protocol work; `received_at` is not such a cursor.
|
||||
|
||||
### Outbox
|
||||
|
||||
The future outbox uses explicit states: `pending -> publishing -> accepted` or
|
||||
`retryable`, with terminal rejection recorded separately. The signed event and
|
||||
its ID are created once and reused for every retry. Sign, persist the durable
|
||||
outbox row, and apply its optimistic projection in one local transaction before
|
||||
any socket write. Publishing then claims a persisted row through an atomic
|
||||
compare-and-set, so exactly one worker owns an attempt. Relay `OK,true` records
|
||||
acceptance idempotently; echoed live ingest and relay event-ID dedupe reconcile
|
||||
the projection without minting or inserting a second event. Restart treats
|
||||
`publishing` as ambiguous, first reconciles by event ID, then retries the same
|
||||
signed bytes under bounded attempts/backoff. Rebuild or downgrade must preserve
|
||||
pending, publishing, and retryable rows independently of rebuildable
|
||||
`replica.db`. No outbox cutover is combined with read-path cutover.
|
||||
|
||||
### Projection and delta contract
|
||||
|
||||
Projection rows are normalized, versioned pure data, never cached React output.
|
||||
Window deltas are versioned and contain `prepend`, `append`, positioned
|
||||
`insert`, `patch`, and `removeIds`. Backdated events are positioned inserts
|
||||
under NIP-CW ordering, not appends. A version gap, overflow, or unknown geometry
|
||||
invalidates the window and triggers a local projection reread.
|
||||
|
||||
Row preprocessing is incremental and bounded at both temporary receipt points:
|
||||
legacy query response and native engine delta. Track per-switch preprocessing,
|
||||
per-delta work, loaded-window count, mounted DOM, retained nodes, and memory
|
||||
plateau. Markdown cache keys include every parse input that changes output,
|
||||
including mention/channel context and custom emoji, rather than event ID alone.
|
||||
|
||||
## Timeline ownership contracts
|
||||
|
||||
### One scroll owner
|
||||
|
||||
One channel-scoped state machine is the only component allowed to write the
|
||||
active scroller. Virtualizer, pagination, router restoration, targeting, and row
|
||||
measurement submit intents. Its states are:
|
||||
|
||||
- **BOOTSTRAP:** perform initial/deep-link placement, then enter `FOLLOW_TAIL` or
|
||||
`READER`;
|
||||
- **FOLLOW_TAIL:** own true-bottom writes and exit immediately on reader input or
|
||||
loss of tail intent;
|
||||
- **READER:** wheel, touch, and keyboard own position; programmatic writes are
|
||||
forbidden;
|
||||
- **PREPEND_HOLD:** after input/momentum is quiet, preserve an ID plus viewport
|
||||
offset while admitting a pure prepend, then return to `READER`;
|
||||
- **GEOMETRY_HOLD:** preserve an ID plus offset across measured-height change,
|
||||
then return to the prior logical mode; and
|
||||
- **RESTORE:** perform one channel/history restoration, then enter
|
||||
`FOLLOW_TAIL` or `READER`.
|
||||
|
||||
Every programmatic write carries `{channelId, epoch, intentId}`. Its matching
|
||||
scroll event is self-authored and ignored. Real reader input or channel teardown
|
||||
increments the epoch and cancels stale writes, animation frames, measurements,
|
||||
targets, and asynchronous completions immediately.
|
||||
|
||||
The machine covers:
|
||||
|
||||
- initial bottom placement and bottom-follow;
|
||||
- prepend and positioned insert above, inside, or below the viewport;
|
||||
- late growth from media, code highlighting, embeds, reactions, and thread
|
||||
summaries;
|
||||
- deep-link, search, and unread targeting;
|
||||
- real user interruption/cancellation;
|
||||
- channel teardown/reopen and restoration; and
|
||||
- resize, width, zoom, text scale, density, and font changes.
|
||||
|
||||
Current tail buffering, settle-gated prepend (including the current 80 ms
|
||||
input-quiet rule), bottom settle, Virtua `shift`, and anchored-scroll defenses
|
||||
stay in place until the replacement state machine has passed their adversarial
|
||||
cases. The presently encoded correctness ceilings remain hard gates: physical
|
||||
bottom at most 1 px; cascading and settle-gated prepend anchor drift below 5 px
|
||||
with no reversal or snap-to-newest; rich-row resize drift at most 2 px; and
|
||||
mounted rows below 400 in the existing deterministic fixture. Defenses are
|
||||
removed one at a time behind independent flags.
|
||||
|
||||
### Geometry
|
||||
|
||||
A measurement keyed only by event ID is invalid. Geometry identity is never a
|
||||
list index; it includes stable event ID and content/revision, available-width
|
||||
bucket, zoom/text scale and font generation, density and metric-affecting theme
|
||||
inputs, markdown/parser/render-schema version, media/embed reserve and realized
|
||||
state, reaction/aux/thread-summary version, and row chrome/grouping state.
|
||||
Reserve aspect-ratio or bounded geometry for known media and embeds. Unknown or
|
||||
late size changes flow through `ResizeObserver` as `GEOMETRY_HOLD` intents.
|
||||
Before remeasurement capture `{messageId, viewportOffset}` and restore the same
|
||||
ID; if it was removed, choose a deterministic adjacent survivor, never the old
|
||||
index. Invalidate one row for edit/content/aux/media realization, affected
|
||||
neighbors for grouping changes, the channel for width/scale/density/font/parser
|
||||
changes, and all persisted geometry on schema mismatch.
|
||||
|
||||
## Evidence matrix
|
||||
|
||||
There is no packaged macOS WKWebView acceptance runner today. The release smoke
|
||||
serves `desktop/dist` to Desktop Chrome; it is deterministic merge evidence, not
|
||||
proof of the shipped Tauri shell. Chromium Playwright instruments therefore
|
||||
remain frozen diagnostics and regression comparators, not release gates for
|
||||
WKWebView compositor behavior:
|
||||
|
||||
- `cold-switch-longtask.perf.ts`: cold-switch longest/total long tasks;
|
||||
- `warm-switch-markdown.perf.ts`: warm-switch paint and markdown cost;
|
||||
- `typing-latency.perf.ts`: Event Timing and busy/storm deltas;
|
||||
- `scroll-smoothness.perf.ts`: layout/style and prepend cost;
|
||||
- `scrollback-buzzbugs.perf.ts`: pagination latency, payload, and duplicate
|
||||
behavior; and
|
||||
- deterministic timeline/virtualization tests: correctness gates, including the
|
||||
existing physical-bottom, anchor-drift, reversal, input-quiet, buffering, and
|
||||
mounted-row ceilings.
|
||||
|
||||
Milestone 0 must establish a package-equivalent Tauri `.app` runner using release
|
||||
features, sidecars, entitlements, and bundled frontend against an isolated,
|
||||
deterministic relay fixture. This is a real-WKWebView lane, not another
|
||||
Playwright project: the runner contract permits a native macOS automation driver
|
||||
or manual reference run until an automatable driver is selected, but it may not
|
||||
substitute Desktop Chrome evidence. Milestone 0 may land the schemas, launch and
|
||||
fixture seams, environment capture, and a reproducible current-main reference
|
||||
run; full automation is not implied before the native driver exists. Drive
|
||||
native wheel/trackpad and keyboard input at narrow/wide widths and every supported
|
||||
zoom/text-scale setting. Record raw sample JSON with build SHA,
|
||||
package/signing identity, fixture and relay versions, hardware, macOS/WebKit,
|
||||
display refresh, and power/thermal state. The matrix exercises:
|
||||
|
||||
- cold/warm switch and repeated channel teardown/reopen;
|
||||
- prepend during momentum and backdated insert at every viewport position;
|
||||
- append at bottom and while reading;
|
||||
- late media/code/embed/reaction/thread-summary growth;
|
||||
- deep-link/search/unread targets plus user interruption;
|
||||
- reconnect, offline recovery, restart, migration, and injected crash points;
|
||||
- agent-storm IPC/backpressure; and
|
||||
- repeated full traversal/switches until DOM, memory, and disk reach a plateau.
|
||||
|
||||
Collect p50/p95/p99 switch-to-stable-first-paint and input responsiveness,
|
||||
frame/presentation evidence, longest and total long tasks, anchor drift during
|
||||
motion, mounted/retained node counts, native/webview RSS, replica size, and IPC
|
||||
queue/overflow counters. Continuously assert no blank or stale-channel frame,
|
||||
scroll reversal, target yank, or regression of the existing physical-bottom and
|
||||
anchor tolerances. Preserve commit-at-rest and rollback tests until their
|
||||
successor proves the same invariant in packaged WKWebView.
|
||||
|
||||
Baseline current main and each candidate in alternating AB/BA order on a pinned
|
||||
reference device and fixture. Use at least five clean launches and 200 measured
|
||||
observations per build/scenario; cold samples reset relevant state and warm
|
||||
samples get one untimed prime. Report nearest-rank percentiles with paired
|
||||
bootstrap 95% confidence intervals, retain raw samples, and repeat release gates
|
||||
on a second supported macOS/WebKit version. Never pool unlike hardware or OS.
|
||||
Until that baseline exists, correctness ceilings, any unbounded DOM/RSS/disk/IPC
|
||||
slope, and queue overflow are hard failures. A relative performance lane fails
|
||||
only when its paired regression is materially larger than 5% at p50, 10% at
|
||||
p95, or 15% at p99 and the confidence interval excludes parity. Freeze named
|
||||
absolute latency, plateau, disk, and IPC budgets only after repeated packaged
|
||||
runs; do not manufacture them from Chromium.
|
||||
|
||||
## Milestone 0 delivery
|
||||
|
||||
Milestone 0 is exactly two PRs:
|
||||
|
||||
1. **Reconnect correctness:** change the existing TypeScript replay floor from
|
||||
five seconds to the derived, channel-scoped 1,865 seconds required by its
|
||||
event-time cursor (`900 + 960 + 5`), preserving the current batching,
|
||||
pinned-floor, rate-limit, and generation behavior. TypeScript remains the
|
||||
replay owner/orchestrator, but history pages use a narrow Tauri command over
|
||||
the existing authenticated `POST /query` bridge so pagination is tie-complete
|
||||
under `(created_at DESC, id ASC)` with composite `(until, before_id)`
|
||||
continuation. Do not use timestamp-only WebSocket paging or skip the
|
||||
remainder of a full boundary second. The paged repair floor overrides the
|
||||
live filter's mount-time `since`; the restored live REQ itself remains
|
||||
unchanged. Do not use a client-clock `until` for the first page, and dedupe
|
||||
event IDs across restored-live/history overlap before consumer dispatch.
|
||||
Acceptance covers source-constant coupling, exact floor math, a
|
||||
future-dated cursor followed by an older accepted event, backdated kinds
|
||||
`5`/`9005` and `40003`, overlap dedupe, failed replay floor pinning,
|
||||
multi-page replay, an unbounded first page followed by more than one full
|
||||
page sharing one `created_at`, and exclusion of channel-less/global
|
||||
subscriptions.
|
||||
2. **Contracts and baseline harness:** land this document, immutable fixture and
|
||||
result schemas, package-equivalent WKWebView launch/driver seams, environment
|
||||
capture, manifest/report validation tooling, and a reproducible current-main
|
||||
reference run where the available driver supports it. It changes no relay
|
||||
ownership, replica, renderer delta, timeline routing, or authoritative read
|
||||
behavior. Existing deterministic timeline correctness remains gating;
|
||||
Chromium perf specs remain non-gating diagnostics; packaged correctness and
|
||||
boundedness become gating per scenario as the real-WKWebView driver can
|
||||
exercise them, while absolute budgets remain `TBD-by-M0-baseline` until
|
||||
frozen from repeated evidence. An unavailable native driver is recorded as a
|
||||
release-evidence blocker, never silently replaced by Chromium.
|
||||
|
||||
The first implementation PR after milestone 0 is persistent fan-out only. It
|
||||
must not include a replica DB, renderer deltas, timeline routing, or cutover, and
|
||||
must not disturb `set_subscriptions`, immutable subscription/filter identity,
|
||||
reconnect reconciliation, or CLOSED/rate-limit recovery.
|
||||
|
||||
## Milestones
|
||||
|
||||
1. **M0, contracts and proof:** land this contract, baseline current main,
|
||||
establish packaged-app harness seams, and fix reconnect replay independently.
|
||||
2. **M1, native foundation:** first land persistent fan-out only; then add the
|
||||
replica writer/schema in shadow; then compare real channels. No renderer
|
||||
deltas or query routing yet.
|
||||
3. **M2, sidebar:** shadow/compare the projection, flag the local read, retain
|
||||
polling fallback for one release, then remove polling.
|
||||
4. **M3, channel windows:** use `get_channel_window` as the strangler seam;
|
||||
local read inside coverage, bridge on miss; shadow equality before flagged
|
||||
cutover.
|
||||
5. **M4, threads and timeline:** project aux/thread data, land the single scroll
|
||||
owner and geometry contract, then remove existing defenses individually.
|
||||
6. **M5, socket consolidation:** move presence, typing, and huddle subscriptions;
|
||||
ship native-authoritative with renderer fallback before deleting the renderer
|
||||
relay stack.
|
||||
7. **M6, measured optimizations:** negentropy first; ingest cursor, FTS, or LMDB
|
||||
only if evidence justifies their separate relay/storage cost.
|
||||
|
||||
The implementation should prefer the fewest PRs that preserve these review and
|
||||
rollback boundaries. M0 targets one documentation/harness-foundation PR and one
|
||||
focused reconnect-correctness PR. M1 fan-out is independently reviewable from
|
||||
replica persistence because it changes socket delivery semantics and is the
|
||||
prerequisite for attaching shadow ingest safely.
|
||||
Reference in New Issue
Block a user