test: add deterministic desktop release smoke (#5699)

## Summary

- add `just desktop-release-smoke`, a deterministic desktop
correctness/reachability smoke against an ephemeral real local relay
- preserve existing DM history when the first live DM enters a pageless
query window, the desktop-v0.5.10 disappearing-DM regression
- enforce foreground JS ordering: a frame and actionable sidebar input
must dispatch before mounted stale queries begin resume refetches, while
separately requiring the navigation to commit promptly
- seed a 10,000-event dense-second fixture and verify exact event-ID
reachability, SHA-256 identity, ordering, duplicate absence, bounded
mounted rows, and drained render work
- isolate Postgres per run, serialize the shared Redis DB, retain
phase/relay/Playwright diagnostics, and gate desktop release manifest
assembly on the smoke

This is deliberately **not a performance-regression gate**. CDP and
action timing fields are informational only. There is no
candidate/baseline comparison or threshold. A future performance lane
needs repeated equivalent fixtures, discrete interaction samples, and an
explicit comparator/noise policy.

The diagnostics record the fixture version, row count, wall-clock base
timestamp (`fixtureSecond`), expected event-ID hash, observed state, and
measurements. Because the created-at floor requires a current timestamp,
paired comparison remains disabled.

The release job runs on an isolated GitHub-hosted runner. The script
also guards automatic local runs with a Redis allocation lock. Its
remaining direct-PID cleanup and free-port selection race mean it should
not be repurposed onto a persistent concurrent shared runner without
first hardening process-group cleanup and port reservation.

### Related issue

N/A

### Testing

- `pnpm --dir desktop typecheck`
- focused real-local-relay release smoke passed after adversarial review
fixes
- identical DM witness passed current and failed `desktop-v0.5.10` with
the history-loss signature
- identical foreground witness bytes
(`2c1e97df04c9b8ca0304b66bbbe9bdb4d08924ad8ce0f68a9c490458fcc3aca8`)
failed `desktop-v0.5.10` structurally: the first resume fetch was marker
1, before first frame/sidebar dispatch at marker 8
- with PR #5696 (`59f613c40`) merged, the witness showed focus at 951.3
ms, first frame at 951.6 ms, click dispatch at 952.1 ms, first resume
fetch at 968.9 ms, and route commit at 992.4 ms
- the gate therefore protects first paint and actionable input dispatch;
route commit is a bounded responsiveness witness, not a prerequisite for
resume work
- the corrected focused foreground scenario passed at
`6d9b5be40da58bbee92a856b04c3558946d0a950`; the prior merged-tree full
run passed DM retention and 10k reachability before exposing this
contract mismatch
- pre-push passed on exact pushed head
`6d9b5be40da58bbee92a856b04c3558946d0a950`, including desktop checks,
typecheck, desktop tests, Rust tests, mobile tests, and Tauri checks
- full 10,000-event scenario reached 10,000/10,000 exact IDs with
matching SHA-256, 199 continuation requests, and 95 mounted rows in
about 4.4 minutes
- reduced-row review run passed in 18.4 seconds

### Foreground witness boundary

The Chromium test is a deterministic JS policy gate. Headless Chromium
does not expose an honest blur/focus transition in this fixture, so the
test drives the production focus listener and `document.hasFocus()`
predicate together and records that simulation explicitly. It proves
refetch fan-out ordering, not AppKit activation, WKWebView paint, or an
activating physical click. A packaged macOS native lane is still
required before claiming the actual desktop activation experience is
certified.

---------

Signed-off-by: Wes <wesbillman@users.noreply.github.com>
Co-authored-by: Carl <c7ebe626f000404285d3686e1dc74cc07cc60a9754a150041ba132e14bd3e2ec@buzz.block.builderlab.xyz>
This commit is contained in:
Wes
2026-08-13 15:49:40 -06:00
committed by GitHub
parent e0940927ff
commit 76f114a252
10 changed files with 1008 additions and 1 deletions

View File

@@ -775,6 +775,63 @@ jobs:
${{ steps.artifacts.outputs.exe }}
${{ steps.artifacts.outputs.sig }}
desktop-release-smoke:
name: Desktop release smoke
if: github.repository == 'block/buzz'
runs-on: ubuntu-latest
needs: setup
timeout-minutes: 20
permissions:
contents: read
env:
PLAYWRIGHT_BROWSERS_PATH: ${{ github.workspace }}/.cache/ms-playwright
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
ref: ${{ needs.setup.outputs.source_sha }}
persist-credentials: false
- uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1
- name: Install desktop dependencies
run: just desktop-install-ci
- name: Get Playwright version
id: pw-version
run: echo "version=$(cd desktop && node -e \"console.log(require('@playwright/test/package.json').version)\")" >> "$GITHUB_OUTPUT"
- name: Restore Playwright browser cache
id: playwright-cache
uses: actions/cache/restore@caa296126883cff596d87d8935842f9db880ef25 # v5
with:
path: ${{ env.PLAYWRIGHT_BROWSERS_PATH }}
key: playwright-${{ runner.os }}-${{ steps.pw-version.outputs.version }}
- name: Install Playwright Chromium
if: steps.playwright-cache.outputs.cache-hit != 'true'
run: cd desktop && pnpm exec playwright install chromium
- name: Install Playwright system dependencies
run: cd desktop && pnpm exec playwright install-deps chromium
- name: Save Playwright browser cache
if: steps.playwright-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@caa296126883cff596d87d8935842f9db880ef25 # v5
with:
path: ${{ env.PLAYWRIGHT_BROWSERS_PATH }}
key: playwright-${{ runner.os }}-${{ steps.pw-version.outputs.version }}
- name: Build test relay
run: cargo build --profile ci -p buzz-relay
- name: Run deterministic correctness smoke
env:
BUZZ_E2E_RELAY_BIN: ${{ github.workspace }}/target/ci/buzz-relay
BUZZ_RELEASE_SMOKE_ARTIFACT_DIR: ${{ github.workspace }}/release-smoke-artifacts
run: just desktop-release-smoke
- name: Upload release-smoke diagnostics
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: desktop-release-smoke
path: |
release-smoke-artifacts
desktop/test-results
desktop/playwright-release-smoke-report
if-no-files-found: warn
retention-days: 14
assemble-manifest:
name: Assemble multi-platform latest.json
# Only the tag-bound setup path can reach this job.
@@ -785,9 +842,10 @@ jobs:
needs.release-macos-x64.result == 'success' &&
needs.release-linux.result == 'success' &&
needs.release-windows.result == 'success' &&
needs.desktop-release-smoke.result == 'success' &&
github.ref == format('refs/tags/desktop-v{0}', needs.setup.outputs.version)
runs-on: ubuntu-latest
needs: [setup, release, release-macos-x64, release-linux, release-windows]
needs: [setup, release, release-macos-x64, release-linux, release-windows, desktop-release-smoke]
timeout-minutes: 10
permissions:
contents: write