mirror of
https://github.com/block/buzz.git
synced 2026-10-27 12:40:30 +00:00
test: add deterministic desktop release smoke (#5699)
## Summary - add `just desktop-release-smoke`, a deterministic desktop correctness/reachability smoke against an ephemeral real local relay - preserve existing DM history when the first live DM enters a pageless query window, the desktop-v0.5.10 disappearing-DM regression - enforce foreground JS ordering: a frame and actionable sidebar input must dispatch before mounted stale queries begin resume refetches, while separately requiring the navigation to commit promptly - seed a 10,000-event dense-second fixture and verify exact event-ID reachability, SHA-256 identity, ordering, duplicate absence, bounded mounted rows, and drained render work - isolate Postgres per run, serialize the shared Redis DB, retain phase/relay/Playwright diagnostics, and gate desktop release manifest assembly on the smoke This is deliberately **not a performance-regression gate**. CDP and action timing fields are informational only. There is no candidate/baseline comparison or threshold. A future performance lane needs repeated equivalent fixtures, discrete interaction samples, and an explicit comparator/noise policy. The diagnostics record the fixture version, row count, wall-clock base timestamp (`fixtureSecond`), expected event-ID hash, observed state, and measurements. Because the created-at floor requires a current timestamp, paired comparison remains disabled. The release job runs on an isolated GitHub-hosted runner. The script also guards automatic local runs with a Redis allocation lock. Its remaining direct-PID cleanup and free-port selection race mean it should not be repurposed onto a persistent concurrent shared runner without first hardening process-group cleanup and port reservation. ### Related issue N/A ### Testing - `pnpm --dir desktop typecheck` - focused real-local-relay release smoke passed after adversarial review fixes - identical DM witness passed current and failed `desktop-v0.5.10` with the history-loss signature - identical foreground witness bytes (`2c1e97df04c9b8ca0304b66bbbe9bdb4d08924ad8ce0f68a9c490458fcc3aca8`) failed `desktop-v0.5.10` structurally: the first resume fetch was marker 1, before first frame/sidebar dispatch at marker 8 - with PR #5696 (`59f613c40`) merged, the witness showed focus at 951.3 ms, first frame at 951.6 ms, click dispatch at 952.1 ms, first resume fetch at 968.9 ms, and route commit at 992.4 ms - the gate therefore protects first paint and actionable input dispatch; route commit is a bounded responsiveness witness, not a prerequisite for resume work - the corrected focused foreground scenario passed at `6d9b5be40da58bbee92a856b04c3558946d0a950`; the prior merged-tree full run passed DM retention and 10k reachability before exposing this contract mismatch - pre-push passed on exact pushed head `6d9b5be40da58bbee92a856b04c3558946d0a950`, including desktop checks, typecheck, desktop tests, Rust tests, mobile tests, and Tauri checks - full 10,000-event scenario reached 10,000/10,000 exact IDs with matching SHA-256, 199 continuation requests, and 95 mounted rows in about 4.4 minutes - reduced-row review run passed in 18.4 seconds ### Foreground witness boundary The Chromium test is a deterministic JS policy gate. Headless Chromium does not expose an honest blur/focus transition in this fixture, so the test drives the production focus listener and `document.hasFocus()` predicate together and records that simulation explicitly. It proves refetch fan-out ordering, not AppKit activation, WKWebView paint, or an activating physical click. A packaged macOS native lane is still required before claiming the actual desktop activation experience is certified. --------- Signed-off-by: Wes <wesbillman@users.noreply.github.com> Co-authored-by: Carl <c7ebe626f000404285d3686e1dc74cc07cc60a9754a150041ba132e14bd3e2ec@buzz.block.builderlab.xyz>
This commit is contained in:
60
.github/workflows/release.yml
vendored
60
.github/workflows/release.yml
vendored
@@ -775,6 +775,63 @@ jobs:
|
||||
${{ steps.artifacts.outputs.exe }}
|
||||
${{ steps.artifacts.outputs.sig }}
|
||||
|
||||
desktop-release-smoke:
|
||||
name: Desktop release smoke
|
||||
if: github.repository == 'block/buzz'
|
||||
runs-on: ubuntu-latest
|
||||
needs: setup
|
||||
timeout-minutes: 20
|
||||
permissions:
|
||||
contents: read
|
||||
env:
|
||||
PLAYWRIGHT_BROWSERS_PATH: ${{ github.workspace }}/.cache/ms-playwright
|
||||
steps:
|
||||
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
with:
|
||||
ref: ${{ needs.setup.outputs.source_sha }}
|
||||
persist-credentials: false
|
||||
- uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1
|
||||
- name: Install desktop dependencies
|
||||
run: just desktop-install-ci
|
||||
- name: Get Playwright version
|
||||
id: pw-version
|
||||
run: echo "version=$(cd desktop && node -e \"console.log(require('@playwright/test/package.json').version)\")" >> "$GITHUB_OUTPUT"
|
||||
- name: Restore Playwright browser cache
|
||||
id: playwright-cache
|
||||
uses: actions/cache/restore@caa296126883cff596d87d8935842f9db880ef25 # v5
|
||||
with:
|
||||
path: ${{ env.PLAYWRIGHT_BROWSERS_PATH }}
|
||||
key: playwright-${{ runner.os }}-${{ steps.pw-version.outputs.version }}
|
||||
- name: Install Playwright Chromium
|
||||
if: steps.playwright-cache.outputs.cache-hit != 'true'
|
||||
run: cd desktop && pnpm exec playwright install chromium
|
||||
- name: Install Playwright system dependencies
|
||||
run: cd desktop && pnpm exec playwright install-deps chromium
|
||||
- name: Save Playwright browser cache
|
||||
if: steps.playwright-cache.outputs.cache-hit != 'true'
|
||||
uses: actions/cache/save@caa296126883cff596d87d8935842f9db880ef25 # v5
|
||||
with:
|
||||
path: ${{ env.PLAYWRIGHT_BROWSERS_PATH }}
|
||||
key: playwright-${{ runner.os }}-${{ steps.pw-version.outputs.version }}
|
||||
- name: Build test relay
|
||||
run: cargo build --profile ci -p buzz-relay
|
||||
- name: Run deterministic correctness smoke
|
||||
env:
|
||||
BUZZ_E2E_RELAY_BIN: ${{ github.workspace }}/target/ci/buzz-relay
|
||||
BUZZ_RELEASE_SMOKE_ARTIFACT_DIR: ${{ github.workspace }}/release-smoke-artifacts
|
||||
run: just desktop-release-smoke
|
||||
- name: Upload release-smoke diagnostics
|
||||
if: always()
|
||||
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
||||
with:
|
||||
name: desktop-release-smoke
|
||||
path: |
|
||||
release-smoke-artifacts
|
||||
desktop/test-results
|
||||
desktop/playwright-release-smoke-report
|
||||
if-no-files-found: warn
|
||||
retention-days: 14
|
||||
|
||||
assemble-manifest:
|
||||
name: Assemble multi-platform latest.json
|
||||
# Only the tag-bound setup path can reach this job.
|
||||
@@ -785,9 +842,10 @@ jobs:
|
||||
needs.release-macos-x64.result == 'success' &&
|
||||
needs.release-linux.result == 'success' &&
|
||||
needs.release-windows.result == 'success' &&
|
||||
needs.desktop-release-smoke.result == 'success' &&
|
||||
github.ref == format('refs/tags/desktop-v{0}', needs.setup.outputs.version)
|
||||
runs-on: ubuntu-latest
|
||||
needs: [setup, release, release-macos-x64, release-linux, release-windows]
|
||||
needs: [setup, release, release-macos-x64, release-linux, release-windows, desktop-release-smoke]
|
||||
timeout-minutes: 10
|
||||
permissions:
|
||||
contents: write
|
||||
|
||||
Reference in New Issue
Block a user