mirror of
https://github.com/block/buzz.git
synced 2026-11-08 02:11:36 +00:00
64144d2f4f71a2f32884ae31b07b06de4e7b762e
8 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Dawn
|
64144d2f4f |
perf(desktop): move the local-archive subscription into Rust
The renderer owned the whole archive path: it listed saved subscriptions, opened a live REQ per scope, buffered inbound frames, batched them, and handed each batch back over IPC to be written to SQLite. Every archived frame therefore made a round trip into JS for no reason other than history — nothing in that path is a rendering concern. `archive::sync` now runs the whole pipeline natively. It subscribes from the saved subscriptions, buffers, flushes on the same thresholds the JS manager used (FLUSH_BATCH_SIZE 25 / 2000ms deadline — parity, not a retune), archives via SQLite directly, and emits `archive-agent-metrics-changed` when a batch actually persisted metric rows. `archiveSyncManager.ts` is deleted. `native_relay_client` is the shared piece underneath: one authenticated socket per (relay, pubkey), multiplexed subscriptions, declarative `set_subscriptions` reconciliation, and exponential reconnect backoff. A 30s read timeout is idle, not failure, and is discriminated by error variant rather than message text so a reworded error cannot turn every quiet period into a reconnect storm. A relay CLOSED drives its own recovery: per-id retry state lives next to `open` in the connection (not in `desired`, which is reloaded from SQLite and would resurrect a deletion), a dedicated select! deadline arm fires the reopen and is disabled when nothing is scheduled, and the CLOSED message is classified terminal / rate-limited / retryable with the same prefixes the renderer used, `auth-required:` deliberately retryable. Rate-limited arms the shared relay_admission gate and waits max(backoff, hint); backoff is 1s→30s saturating; attempts reset on EVENT or EOSE. Terminal suppression is per-socket by design: a reconnect retries a terminal id once through the normal path, because relay policy can change and one REQ per reconnect is bounded. The sync lifecycle is owned, not raced. The renderer allocates a monotonic lease synchronously in effect order — intent order, which IPC completion order is not — and Rust ignores any start/stop older than the highest (epoch, lease) mark it has seen; a stop advances the mark, so a delayed start cannot resurrect a stopped task. Above the lease, Rust mints a realm epoch, published atomically with minting under the same lock that orders lifecycle calls: announcing IS what supersedes the previous realm, so a separately-held counter would leave a window in which a dead realm's delayed calls still win. The renderer awaits the epoch before its first lifecycle command. Ownership is main-window-only via the established huddleWindowChannelId() exclusion: a huddle companion mounts the same tree in a concurrent realm, and concurrent owners cannot be ordered by any newest-wins clock. What stays in JS is the start gate, deliberately. Kind 24200 is relay-ephemeral, so frames emitted before the listener opens are lost permanently, and only the renderer knows when observer reconciliation finished seeding 24200 into the saved subscription. The backend task is therefore not self-starting: `useArchiveSync` starts it once reconciliation resolves and stops it on unmount. `RelaySession.revision` was documented as rejecting stale in-flight reconciliation but never did — removed rather than repaired: declarative reconciliation re-reads the desired set every pass, so there is no generation to guard. Tests: archive/sync_tests.rs drives the real run_sync body through a fake IO seam (filter parity, flush thresholds, failure isolation) plus the ownership contract (out-of-order start/stop both directions, announcement supersedes a dead realm's delayed calls before any new lifecycle call, publish-(epoch,0) does not lock out the announcing realm). native_relay_client's stub-relay test completes the NIP-42 handshake over a real TCP socket, injects CLOSED with the desired set unchanged, and proves the REQ is retried by the deadline arm — the relay-backed #[ignore] test additionally proves the REQ shape against a real relay. useArchiveSync.test.mjs owns the start gate, realm ownership, and post-reload realm supersession via fresh module instances. observer-archive-policy.spec.ts owns wiring, with payload receipts that announce precedes start and start carries a numeric epoch. Every claim was mutation-checked; the vacuous first drafts (wake-masked reopen, policy re-implementation, precondition-rebuilding supersession, same-scope no-op escape) were each caught by their mutants and rewritten. Includes one move-only hunk that is not archive work: the push-to-talk global-shortcut handler moves from lib.rs into `ptt_shortcut::install`, mirroring the existing `app_menu::install` seam, paying the 1000-line ratchet budget in the module that owns the registration lifecycle. The handler body is proven token-identical with a mutated-body negative control. lib.rs is 917 lines. Co-authored-by: Tyler Longwell <tlongwell@squareup.com> Signed-off-by: Tyler Longwell <tlongwell@squareup.com> |
||
|
Will Pfleger
|
5e4c05f90b |
feat(desktop): NIP-AM agent-usage backend — P2 emission/transport/archive + P4a aggregation/D6 (#4000)
## What Implements Phases 2 and 4a of the Usage v2 plan (plan events `d0268cd0`/`0e95b035`), extending the archive backend to emit, transport, archive, and aggregate both cache categories and billing identity fail-closed. ### P2 — emission, transport, archive **Tri-state accumulators** (`Unseen`/`Exact`/`Unknown`) for cache-read and cache-write in `buzz-agent` turn and session state. Absent field = Unknown (never zero) through the full pipeline. No `unwrap_or(0)` on the cache path. Both cache folds are gated on usage-bearing responses (same gate as the total-state and identity folds) — a response with no usage at all must not poison either accumulator. **Overflow-aware input token parsing and accumulation** — closed end-to-end from parse through wire to ACP: - `sum_usage()` returns `SumUsageResult` (`Exact(u64)` | `Overflow`) — checked arithmetic, never clamps. `anthropic_input_tokens()` returns `Option<SumUsageResult>` since it sums three fields (`input_tokens + cache_read_input_tokens + cache_creation_input_tokens`) that can collectively overflow. Single-field callers (`prompt_tokens`, `completion_tokens`, etc.) convert via `.into_exact()` — their single-field sums cannot overflow. - `LlmResponse.input_tokens_overflowed: bool` propagates the parse-layer signal into the run loop. When set, `input_tokens` is `None` (clamped value discarded), the context-gate baseline (`last_request_input_tokens`) is frozen at its prior reading, and `turn_input_tokens` is poisoned to `TurnIOState::Poisoned` before any emission — including mid-turn `emit_usage_update` calls. A dedicated enum on `LlmResponse.input_tokens` would ripple into ~20 existing test assertions on `r.input_tokens == Some(...)`; the bool flag confines the change to the two call sites that check it. - `TurnIOState` (`Unseen`/`Exact`/`Poisoned`) for input and output: per-round fold uses `checked_add`; overflow poisons permanently at turn and session level, no healing. Absence does not poison (pass-2-cleared contract unchanged). Wire emission omits `accumulatedInputTokens`/`accumulatedOutputTokens` when poisoned — never null, never `u64::MAX`. ACP treats absent = publisher-poisoned: `delta_reliable: false`, null turn fields, null cumulative for that category; session cumulative stays unknown for all subsequent turns once poisoned. **Conditional wire emission** for `accumulatedCachedInputTokens` and new `accumulatedCacheWriteTokens`: fields are omitted when the cumulative is Unseen or Unknown. ACP `_goose/unstable/session/update` contract documented next to the payload with tests for all absence/zero variants. **`PricingIdentity` stamping (publisher-side)**: - `pricing_authority()`: canonical parsed-URL endpoint comparison against the official allowlist — HTTPS only, exact allowlisted host (lookalike-safe), default port (omitted or explicit :443), required API base path, rejects userinfo/query/fragment/path-prefix lookalikes. - Model: the actually-requested `request_model` after mesh/auto resolution (not `effective_model_str`). - Turn discipline: identity retained only while ALL usage in the current turn carries one identical proven identity; any mismatch, unproven-usage-bearing response, or unpaired cumulative snapshot poisons to absent; a later matching notification does not heal a mixed turn. **ACP `UsageTracker` identity fold**: per-in-flight-turn tri-state identity accumulator replacing last-update-wins. Any absent identity on a token-advancing notification or exact mismatch poisons to absent; poison survives later updates; reset in `begin_turn()`/`take()`; reset also when a request fails (baseline cleared so preflight gate cannot stay frozen sub-threshold on retries). **M3 migration**: adds `turn_cache_write_tokens`, `cumulative_cache_write_tokens`, `pricing_authority`, `pricing_model`, `pricing_cache_class` to `agent_metric_index`. Additive, idempotent, guarded per-column by marker. M2 migration also guarded per-column (turn and cumulative cache-read columns checked and added independently; marker commits only after both are present). Fresh-DB schema includes all columns. **First-turn baselines**: `seed_zero_baseline` seeds `last_input: Some(0)`, `last_output: Some(0)`, `last_cached_input: Some(0)`, `last_cache_write: Some(0)`, and `last_total: Some(0)` — all have the known-zero-at-spawn argument. Absent fields from incoming snapshots still produce unknown (tri-state unchanged). Sessions buzz-acp did not spawn (no seed) remain fail-closed on turn one. **`ReportedUsage` TS mirror**: `cacheReadTokens`, `cacheWriteTokens`, `freshInputTokens` added to `tauriArchive.ts` as `UsageField` members, field-for-field with the Rust struct. ### P4a — aggregation layer **Extended S-1 ladder** to cache-read and cache-write via the same `ladder_token` path as the existing token fields. **`freshInputTokens` derivation**: checked arithmetic, fail-closed — absent cache fields produce Unknown (not zero), overflow and `cacheRead+cacheWrite > input` both produce `incomplete: true`. Aggregated as a `UsageField`. **D6 comparator**: `sort_value()` = provider total when known, else `input+output` when both known, else `None` (unknown-last). Replaces the prior total-only comparator for both agent-level and model-level sort. Ships a pinned test vector that the TS render layer (P5) must match. ## Test coverage - `buzz-agent`: 440 lib + 15 integration (golden_transcripts) — includes 13 new `cache_total_state_tests`; 14 new `turn_io_state_tests`; 3 new `sum_usage_*` tests (exact single-field, exact two-field, overflow signals correctly); 3 new `parse_anthropic_*` tests (overflow flag set + value cleared, normal sum no flag, absent usage no flag); end-to-end golden transcript drives real subprocess with Anthropic-shaped `input_tokens: u64::MAX, cache_read: 1` response and asserts `accumulatedInputTokens` absent from the emitted `usage_update` — no logic duplication; 3 wire pin tests; 4 `fold_pricing_identity_*` tests; `pricing_authority()` explicit-:443 acceptance - `buzz-acp`: 700 tests (691 lib + 9 integration) — 4 new usage tests (absent input → unreliable+null; absent output → unreliable+null; goose-shaped both present unchanged; poison mid-session); 3 ACP behavior tests; 7 pool lifecycle tests - Desktop (Rust): 2259+ tests — 14 new P4a pinned tests; 2 M3 round-trip tests; 1 serde key-shape test; 2 M2 partial-schema migration tests; first-turn cache round-trip test ## Related PRs - P1 NIP-AM spec: [#4632](https://github.com/block/buzz/pull/4632) - P3 pricing table: [#4629](https://github.com/block/buzz/pull/4629) - UI (P5): [#4001](https://github.com/block/buzz/pull/4001) --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> |
||
|
Will Pfleger
|
5179726737 |
fix(local-archive): default both archive settings to enabled (#4750)
## Overview
Both local archive settings — "Archive my agents' observer frames" (kind
24200) and "Archive my agents' turn metrics" (kind 44200) — previously
defaulted to OFF in OSS builds, controlled by build-time env vars. This
had an irreversible cost: observer frames are ephemeral (not stored by
the relay), so any missed events are permanently unrecoverable. This PR
makes both settings default to enabled for all builds and removes the
build-time flag machinery entirely.
## What changed
### Rust
- `observer_archive_default_enabled()` — returns `true` unconditionally;
removed `option_env!("BUZZ_DESKTOP_BUILD_OBSERVER_ARCHIVE_DEFAULT")`
check and `nest_is_dev()` runtime fallback.
- `agent_metric_archive_default_enabled()` — returns `true`
unconditionally; removed
`option_env!("BUZZ_DESKTOP_BUILD_AGENT_METRIC_ARCHIVE_DEFAULT")` check
and its OSS-build test.
- `build.rs` — removed both `rerun-if-env-changed` declarations
(`BUZZ_BUILD_OBSERVER_ARCHIVE_DEFAULT`,
`BUZZ_BUILD_AGENT_METRIC_ARCHIVE_DEFAULT`) and the two baked-env
emitting blocks.
### Build / CI
- `Justfile` — removed `desktop-tauri-test-compiled-flags` recipe (the
dual-compile test machinery).
- `.github/workflows/ci.yml` — removed the "Desktop Tauri compiled-flag
verification" CI step.
### TypeScript
- `useObserverArchiveSeed.ts` — removed `observerArchiveDefaultEnabled`
dep from `ObserverArchiveSeedDeps` and the `policyOn` gate in
`reconcileObserverArchive`; the function now unconditionally calls
`mergeSaveSubscriptionKinds`.
- `useAgentMetricArchiveSeed.ts` — removed
`agentMetricArchiveDefaultEnabled` dep from `AgentMetricArchiveSeedDeps`
and the `defaultOn` flag-check path in `maybeSeed`; the
`hasExplicitChoice` guard is preserved as the sole gate against
re-seeding.
- `LocalArchiveSettingsCard.tsx` — removed `policy` prop,
`observerPolicy` state, and `observerArchiveDefaultEnabled` fetch from
`ObserverArchiveSection`; toggle is now always enabled (just `toggling`
disables it); removed the stale "Always on for internal builds" copy
branch; removed the `observerPolicy !== false` guard from
`handleObserverToggle`.
- `tauriArchive.ts` — updated JSDoc on both default-enabled functions to
reflect always-true.
- `e2eBridge.ts` — changed both mock defaults from `?? false` to `??
true` so E2E tests without an explicit mock override exercise the real
default behavior.
### Tests
- `useObserverArchiveSeed.test.mjs` — replaced `policyOn` dep with
direct merge dep; updated `test_oss_policy_off_no_merge` →
`test_reconcile_always_seeds_24200`; all cancellation, identity-switch,
and ordering tests adapted.
- `useAgentMetricArchiveSeed.test.mjs` — removed `defaultOn` dep and
`test_oss_build_does_not_seed`; updated
`test_internal_build_unset_seeds_*` → `test_default_enabled_*`;
`hasExplicitChoice` guard tests unchanged.
## Preservation of explicit opt-outs
Users who have previously toggled the setting off are unaffected:
- `useAgentMetricArchiveSeed` skips seeding when
`hasExplicitChoice(pubkey)` returns true (localStorage-persisted per
identity).
- Observer archive reconciliation now unconditionally calls
`mergeSaveSubscriptionKinds`, but a user who already deleted the
subscription can turn it off via the Settings toggle, which calls
`removeSaveSubscriptionKind` — this is the existing explicit opt-out
path, and the toggle is now always enabled (not locked by a policy
flag).
## Result
- No `BUZZ_BUILD_*_ARCHIVE_DEFAULT` /
`BUZZ_DESKTOP_BUILD_*_ARCHIVE_DEFAULT` references remain outside
CHANGELOG/history.
- Desktop node tests: 4168 pass, 0 fail.
- `just desktop-tauri-check`: clean.
- `just desktop-tauri-test`: all pass.
---------
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
|
||
|
Will Pfleger
|
8969745549 |
fix(desktop): enforce observer archive policy reconciliation on internal builds (#1923)
Signed-off-by: Will Pfleger <pfleger.will@gmail.com> |
||
|
Will Pfleger
|
fd5d04de05 |
fix(desktop): scope observer feed by channel and add session-boundary dividers (#1634)
Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@sprout-oss.stage.blox.sqprod.co> |
||
|
Will Pfleger
|
484bede759 |
fix(archive): atomic remove-kind path + split test modules (#1562)
Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: npub1fgdl5qqnh3k3f2xkqrvt7cujalhm623x4s7fdjdj5yrtp5fzjl9qrjpucw <4a1bfa0013bc6d14a8d600d8bf6392efefbd2a26ac3c96c9b2a106b0d12297ca@sprout-oss.stage.blox.sqprod.co> |
||
|
Will Pfleger
|
429f54b4f4 |
feat(archive): add agent turn-metric (kind 44200) local archive (#1555)
Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@sprout-oss.stage.blox.sqprod.co> |
||
|
Will Pfleger
|
7110788077 |
feat(archive): add local-save archive with observer-feed default-on option (#1442)
Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@sprout-oss.stage.blox.sqprod.co> |