Commit Graph

5 Commits

Author SHA1 Message Date
Tom Brow
868e2f8f32 refactor(push): one app profile per gateway (#8206)
Buzz push currently selects an application profile, such as `dogfood`,
during client registration. We want each gateway to serve one Apple app
configured by the operator instead. This removes profile selection from
the client, relay, and gateway, while allowing internal and public
gateways to use their own Apple identities.

This PR implements that model in the Buzz source and upstream deployment
chart. Discovery, enrollment, grants, and leases no longer carry an
application-profile selector. Each gateway has one configured Apple
application and APNs transport.

The protocol changes are incompatible with the old registration format,
so relays advertise `buzz-push-v1` instead of `nip-pl`. Old clients do
not discover the new push capability. Updated clients use separate
Keychain records and register again. The [push
notes](https://github.com/block/buzz/blob/push-capability-v1/docs/buzz-push.md)
describe the experimental contract.

Landing this PR is the first step. Publish the matching gateway image
and upstream chart, then land the [shared chart
update](https://github.com/squareup/cash-helm-charts/pull/5740) and
[gateway deployment
update](https://github.com/squareup/builderbot-platform-core-infrastructure/pull/1085).
Before cutover, stop old gateway instances and check legacy registration
state. Migrations refuse to proceed if old gateway installations or
active relay leases remain; they do not delete that state.
Physical-device verification remains necessary before production
activation.

### Validation
Gateway and focused Flutter push tests pass, as do the Swift package,
simulator build, and native notification/enrichment tests. A local relay
advertises `buzz-push-v1` without `nip-pl` or `app_profiles`. The
PostgreSQL suite passed 868 tests; its two migration-test failures were
corrected and pass a focused rerun together with the gateway cutover
guard. Chart rendering and release-contract checks pass.

Desktop JavaScript tests, desktop and web builds, native desktop
compilation, mobile analysis/tests, and desktop native tests pass. The
full Rust unit lane and prescribed `just test` command pass, including
workspace integration tests against task-isolated local services. The
production overlay now requires an explicitly supplied APNs topic, with
omission and rendered-value regression coverage. The corrected Swift
checksum assertion passes the complete package suite: 78 XCTest tests
and 42 Swift Testing tests. No physical-device enrollment or production
cutover is claimed.

Local merge validation: the embedded migration test and formatting pass
after reserving profile migration 0060 behind rollback migration 0059.
The local pre-push Rust lane failed ACP timing/steering tests
(`idle_resets_on_stdout_activity`,
`keepalive_resets_idle_past_deadline`, and the failed/missing-outcome
steering tests). ACP and its dependency manifests are unchanged by this
branch; all four pass in an isolated single-thread rerun. The documented
local-check exception skips only the Rust pre-push lane for publication;
required CI remains mandatory. Mobile and desktop pre-push lanes passed.

Combined migration validation: after rollback 0059 landed, the main
merge preserves profile migration 0060 and both embedded migration
assertions. The focused migration test and formatting pass. The next
local push again failed only the unchanged ACP idle/keepalive timing
tests (1053 passed, 2 failed, 3 ignored); other lanes passed.
Publication uses the same documented local Rust-lane exception; required
CI remains mandatory.

Follow-up review fix: removed the legacy profile column from the
copied-lease fixture. All four event-follow-up contract tests pass on
isolated PostgreSQL databases, covering desired and migrated schemas in
dual and app-only modes. Formatting passes. Publication retains the
prior passing local-lane evidence and uses the authorized pre-push
exception for the known unrelated ACP timing failures; required CI
remains mandatory.

---------

Signed-off-by: Tom Brow <tomb@block.xyz>
2026-10-09 15:09:53 -07:00
Tom Brow
5b907b838a fix(push): isolate push enqueue and stop workers on rollback (#8216)
Push enqueueing previously ran inside the message transaction, so a push
lock or database error could block or reject a message. Configured relay
writers now submit bounded best-effort work only after commit. A
dedicated connection and bounded queue isolate enqueue waits and
failures from message-write connections. Overload, errors, process loss,
or shutdown may drop a notification; the committed message remains
authoritative.

`BUZZ_PUSH_ENABLED=false` disables the application producer and workers.
Updated writer connections always bypass any surviving enqueue trigger.
The overlap migration updates an existing function without resurrecting
retired objects. Apply it before relying on isolation with a surviving
trigger, and replace every old enabled process and writer connection:
missing settings preserve legacy behavior during overlap.

SIGTERM/Ctrl-C cancels enqueue, matcher, and delivery work before
listener drain. Durable queues, claims, leases, preferences, and
delivery configuration remain intact. Claims expire normally; gateway
requests already accepted may still deliver. Reactivation requires an
explicit operator decision about retained work. There is no automatic
purge, startup interlock, or backfill of messages written while
disabled.

Built on merged [#8135](https://github.com/block/buzz/pull/8135),
including [#8178](https://github.com/block/buzz/pull/8178). Trigger
retirement [#8210](https://github.com/block/buzz/pull/8210) remains
open; this change is tested with the trigger present and after removal.
Operational details are in [the rollback and reactivation
guide](https://github.com/block/buzz/blob/relay-push-rollback/docs/push-gateway-deployment.md#relay-rollback-and-reactivation).

### Validation


[9d3d710](9d3d710dc1)
changes documentation and Rust comments only, with no executable or
assertion changes. Formatting and diff checks passed;
[12f1d1e](12f1d1e814)
additionally restores the reviewer-requested caveat that active lease
renewal may suppress a notification. This is also documentation-only;
Owner approval was confirmed on 12f1d1e.
[a4f602d](a4f602df49)
subsequently merges main to resolve a migration-count conflict after
main added migration 0058. Both 0058 and rollback migration 0059 are
retained; the focused embedded-migrator test, formatting, and applicable
desktop pre-push checks passed. Current-head [owner
approval](https://github.com/block/buzz/pull/8216#pullrequestreview-5474958334)
is confirmed. [CI attempt 2
passed](https://github.com/block/buzz/actions/runs/37985516452/attempts/2),
with required checks passing or appropriately skipped. The first attempt
failed only the unchanged replica-floor probe test with `MaskedActivity
{ masked: 1 }`; one failed-jobs retry passed without code changes. All
four push contract tests passed. The current-head [advisory security
review](https://github.com/block/buzz/actions/runs/37985514346/job/114006457769)
has remained in its review step since 20:18:51 UTC without a published
result for over 25 minutes. It is stalled under the advisory-review
policy, not a clean security review.

[Required CI checks passed or were
skipped](https://github.com/block/buzz/actions/runs/37964035783) on
[3164b25](3164b25cde).
The advisory security review was cancelled without a published result.
Codex's durable-retry suggestion was [rebutted and
resolved](https://github.com/block/buzz/pull/8216#discussion_r4232674980)
under the explicit best-effort notification contract. The [lease
activation-order
observation](https://github.com/block/buzz/pull/8216#discussion_r4233051799)
is correct. Under the clarified product requirement, either notification
outcome is acceptable during overlapping registration and message
arrival. Exact commit ordering is not required; deliberate historical
backfill remains prohibited. The bot thread is resolved against that
clarification; [Owner review accepted the clarified overlap
contract](https://github.com/block/buzz/pull/8216#pullrequestreview-5474720278)
on 9d3d710.

All 488 PostgreSQL tests passed. The final four-way producer contracts
passed on desired/migrated schemas, with/without triggers. Added
coverage verifies message commit under push-gate and queue-table locks,
recovery after enqueue failure, in-flight enqueue cancellation, the
lease timestamp filter (not production activation ordering),
serving-lease writers, rollback preservation, reconnects, and
reactivation without backfill. Queue saturation and all three relay
worker shutdown tests passed. Rust lint and all 17 source-policy checks
passed.

Full local CI/repository runs encountered the unchanged ACP
`keepalive_resets_idle_past_deadline` timing failure. Earlier desktop
subprocess timing tests also failed under parallel load and passed as an
isolated module. Under Tom's local pre-push exception, this update
excluded `rust-tests` and `desktop-tauri-checks`; remaining hooks stayed
enabled. Required CI validation remains necessary before merge. A
focused run competing with the full database suite hit wall-clock
deadlines; tests now use a short database lock timeout to distinguish
unwanted push dependencies, and their final isolated run passed.

A local process-replacement rehearsal on 3164b25 used the real relay
binary and a mock gateway holding a delivery request open. SIGTERM
closed that request in under 1 ms; the relay exited in 5.02 seconds.
Restarting the same binary with push disabled produced no further
gateway requests over 35 seconds, beyond the 30-second claim expiry. The
outbox row, lease, and its subscriptions remained byte-for-byte
unchanged. This used a seeded outbox and did not exercise APNs or a
mobile device.

Production writer inventory, shutdown deadline, and retained-queue
disposition still require operational validation before rollout. The PR
is marked ready for review. The overlap behavior is accepted by the
product requirement; owner review accepted that contract. No production
deployment has occurred.

---------

Signed-off-by: Tom Brow <tomb@block.xyz>
2026-10-09 13:48:06 -07:00
Bradley Axen
1e80066c93 feat(relay): reshape read-state API to the sidebar wire format (#8220)
## Summary

Changes the private read-state API (`/buzz/v1`) to the sidebar wire
format agreed in #buzzin-in-secret. **The counting code is unchanged.**
Requests and responses take the new shape, so the client integration can
build against it now and later behavior fixes (counting rules, followed
threads, another-device pings) ship without another wire change. Nothing
ships against v1 yet (no buzz-app use on `main`), so this replaces
version 1 directly.

Supersedes #8188 (same branch content, rebased onto current `main`, plus
the thread-cap and doc changes below).

```
GET /buzz/v1/me/sidebar?limit=&cursor=  |  ?channel_ids=a,b,…
→ { channels: [channel], next_cursor }
channel: { channel_id, unread: bool, mentions, read_through_id|null, latest_id|null, threads: [thread] }
thread:  { root_id, unread: bool, mentions, read_through_id|null, latest_id }

POST /buzz/v1/me/read-state
{ intents: [{ type: "mark_through", target: { channel_id [, root_id] }, message_id }] }
→ { outcomes: [applied|blocked|invalid|unknown], channels: [channel] }
```

What changed and why:
- **Channel and thread positions are fully separate.** Channel
`unread`/`mentions` count only top-level messages. Thread replies count
only on their thread row. `mark_channel_read` and its whole-channel cut
are gone. "Mark channel read" sends a channel `mark_through` at
`latest_id`, plus one per thread row to clear threads too.
- **Channel `latest_id` is always a top-level message.** It used to be
able to point at a reply, and a channel `mark_through` rejects a reply
as its anchor. It also skips a reply-marked message whose ancestry never
resolved, because marking through one would fail on every retry.
- **`read_through_id`:** frontiers stored only the anchor's arrival
time. Migration 0058 adds `personal_read_frontiers.through_message_id`,
which follows the greatest arrival, and drops the now-unused
`threads_through_timestamp`. This is the only storage change.
- **Counts are plain numbers.** The exact/at_least/unknown wrappers are
gone. Evidence the relay can't use (unusable tags, unresolved ancestry,
undecided conversation membership, past the scan cap) leaves the message
out, so counts can undercount.
- **POST returns the updated row** of each distinct channel it applied
to, so the client doesn't refetch after every read. If those rows can't
be read after the writes commit, `channels` is omitted and the outcomes
still stand. The row refresh shares the 8 s intent deadline.
- **Up to 25 thread rows per channel** (was 5). Five hid a mention in a
sixth unread thread. 25 is the most that keeps a POST's rows for
`MAX_INTENTS` (100) channels inside the 1 MiB response limit; 50 would
be ~1.4 MB, a 503 that hides already-committed outcomes. A unit test
builds that worst-case response and fails if the cap outgrows the limit.
- **The doc separates shape from behavior.** `/buzz/v1` promises a
stable shape (fields, types, status variants, what `mark_through` does).
What the counts include (which messages are unread or directed, which
threads are listed, the bounds) may change while v1 is pre-release, so
clients replace their rows with each response instead of reconciling
against local counts.
- **Removed from the wire:** the explicit-contexts read (`GET
/me/read-state?targets=`), `account`, channel name/type/archived/hidden
(these come from Nostr metadata), `latest_message_at`/`_complete`, and
the NIP-11 `max_contexts`/`max_context_messages`/`max_thread_summaries`.

Deliberately deferred (behavior or additive, no shape change):
- Thread counting rule (relay counts a reply if you wrote or replied to
its direct parent; the client uses "posted anywhere in the thread or
followed"), the workflow-owner attribution p-tag, and thread-row
`mentions` counting every counted reply rather than directed ones.
- `follow`/`unfollow` intents, listing followed threads that are already
read, and a truncation flag for the thread list.

Thread: buzz://channel/6f773b94-34d0-4004-bec9-a100b6ad17d5 (#buzz-bff,
root 31020fff)

### Related issue
Follows #8157. Supersedes #8188; replaces the wire parts of the closed
#8155.

### Testing
- `cargo test --no-fail-fast -p buzz-db -p buzz-relay` at 1f041e46: all
pass except 7 relay unit tests (`api::media::tests` ×6 needing a seeded
local DB, `api::mesh_demo` round trip) that fail identically on `main`
(394d4a43) in the same environment.
- `scripts/postgres-test-run.sh -p buzz-db -p buzz-relay` against local
PG14 + Redis at 88d2f6fc: 836 passed, 26 failed. 25 also fail on `main`
in the same environment (partition maintenance tests and
`huddle_started_links`, which need PG16). The 26th,
`boot_lifecycle::hanging_redis_peer_times_out…`, is a timing assertion
that failed under full-suite load and passed 3/3 when rerun alone on
this branch.
- All `personal_read`, `migration`, `buzz_v1`/`accessory` and NIP-FI
wire-contract tests pass, including the new response-size bound test and
the 25-thread `summarize` cap test.
- `cargo fmt --check` and `cargo clippy -p buzz-db -p buzz-relay --tests
-D warnings` are clean.
- Not done: running a standalone relay binary (the dev object store
rejected writes with HTTP 403). The relay tests drive the real HTTP
router with NIP-98-signed requests. A human test against a local relay
is still needed.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Honey <8e307ae0076a4dab6b94b036ea3edc7e08f823a625269c1e6919e881a048b4d2@buzz.block.builderlab.xyz>
Co-authored-by: Honey <8e307ae0076a4dab6b94b036ea3edc7e08f823a625269c1e6919e881a048b4d2@buzz.block.builderlab.xyz>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 13:07:19 -07:00
Bradley Axen
7afdc2aac9 feat(relay): record when a read-state account starts (#8157)
## Goal

Add `started_at` to read-state accounts from the day the table is
created. This prevents users seeing ALL their channels as unread when
they first install a client update that starts using these APIs

---------

Signed-off-by: hMB2 <0400eb0da32afbbf16cdd1dee6ffd243587e8f3d6a2823fe7dfa95d16b002b7d@buzz.block.builderlab.xyz>
Signed-off-by: ART <ea39c1c8acf57180161ef4ba4ed68abf8589f5291956b909a5aa991fc66664ca@buzz.block.builderlab.xyz>
Signed-off-by: hMurderbot <219c58afff8ac2c76e4e527a329aa61c2ebbb896786b1916d4f9cca5140ecd04@buzz.block.builderlab.xyz>
Co-authored-by: hMB2 <0400eb0da32afbbf16cdd1dee6ffd243587e8f3d6a2823fe7dfa95d16b002b7d@buzz.block.builderlab.xyz>
Co-authored-by: hMurderbot <219c58afff8ac2c76e4e527a329aa61c2ebbb896786b1916d4f9cca5140ecd04@buzz.block.builderlab.xyz>
Co-authored-by: ART <ea39c1c8acf57180161ef4ba4ed68abf8589f5291956b909a5aa991fc66664ca@buzz.block.builderlab.xyz>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 10:04:02 -07:00
Luke Tornquist
745501962e refactor(schema): fan desired-state schema out to one file per table (#8131) 2026-10-08 08:47:44 -04:00