Commit Graph

32 Commits

Author SHA1 Message Date
Will Pfleger
9ab163190b feat(db): add NIP-FI identity and final-admission schema foundation (#6994)
PR 2 of the NIP-FI plan: the schema foundation. Establishes the durable
server-side identity ledger and final-admission surface that the runtime
phases build on. All of Phase A's migrations live here; later phases own
their own deltas.

Depends on nothing — PR 1 (#6776, merged) owned zero migration files.
This PR's relations are shaped to store exactly what PR 1's verifier
produces: issuer-qualified identity and the four denial classes. They
meet in a later PR that writes a verified assertion into these tables in
one transaction.

## Two internally-ordered migrations

- `0041_nip_fi_identity_foundation.sql` (migration A) — core identity +
base-lifecycle relations (5 tables): issuer-qualified `(iss, sub)`
bindings, lifecycle history/selectors, enrollment policies, and
operation receipts. Applies cleanly to current `main`.
- `0042_nip_fi_authorization_foundation.sql` (migration B) — the
final-admission surface (10 tables): authorization events + capacity,
admission results, replay/receipt guards, audit, invalidation
domains/floors, protected-object authority, authority epochs, and
restore version deltas. Applies to A's resulting state.

Fifteen NIP-FI relations total, zero dangling foreign keys. Identity is
issuer-qualified throughout — no single-global-issuer assumption in any
relation, no `Block`-hardcoding. A single deployment may run one issuer;
that is config, not schema.

## Durable, immutable ledger posture

All 15 relations are append-only (immutable `no_delete`/`no_truncate`
triggers) and carry `community_id` as provenance, not ownership. Both
migrations widen the single SQL source of truth
`community_write_fence_excluded_table` so the relations are never
fence-attached, never purged on community deletion, and never counted as
tenant-scoped drift by the deletion control plane's exact-set catalog
check — the same posture main already applies to `product_feedback` and
`rate_limit_violations`. `schema/schema.sql` keeps one consolidated
definition of that function whose exclusion array byte-matches `0042`,
guarded by a parity assertion so a future consolidation cannot silently
drop NIP-FI relations from the ledger.

This makes a tenant's identity/authorization ledger survive community
deletion, per the spec's `FI-INV-02` (durable binding) and `FI-INV-03`
(tombstone monotonicity) and `NIP-FI.md`'s "durable server state"
ruling. `communities(id)` FK never dangles: community rows become
permanent tombstones, never hard-deleted.

## Authorization shape and cardinality contracts

Authenticated `OperatorDenied` events (`actor_kind` 1–3, non-null
`request_fingerprint`) carry a null `semantic_fingerprint` and commit
without a denial-attempt row. The denial-attempt cardinality and shape
guards are scoped to unresolved pre-auth kind-9 events (`actor_kind =
4`). Applied and no-op lifecycle receipts (`outcome_code IN (1, 3)`)
require exactly one mapped success-transition event; denied lifecycle
receipts (`outcome_code = 2`) require zero events from the complete core
lifecycle success-transition class (kinds 1, 2, 3, 6: enrolled, revoked,
rotated, retired) — any such event paired with a denied receipt would
record a transition that never occurred.

## Mined vs. new

Re-cut from Franco's #1476 (`0029`/`0030`) and Cea's #4772 committer
schema, re-cut along FK topology and renumbered above the live `main`
tip. The buzz-auth core of #1476 is Cea-authored; `Co-authored-by`
reflects verified per-commit authorship of the mined schema.

Zero Rust/`deletion.rs` edits — the migration-only exclusion widening
keeps `EXPECTED_SCOPED_TABLES` untouched.

---------

Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Hayt <9e1c23a3fd83f61da34420e4e88ff1b16e45cafcc0cd9019eb07d4ecfa8ca9b0@buzz.block.builderlab.xyz>
Co-authored-by: Cea Stapleton Cordasco <261786559+cea@users.noreply.github.com>
Co-authored-by: Hayt <9e1c23a3fd83f61da34420e4e88ff1b16e45cafcc0cd9019eb07d4ecfa8ca9b0@buzz.block.builderlab.xyz>
2026-08-31 19:40:35 -04:00
Tom Brow
c432a111ca feat(mobile): push notifications MVP (#6269)
This PR implements MVP, iOS-only,
[NIP-PL](8d2d0ff5ad/docs/nips/NIP-PL.md)-compliant
push notifications.

A relay with `BUZZ_PUSH_ENABLED` will send a push notification for any
message that appears in the in-app Notifications tab.

## Enrollment flow
The first time the client first connects to a relay with
`BUZZ_PUSH_ENABLED`:
```mermaid
sequenceDiagram
    autonumber
    participant App as Buzz iOS app
    participant iOS
    participant Relay as Buzz relay
    participant Attest as Apple App Attest
    participant Gateway as Push gateway

    App->>Relay: Fetch NIP-11 push capability
    Relay-->>App: Push profile, current relay public key, and limits

    par
        App->>iOS: Request notification permission
        iOS-->>App: Permission result
    and
        App->>iOS: Register for remote notifications
        iOS-->>App: Device token
    end

    App->>Gateway: Request installation challenge
    Gateway-->>App: Single-use challenge
    App->>Attest: Attest installation transcript
    Attest-->>App: Attestation proof
    App->>Gateway: Enroll device token and proof
    Gateway-->>App: Installation handle

    App->>Gateway: Request delegation challenge
    Gateway-->>App: Single-use challenge
    App->>Attest: Assert relay-key delegation
    Attest-->>App: Assertion
    App->>Gateway: Create delegation
    Gateway-->>App: Opaque endpoint grant

    App->>Relay: Publish encrypted push lease and filters
    Relay-->>App: Lease acknowledged
```

## Push-time flow

When a notification-eligible event is received by the relay:

```mermaid
%%{init: {
  "sequence": {
    "actorMargin": 20,
    "width": 110,
    "messageMargin": 18,
    "diagramMarginX": 8,
    "wrap": true
  }
}}%%
sequenceDiagram
    autonumber
    participant Relay as Buzz relay
    participant Gateway as Push gateway
    participant APNs as Apple Push<br/>Notification service
    participant iOS
    participant NSE as Notification service<br/>extension

    Relay->>Gateway: POST /v1/deliveries/apns<br/>opaque endpoint grant, request ID, expiry, NIP-98 authorization

    Gateway->>APNs: POST /3/device/{device-token}<br/>topic, request ID, expiry, constant mutable-content payload
    APNs-->>Gateway: 200 OK: request accepted
    Gateway-->>Relay: 200 OK: accepted status

    APNs-->>iOS: Notification: constant reconnect alert<br/>mutable-content = 1
    iOS->>NSE: Invoke extension<br/>original notification content

    NSE->>Relay: POST /query: subscription filters, limit 10<br/>NIP-98 authorization
    Relay-->>NSE: 200 OK: signed Nostr events<br/>kinds 9, 40002, 45001, or 45003

    NSE->>iOS: Complete notification: title, body, subtitle<br/>thread ID, exact-message target
```

relay → push gateway → APNs -> NSE -> Notification Center

## Known limitations

The APNs wake payload is intentionally constant and opaque: it contains
no originating community or message identifier, in keeping with the
implemented NIP-PL privacy design.

The Notification Service Extension must therefore reconnect to the relay
and resolve eligible messages after each wake. Around overlapping wakes,
timing boundaries, or resolution windows, notification presentation may
occasionally omit an expected message or display a message more than
once.

This best-effort behavior is deliberately accepted for the current
implementation and will be measured during the internal rollout to
determine whether the user experience is acceptable before any broader
deployment; the implementation does not claim exactly-once presentation.

## Validation

Live end-to-end hardware validation used an internal remotely hosted
development relay and push gateway, the APNs sandbox, and a physical
iPhone 12 mini:

- A second real Buzz client published a uniquely marked message through
the hosted relay.
- The relay matched the message and sent the constant opaque wake
through the hosted gateway. The gateway made an actual APNs request; no
`simctl push` or simulated notification was used.
- The iPhone received the notification on its lock screen. The
Notification Service Extension reconnected to the relay, fetched the
event, verified its ID and signature, and replaced the placeholder
content with the real notification title and body.
- After the app populated its shared presentation cache, a final marked
notification visibly showed the sender display name, sender avatar, and
hashtag-prefixed channel name.
- Tapping a lock-screen notification opened Buzz and exercised the
notification-response path and navigated to the corresponding message.

Final validation with a dogfood-signed artifact and production App
Attest/APNs configuration remains a release step.

## Independent pre-reviews

- **First pass:**
[Carl](buzz://message?channel=18882f4c-289f-41db-942f-81f6f8066da1&id=74ab9a93bb227f3e762568f1cf9fee66d7495b0edc3918735ff787238b9cc585)
found missing transient retries, executor-key rotation suppression,
duplicate installation renewal, and an unauthenticated challenge write
amplifier. These were resolved by [retry-safe
bootstrap](https://github.com/block/buzz/pull/6269/changes/12c66ea62)
and [authenticated renewal plus a cross-replica
quota](https://github.com/block/buzz/pull/6269/changes/8e5ece0bd).
[sol-max](buzz://message?channel=ad83385f-8e9e-4461-9a35-c1bf2e208532&id=d26d53daa4684669e2ed354638241f13f36c3a97027fe8b4dd738aff09038962)
found delegation generation burning and an edited applied migration,
resolved by [exact-generation
revocation](https://github.com/block/buzz/pull/6269/changes/c26d2159d)
and a [forward-only
migration](https://github.com/block/buzz/pull/6269/changes/956c1d099).
[k3-max](buzz://message?channel=5e46055d-a766-4065-ae25-05d1e4aaa6b2&id=d43139138a0b15f806cbdbeeedd8f69d992cadf2e805876db6fdde6a34c7eda1)
found no blockers.
- **Exact-head re-review:**
[Carl](buzz://message?channel=18882f4c-289f-41db-942f-81f6f8066da1&id=a897721673459301b0cf26e8b85a1478d7ebbb56a4621f93d774c98d395b8f68),
[sol-max](buzz://message?channel=ad83385f-8e9e-4461-9a35-c1bf2e208532&id=fb2159f709ec68f74f7b21459acd76da0e8a7c5c0f3d469f99826b0cc2380849),
and
[k3-max](buzz://message?channel=5e46055d-a766-4065-ae25-05d1e4aaa6b2&id=2ce2842910435f562e9d9cc718595848f281b122c94605e523a4b964254b8bfb)
independently returned **NO BLOCKERS** at `7eb3a650b`; k3-max also
revalidated every remediation and the endpoint-specific App Attest
enrollment bound.

---------

Signed-off-by: Tom Brow <tomb@squareup.com>
Signed-off-by: Tom Brow <tomb@block.xyz>
Co-authored-by: Tom Brow <tomb@squareup.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Jordan Mecom <jm@squareup.com>
2026-08-28 15:15:09 -06:00
Will Pfleger
86b9142a09 refactor(relay): NIP-98 admin auth with Operator/Moderator roles and NIP-11 discovery (#3777)
Adds authenticated, role-based moderation to the relay admin API. On
`main` the admin API is read-only and gated only by `Host`/`Origin`
matching; this branch adds NIP-98 authentication, a two-tier
Operator/Moderator principal model, mutation and staffing endpoints, and
NIP-11 auto-discovery so clients never type the admin URL by hand.

## Authentication (`BUZZ_ADMIN_AUTH`)

`BUZZ_ADMIN_AUTH` accepts `nip98` or `disabled`. Leaving it unset
defaults to `nip98` (fail-secure). Configuration fails closed: any other
value aborts startup, while a lingering `BUZZ_ADMIN_TOKEN` is ignored
with a startup warning — token (bearer) authentication is not supported.
`Host`/`Origin` matching is retained in every mode as defense-in-depth.

- **`nip98`** (default) — per-request signed NIP-98 (kind 27235) events,
resolved to an Operator or Moderator principal with per-person
attribution and individual revocability. Read-write per resolved
principal.
- **`disabled`** — no credential; relies entirely on network-layer
controls (reverse proxy, VPN, firewall) and logs a `WARN` on every boot.
Always read-only: `authorize()` resolves no principal, so mutation and
staffing routes always `403`.

## Roles

Buzz has two independent authority axes after this change.
**Relay-level** roles (new here) are deployment-global: they act across
every community on the relay, through the admin API. **Community-level**
roles (pre-existing, unchanged by this PR) are tenant-scoped: they act
inside one community, through signed Nostr moderation commands.

### Relay level (new)

| Role | Description |
|---|---|
| **Operator** | Full control of the deployment's moderation surface:
read all reports, feedback, and attachments across every community;
resolve reports with enforcement (`delete`/`kick`/`ban`/`timeout`) or
decisions (`dismiss`/`escalate`); reopen and cancel; update feedback
status; and manage the Operator/Moderator roster via the staffing
endpoints. |
| **Moderator** | Day-to-day triage: everything an Operator can do
except staffing — cannot view or change the roster. |

How a pubkey acquires a relay role (resolution order; config always
outranks DB):

1. Listed in `RELAY_OPERATOR_PUBKEYS` → **Operator** (source `config`)
2. Equals `RELAY_OWNER_PUBKEY` while `RELAY_OPERATOR_PUBKEYS` is empty →
**Operator** (source `owner_fallback`, a break-glass grant for
self-hosters that deactivates once any operator is configured)
3. Row in the `relay_operators` table → **Operator** or **Moderator**
(source `db`, managed via the staffing endpoints)
4. No match → `403`

### Community level (pre-existing, unchanged)

| Role | Description |
|---|---|
| **Owner** (community) | Full authority within their community: every
moderation action (delete, kick, ban/unban, timeout/untimeout, resolve
reports, view queue) plus member, role, and invite management. No guard
rails. |
| **Admin** (community) | Same community-wide moderation capabilities as
owner, except an admin cannot ban or time out the owner or a fellow
admin — only the owner may action an admin. Manages members and invites;
only the owner grants the admin role. |
| **Member** (community) | Standard participant; no moderation
capability. |
| **Owner / Admin** (channel) | Channel-local authority only: delete
messages and kick users within their own channel. |
| **Member / Guest / Bot** (channel) | No moderation authority. |

There is no community-level Moderator tier in v1; relay-level Moderator
is the only role by that name.

## Escalation scoping

The operator report queue is an escalation backstop, not the community's
day-to-day triage surface (per `VISION_MODERATION`, the severe class is
the platform's to review rather than the community's). Two rules enforce
that:

- **Escalated-by-default listing.** `GET /reports` with no `status`
parameter returns only `escalated` reports. An explicit
`status=<open|resolved|dismissed|escalated>` filter is always honored as
given, and full visibility across every status stays available for
platform-safety and legal review via `scope=all` (which lists reports
regardless of status). `scope` accepts only `all` and is ignored when an
explicit `status` is present.
- **Auto-escalated `illegal` reports.** Member reports whose category is
`illegal` are ingested with `status=escalated` rather than `open`, so
the severe class reaches the operator backstop without waiting for a
community admin to forward it. Every other category still lands `open`.
Auto-escalation only sets the queue status — it records no moderator
decision and stamps no resolver, so an auto-escalated report is
indistinguishable downstream from an admin-escalated one: the reopen
route returns it to `open` on the same terms, keyed only on status,
never on how the report became escalated.

## Principal resolution and NIP-98 admission

`resolve_admin_principal()` returns `AdminPrincipal { pubkey, role,
source }` per the resolution order above; `None` never falls through as
a role. Admission is ordered so the replay guard is a privilege, not a
public surface: signature/URL/method/payload-hash verification first,
roster check second, and only then is the deployment-scoped replay id
atomically consumed — a validly-signing but unrostered key never
allocates a replay slot. Redis failure fails closed.

## Report resolution, recovery, and enforcement provenance

`POST /reports/{id}/resolve` is a crash-safe enforcement state machine:
decision-only outcomes (`dismiss`/`escalate`) are a single
CAS-plus-audit transaction; enforcement
(`delete`/`kick`/`ban`/`timeout`) claims the report
(`open`→`processing`), runs the durable mutation, then finalizes — a
re-drive resumes at the step marker and converges to exactly-one
enforcement, fenced by a lease and an outbox claim token.

Person-directed enforcement on an `event`-kind report derives its target
from the stored event's author (server-owned truth, never the reporter's
`p` tag) via a single `derive_enforcement_target` shared by the HTTP
driver and the recovery worker. If the reported event was purged before
its author could be read, person-directed actions are rejected pre-claim
and the report stays `open`; `delete` needs only the event id and is
exempt.

`GET /reports/{id}` and the resolve response carry an `activeAction`
field surfacing the enforcement that actually executed — a report
dismissed after a reopen still reports the ban that ran. `POST
/reports/{id}/reopen` returns a terminal report to `open` (idempotent on
`requestId`). `POST /reports/{id}/cancel` is the sole recovery path for
a pre-mutation `failed` action, attributed via
`relay_admin_actions.cancelled_by`.

## Feedback

`GET /feedback` and `/feedback/{id}` survive a tenant purge: provenance
columns are severed to `NULL` rather than cascade-deleted, and the
attachment path fails closed to `404` on a severed row. `PATCH
/feedback/{id}` updates lifecycle `status`
(`new`/`reviewed`/`archived`).

## Staffing and probe

`GET/PUT/DELETE /operators/{pubkey}` are Operator-only; mutating a
config-backed pubkey returns `409 Conflict`. `GET /operators` returns
the union of config and DB principals with per-entry `source`. `GET
/probe` reports auth mode, role, source, `canAct`, and `canStaff` for
the desktop console.

## NIP-11 auto-discovery

The NIP-11 relay-information document gains an optional `admin_api`
field carrying the canonical admin origin (`scheme://host[:port]`, no
path), present iff `BUZZ_ADMIN_HOST` is set and omitted otherwise. The
scheme follows the same loopback rule as NIP-98 `u`-tag verification via
a shared `scheme_for_host` helper, so the advertised origin and the
origin the relay verifies against can never diverge.

## Operator API origin decoupling

`RELAY_OPERATOR_API_ORIGIN` is no longer required at boot when
`RELAY_OPERATOR_PUBKEYS` is set — it is used only by the
community-provisioning endpoints, which fail closed at request time
(with a boot-time `WARN`) until it is set. The admin console needs no
origin.

## Admin-web adaptation

The standalone `admin-web` dashboard signs each request as a NIP-98
event via a NIP-07 browser extension, discovers the auth mode with a
single unauthenticated probe (`200` → `disabled`, anything else →
`nip98`, fail-secure), and carries no token entry surface. Playwright
coverage exercises the NIP-98 and CSP paths.

## Security hardening

Three findings from security review are folded in:

- **Append-only roster audit.** `PUT`/`DELETE /operators/{pubkey}`
mutate the deployment-wide root of trust, but the upsert overwrites
`role`/`added_by` in place and the delete removes the only row — so a
grant→revoke sequence left no trace of who was ever granted or by whom.
Each mutation now writes an `relay_operator_audit` row (actor, target,
`grant`/`revoke`, pre-image `prev_role`, `new_role`, timestamp) inside
the same transaction as the mutation. A per-target transaction-scoped
advisory lock serializes concurrent mutations of the same pubkey before
the pre-image read, so the recorded `prev_role` is always the true
predecessor even under a concurrent-grant race. Chronology is keyed on a
`BIGINT GENERATED ALWAYS AS IDENTITY` `seq` column, not the wall clock:
the serializing lock guarantees insertion order and `seq` captures it,
so ordered reads (`ORDER BY seq`) follow the true privilege chain even
across a backward NTP step that a `clock_timestamp()` ordering would
invert. `created_at` (`clock_timestamp()`) is retained as informational
occurrence time only. Append-only by construction — no `UPDATE`/`DELETE`
path and no API surface. A no-op delete writes nothing.
- **`expirationSecs` overflow.** The timeout path built `Utc::now() +
Duration::seconds(secs as i64)` from an attacker-controlled `u64`:
`i64::MAX` panicked the handler, and a wrapped-negative magnitude minted
a *past* expiry that still passed validation. `compute_timeout_until`
now rejects zero, rejects magnitudes above a documented
`MAX_TIMEOUT_SECS` (365 days), and uses checked
`try_seconds`/`checked_add_signed` so no input can panic or produce a
past expiry — over-cap, zero, `i64::MAX`, and wrapping-negative inputs
all return a clean `4xx`.
- **Uppercase-hex config-backed bypass.** Config pubkeys are lowercased
at parse, but the `409` immutability check raw-string-compared the path
param while `decode_hex_pubkey` accepted uppercase — so `PUT
/operators/{UPPERCASE}` skipped the guard and wrote a shadow row for the
same 32 bytes. The validated param is now canonicalized (lowercased)
before the `409` check, DB write, `DELETE`, and response body.

## Migrations

- `0035_relay_operators.sql` — `relay_operators` roster table
(deployment-global), `actor_authority` on `moderation_actions`,
`processing` status plus `active_action_id` on `moderation_reports`,
`status` on `product_feedback`.
- `0036_relay_admin_actions.sql` — enforcement-action table with a
`request_id` idempotency key, a `step_marker` for crash recovery, and a
`cancelled_by` attribution column.
- `0037_relay_admin_action_lease.sql` — lease fencing for the action
worker.
- `0038_relay_admin_outbox_claim_token.sql` — fenced claim token on the
outbox worker.
- `0039_relay_operator_audit.sql` — append-only `relay_operator_audit`
trail for roster mutations (see Security hardening).

`docs/admin/README.md` documents the full principal model, NIP-98 event
requirements, capabilities by role, the startup error matrix, and the
discovery field.

## Production blast radius

A relay without `BUZZ_ADMIN_HOST` is completely unaffected — the admin
surface stays disabled and `BUZZ_ADMIN_AUTH` is ignored; a lingering
`BUZZ_ADMIN_TOKEN` logs a startup warning and must be removed. Where
`BUZZ_ADMIN_HOST` **is** set, unset `BUZZ_ADMIN_AUTH` defaults to
`nip98` (per-person signed auth); `BUZZ_ADMIN_AUTH=disabled` reproduces
`main`'s prior `Host`/`Origin`-only gating but is read-only (mutation
routes `403`). The five migrations add tables and columns without
touching existing data.

---

Related: [block/buzz#4768](https://github.com/block/buzz/pull/4768)
(desktop admin console consuming the `admin_api` field),
[squareup/bb-public#339](https://github.com/squareup/bb-public/pull/339)
(Phase 4 rollout config)

---------

Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
2026-08-28 11:43:05 -04:00
Luke Tornquist
0ccf934b88 fix(db): disable heartbeat vacuum truncation (#6898)
## Why
`replica_heartbeat` is updated continuously, but autovacuum heap
truncation can take an `ACCESS EXCLUSIVE` lock whose replay cancels
concurrent hot-standby reads. Truncating a fixed, single-row table
provides no meaningful space benefit.

## What
- Add migration 0034 to set `vacuum_truncate = false` on
`replica_heartbeat`
- Keep the desired-state schema aligned for fresh installations
- Extend migration coverage so both paths retain the reloption

## Risk Assessment
Low — this disables only autovacuum's heap-truncation phase for a
single-row table; normal vacuum cleanup remains enabled. The tradeoff is
retaining at most the table's otherwise-truncatable tail pages.

## References
- [Investigation
thread](https://sq-block.slack.com/archives/C0B2WT43BDH/p1787760831375989)
- [PostgreSQL table storage
parameters](https://www.postgresql.org/docs/current/sql-createtable.html#SQL-CREATETABLE-STORAGE-PARAMETERS)

Generated with Codex

---

**Update Aug 27, 2026:** Hardened fresh `pgschema` bootstraps after
review.

- Renamed the shared post-apply script to
`reconcile-schema-after-pgschema.sql` and made it restore the heartbeat
reloption and singleton row idempotently.
- Added live catalog and row assertions that stop bootstrap when the
database does not match the required state.
- Updated every repository `pgschema apply` caller, including the
desktop release smoke path.
- Added a regression test that requires future `pgschema apply` callers
to run reconciliation.
- Documented the `pgschema` DML and storage-parameter limitation in
`AGENTS.md` and at the reconciliation entry point.
- Merged current `main` and renumbered this migration to 0034 after
`main` added migration 0033.

---------

Signed-off-by: Luke Tornquist <tornquist@squareup.com>
2026-08-27 13:01:13 -04:00
Tyler
b3baa56ba8 fix(db): exclude kind:30179 ciphertext from brownfield FTS (#6822)
## Problem

Kind `30179` (NIP-PMA private managed agent — NIP-44 ciphertext carrying
the agent nsec, env vars, prompt) joined `AUTHOR_ONLY_KINDS` in #4593,
but only **fresh** installs stopped indexing it:

- `migrations/0008` installs the positive FTS allowlist only when
`events` is empty;
- `migrations/0014` wraps the retained brownfield expression for `30350`
alone;
- `schema/schema.sql` still carries the legacy negative skip-set without
`30179`.

So a relay upgraded in place keeps tokenizing `30179` ciphertext into
`events.search_tsv`. Found by Wren while reviewing #4999 (which
activates publication of 30179 from Desktop). No readable leak: `/query`
applies `search_hit_accepted` + `event_visible_to_reader` before
serialization and live foreign searches against builderlab returned `[]`
— this is the storage-layer privacy invariant (`docs/nips/NIP-PMA.md`
deployment step 2) plus wasted FTS work.

## Fix

- `migrations/0033_private_managed_agent_fts.sql` — same shape as 0014:
capture the current generated expression via `pg_get_expr`, drop/re-add
`search_tsv` wrapped with `kind = 30179 → NULL`. Every other kind keeps
whatever policy the database already had (fresh allowlist or brownfield
skip-set). Rebuilds the GIN index; no heap-wide policy rewrite.
- `schema/schema.sql` — add `30179` to the desired-state skip-set.

## Tests

- `populated_upgrade_preserves_search_policy_except_for_private_kinds`
(renamed from `…_push_leases`): provisions the **legacy negative
expression** (migrations 1–7), inserts kind 1 / 30179 / 30350 rows,
checkpoints after 0014 (30179 still searchable, 30350 not), then runs to
head and asserts 30179 is NULL. **Proven red** with the 0033 file
removed: `left: [(1, Some(true)), (30179, Some(true)), (30350, None)]`.
- `embedded_migrator_contains_consolidated_initial_schema`: count 32→33,
asserts 0033 shape and `schema.sql` parity.
- `buzz-search/tests/fts_integration.rs` setup applies 0033 so the FTS
tripwires run against the real chain.

## Verification (HEAD 6154cc33d, same shell)

- `cargo test -p buzz-db -- --include-ignored --test-threads=1`: 305
passed / 3 failed — the 3
(`create_community_with_owner_enforces_per_owner_limit`,
`insert_mentions_indexes_rosters_past_bind_parameter_cap`,
`transfer_ownership_returns_limit_reached_for_maxed_transferee`) fail
identically on pristine `main` f24971033 with a fresh DB; unrelated to
this diff.
- `cargo test -p buzz-search -- --include-ignored`: 19 passed (incl.
`author_only_kinds_are_storage_level_unsearchable`,
`p_gated_persistent_kinds_have_storage_null_tsvector`).
- `desired_state_schema_bootstrap_progresses_beyond_fencing` +
`run_migrations_applies_consolidated_initial_schema_on_fresh_database`:
pass.
- `cargo fmt --all --check` clean; `cargo clippy -p buzz-db -p
buzz-search --all-targets -D warnings` clean.

Independent of #4999 — both main and #4999 are brownfield-exposed today;
this lands either order.

---------

Signed-off-by: Meli <5aaa86bce934fc3445fc254aab560a40923f10252f92107e665073dede0e04d3@buzz.block.builderlab.xyz>
Signed-off-by: Eva <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Co-authored-by: Meli <5aaa86bce934fc3445fc254aab560a40923f10252f92107e665073dede0e04d3@buzz.block.builderlab.xyz>
Co-authored-by: Eva <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
2026-08-27 10:47:35 -04:00
Wes
24ec6a468e Repair stale large channel roster snapshots (#6251)
## Summary

- detect relay-authored NIP-29 kind 39002 roster snapshots truncated by
the former 1,000-member query cap and repair stale large rosters during
relay startup
- serialize canonical roster capture and replacement with membership
writes, preserving tenant, channel, signer, pubkey, and role boundaries
through mixed-version deployments
- install migration 0032's fail-closed roster fence on the partitioned
events table and verify its catalog shape plus behavior before opening
relay listeners

## Rollout

Migration 0032 is a hard schema-before-code compatibility boundary.
Apply migrations before rolling this relay version. Startup refuses to
open listeners when the parent/partition triggers are missing, disabled,
mis-shaped, or behaviorally inert. For large installations, prefer
`buzz-admin migrate` and monitor lock acquisition as documented in the
chart README.

## Validation

Exact head: `bcbba271f54bc0046a6683007e5a2b70403a11d5`

- rebased onto `569308c23c9c2bf620dd3a9a5e4baecbcfa22e16`; the nine-file
feature patch is byte-identical to pre-rebase head
`be8ea0084f4d4c78c7c2550baad4399e4df8ce73`
- pre-push hook passed at exact head: branch-skew, file-size, full Rust
unit suite, Desktop Tauri clippy, and Desktop Tauri tests
- `cargo fmt --all -- --check`
- `cargo test -p buzz-relay
group_members_snapshot_keeps_members_past_one_thousand -- --nocapture`
- focused CI-mode Playwright regression: `selected relay agents revoked
after the invite prompt cause no side effects` passed at exact head
- prior exact-patch validation:
`large_roster_reconciliation_candidates_respect_snapshot_count_and_signer`,
mixed-writer locking/rollback, migration admission, partition trigger
coverage, and desired-schema parity regressions

## Review

Independent DB/relay review found no blocking issues in the exact
feature patch. The concurrency fence holds the established replacement
and membership locks on one transaction/connection through replacement;
failures roll back both soft-delete and insert. Reconciliation remains
tenant/channel/signer scoped and validates exact normalized
pubkey-plus-role membership.

The prior red Desktop shard was unrelated to this backend-only diff: its
mocked mention test exercises no relay, database, or migration path. It
reproduced as a timing flake on the old head, passed on retry/base, and
now passes locally after rebasing onto current main.

---------

Signed-off-by: Wes <wesbillman@users.noreply.github.com>
Co-authored-by: Carl <32a2e2c9d428ee08902cab75d956da2c1d235a22d4766b0dd4138bf6e2e5db1d@buzz.block.builderlab.xyz>
2026-08-20 21:22:41 -06:00
Wes
2693e0db1f Make workflow run history authoritative in Desktop (#5780)
## Summary

- persist stable workflow run `error_code` values separately from human
diagnostics
- expose NIP-98 authenticated, channel-authorized run history and
approval reads with stable keyset pagination
- connect Desktop to those authoritative reads and return the
relay-created run ID on trigger
- show truthful loading, failure, and pending-trace states, and do not
render approval actions from non-actionable stored hashes

## Validation

- pre-push `branch-skew`, `desktop-typecheck`, `desktop-test`,
`rust-tests`, `desktop-tauri-checks`, and `desktop-check` all passed on
`a097dbe5f`
- Desktop tests: 4,761 passed, 0 failed
- `cargo check -p buzz-relay`
- `git diff --check`

## Remaining gate

This does not claim a relay-backed Playwright workflow journey. The
browser relay bridge still routes workflow invokes through in-memory
handlers; that production-shaped acceptance gate remains follow-up work
before Workflows can leave preview.

---------

Signed-off-by: Wes <wesbillman@users.noreply.github.com>
Co-authored-by: Carl <c7ebe626f000404285d3686e1dc74cc07cc60a9754a150041ba132e14bd3e2ec@buzz.block.builderlab.xyz>
Co-authored-by: Mongo <5c25403eab7271f9f94ddd4f2b270e8cac2c92e2c830c51877cca6ec974ffb3f@buzz.block.builderlab.xyz>
Co-authored-by: Princess Donut <68157ebd23b3897c1991015c3038658ea916200c67d3a54620b0754d1b92f6e0@buzz.block.builderlab.xyz>
2026-08-13 12:56:37 -07:00
Kalvin C
8a2c9af2db feat(deletion): add durable whole-community deletion (#4425)
## Summary

Adds a durable, operator-controlled V1 for deleting an entire Buzz
community without deleting another tenant's data.

The workflow is exposed through `buzz-admin deletions`:

- `sweep` records independent fleet storage-taxonomy observations
- `submit`, `list`, `inspect`, and `approve` manage a deletion request
- `unblock` resumes a fail-closed request after an operator records
remediation identity and reason
- `run` and `drain` execute bounded work

Requests advance through a PostgreSQL-backed state machine and stop at
`retention_pending` after logical deletion has been independently
verified across PostgreSQL, object storage, and Redis.

This PR ships the engine and CLI, not a continuously running worker or
Kubernetes packaging. For V1, a cluster/VM administrator invokes
`/usr/local/bin/buzz-admin` from the existing relay image, for example
with `kubectl exec` or an equivalent container/VM exec path.

## What whole-community V1 removes

For the target community, V1 removes:

- rows from the allowlisted community-scoped PostgreSQL catalog,
including members, profiles, authored events and bodies, DMs, reactions,
mentions, memberships, tokens, workflows, moderation, audit, feedback,
and rate-limit state
- media sidecars and upload-attribution records under
`_meta/<community>/` and `_uploads/<community>/`
- Git repository pointers under `repos/<community>/`
- Redis keys under `buzz:<community>:*`

The community row survives as a permanent tombstone, and deletion
control-plane records remain as evidence of the request, approval,
execution, and result.

## Safety model

Deletion is not a broad `DELETE CASCADE` followed by optimistic cleanup.
The destructive boundaries are durable and fail closed.

### 1. Inventory and approval

- `submit` resolves the target and freezes the schema plus summary-only
storage inventory.
- Approval is bound to the exact request, community, and frozen
inventory digest.
- Unsupported manifest versions, malformed keys inside the target's
owned prefixes, live scoped-table/write-fence coverage drift,
frozen-inventory mismatch, and approval mismatch block execution rather
than guessing. Migration and catalog revision numbers are not
authorization gates; the executor validates the live safety shape
instead.
- Storage inventory is server-side prefix scoped to exactly:
  - `_meta/<community>/`
  - `_uploads/<community>/`
  - `repos/<community>/`
- The deletion path never lists the whole shared bucket and has no
arbitrary per-community object cap. Its listing work is proportional to
the target community's bindings, not total fleet storage.
- Fleet-wide taxonomy sweeps remain independent observability. They
report unknown writer shapes but do not gate deletion submission,
fencing, or destructive progress. Maintainers must add deletion taxonomy
coverage whenever a new community-owned object-key class is introduced;
writer-coverage tests bind the current media and Git writers to that
contract.

### 2. Quiesce, fence, and destructive freeze

- Writes continue through submission, inventory, and approval. They stop
when execution moves the target into `quiescing` and then establishes
the durable fence.
- Already-admitted external effects finish under heartbeated
serving-write leases; the exact admitted lease may renew while the
community is quiescing, but new lease acquisition is rejected. The
executor drains admitted leases before destructive work.
- Invite minting after quiescing begins fails as typed `AccessDenied`
(HTTP 503 at the relay boundary) before an invite can be persisted.
- Database triggers enforce the community write fence across the
complete catalog of community-scoped tables. Startup/readiness and
destructive execution validate that catalog so a newly added but
unfenced table cannot silently escape.
- **Named isolation assumption — fresh write snapshot.** Every writer
transaction that can reach a community-fenced relation must use
PostgreSQL `READ COMMITTED`; each guarded write therefore observes a
statement snapshot no older than acquisition of the community deletion
lock. `REPEATABLE READ` and `SERIALIZABLE` can retain a pre-fence
snapshot and are unsupported for writers. The writer pool refuses
non-`READ COMMITTED` sessions at connection setup, and both SQL fence
functions reject an explicit per-transaction isolation override with
SQLSTATE `25000`. Configuration-delivered bad isolation can surface
through SQLx as a pool-acquire timeout because every `after_connect`
attempt is rejected; the precise `community writes require READ
COMMITTED isolation` reason remains observable when the SQL guard is
reached. Read-only replica transactions are outside this assumption.
- Holding the shared advisory lock until the guarded write executes is a
separate liveness condition: under `READ COMMITTED`, releasing it early
does not permit resurrection because the trigger rechecks the fence, but
it can turn a fleet sweep into a statement-wide SQLSTATE `55000` abort.
- After the fence closes writers, storage is re-enumerated into chunked
side-table rows. Per-prefix counts and digests bind those concrete keys
to the destructive manifest.
- Manifest chunk insertion, update, and deletion are protected after
freeze. This closes the race where an unbound key could otherwise appear
after the manifest was committed.

### 3. Checkpointed destruction

- Target-owned object bindings are deleted from the frozen destructive
manifest in bounded batches with durable progress.
- The concrete key list lives in chunked side-table rows rather than one
request-row JSON value. It supports large communities, resumable
execution, and terminal cleanup.
- Missing objects are accepted as idempotent crash-window outcomes;
malformed ownership, changed evidence, and unexplained target-prefix
drift fail closed.
- PostgreSQL purging remains scoped by `community_id`, including the
guarded NIP-RS hard-delete path discovered with real Desktop kind
`30078` read-state data.
- Redis cleanup explicitly scans and `UNLINK`s only
`buzz:<community_id>:*`. Natural expiry is insufficient because some
keys, including tunnel generation counters used as fencing state, are
deliberately persistent.

### 4. Independent verification

- PostgreSQL logical absence is checked after purge.
- The three target-owned storage prefixes are freshly inventoried again
and must be empty.
- Redis requires two complete empty namespace scans.
- Only after all three stores pass does the request advance through
`logically_verified` to `retention_pending`.

## What V1 deliberately does not erase

### Shared content-addressed storage

Per-community deletion removes bindings, metadata, attribution records,
and Git pointers. It does **not** physically delete fleet-shared CAS
bytes that another community may still reference:

- media blobs and thumbnails
- Git manifests, packs, and indexes (`manifests/`, `packs/`, and `idx/`)

Safe reclamation requires a separate fleet-wide reachability and
retention GC. Unknown keys elsewhere in the shared bucket do not block
one community's deletion; malformed or unrecognized keys inside that
community's three owned prefixes still fail closed.

### External retained copies

The online logical-deletion proof does not erase object
versions/replicas, database backups/WAL, CDN copies, provider retention
copies, or observability exports. Those require their own retention and
purge controls.

### Member-only erasure

This PR erases a whole community. It does not implement the different
operation "erase one npub while preserving the community."

Removing membership or accepting NIP-09 is not member erasure. A
member-only workflow would need to find and selectively remove or redact
authored event content and pubkeys, profile data, DMs, reactions,
mentions, memberships/roles, tokens, workflows/subscriptions, upload
attribution, moderation/audit history, repository attribution, and
identity embedded in tags or JSON. It would also need explicit rules for
ownership transfer, surviving replies and thread metadata, audit-chain
integrity, immutable Git history, and shared-CAS reachability. That
requires a pubkey-level fence and selective graph rewrite; it is a
separate deletion product, not a safe extension of this whole-tenant
worker.

## In scope

- migration `0029_community_deletion.sql`: requests, approvals, leases,
manifest chunks, checkpoints, tombstones, and the universal write-fence
catalog
- durable executor leases, generations, heartbeats, retry/block state,
and resumable stage transitions
- operator-driven `sweep`, `submit`, `list`, `inspect`, `approve`,
`unblock`, `run`, and `drain` commands
- serving-path fences for database writes and external effects across
event ingest, media, Git, workflow, push, invites, mesh/tunnel, and
related paths
- target-prefix-only storage inventory, summary manifests, post-fence
destructive chunks, and bounded batch deletion
- exact community Redis namespace purge and two-pass absence
verification
- cross-community isolation, crash/resume, manifest-integrity,
writer-taxonomy, and schema/migration regressions
- desired-state `schema/schema.sql` support without requiring a SQLx
migration ledger

## Deferred / not covered

- dedicated Helm/chart worker Deployment, service account, secrets,
probes, resources, and network policy
- autonomous `buzz-admin deletions worker` poll loop and worker-only
health server
- least-privilege separation among migration, relay-serving, and
destructive execution roles
- fleet-wide shared-CAS physical GC
- backup/provider/CDN/observability retention completion
- member-only erasure
- provider-native conditional-delete improvements
- a general force-continue escape hatch; permanent safety failures
remain fail closed unless an operator remediates the cause and records
an audited `unblock`

The removed continuous-worker implementation remains deferred; no remote
follow-up branch is claimed by this PR.

## Validation

### Current PR head and repository state

Current pushed head: `359d8402ee15f049768f54156f67b953c7a7e2ed`, rebased
onto `cc9a2f783375e51a6e8d1f2f9d01d5f7e22813d1` (`origin/main` at push
time). The complete PR diff is now 47 files, 9,834 additions, and 517
deletions.

The bespoke source-scanner stack was removed to keep this PR scoped to
community deletion. Tyler/team requested the underlying fenced-write
safety behavior, not `ast-grep`,
`crates/buzz-db/tests/community_fenced_writes.rs`, its 27 fixtures, or
the new `scripts/lints/community_*.yml` rules. Those scanner-specific
files, dependencies, Hermit links, and runner wiring are absent from the
current tree. The production database write fence, startup/destructive
live-catalog validation, and deletion behavior remain.

Source validation on this exact SHA passed:

- `cargo fmt --all -- --check`
- `bash -n scripts/run-tests.sh`
- `cargo nextest run -p buzz-db --all-targets`: 102 passed, 173 skipped,
0 failed
- `cargo nextest run -p buzz-deletion --all-targets`: 10 passed, 9
skipped, 0 failed
- `cargo nextest run -p buzz-admin --all-targets`: 1 passed, 0 failed
- affected-package/all-target Clippy with warnings denied
- lockfile consistency
- Helm 3.16.4 lint and all 44 chart unit tests
- Helm region controls using that fixture: default
`BUZZ_S3_REGION=us-east-1`, explicit `eu-west-2` override, and
blank-region schema rejection

The prior Kubernetes battery below was run against
`928992237358a3294621ac0280830b77155abc04`. It remains useful evidence
for the patch-equivalent production deletion implementation, but it is
**not** claimed as exact-SHA evidence for current head
`359d8402ee15f049768f54156f67b953c7a7e2ed`; the current cleanup removes
only scanner/test/tooling infrastructure. CI restarted for the new head
after the rebase and is pending. Human review remains
`CHANGES_REQUESTED`.

### Prior-head live Kubernetes deletion and safety gates

The full program used one immutable image, real PostgreSQL, Redis,
MinIO, and a three-relay Kubernetes release:

- source: `928992237358a3294621ac0280830b77155abc04` (**prior head**)
- image: `buzz-e2e:sha-928992237358`
- immutable image digest:
`sha256:a1a204f4618ac22d9e210be5e5290645a15d79831ae30b0e44379357c8e4a895`
- evidence root:
`/tmp/buzz-e2e/20260807T033025Z-928992237358-full-gates/`
- evidence-manifest digest:
`82875c5bc9bea7370b796a7aef3457b3a1c8306c84c59e0f7388bbb5ad30e865`

Passed gates at that prior head:

- **Chart/operator region:** default `us-east-1`, explicit nondefault
propagation, blank-region schema rejection, live in-pod environment, and
an in-pod taxonomy sweep over 18 objects with zero unknown.
- **Fenced writers and lifecycle:** open-write/fence ordering;
100-attempt anti-starvation; invite, push matcher, and exhausted-reaper
bystander isolation; non-`READ-COMMITTED` rejection; manifest/tombstone
contracts; eight-failure stage block and audited `unblock`.
- **Destructive lifecycle:** submit → approve → run →
`retention_pending`; PostgreSQL tombstone and Redis/S3 verification
true; zero retries/errors; terminal reruns rejected with exit 5.
- **Fresh 10,001-object crash boundary:** exactly two chunks (10,000 +
1). The executor deleted chunk 0 from MinIO while its PostgreSQL stamp
was row-lock-blocked, was killed with `SIGKILL`, left one object and
both stamps absent, then resumed the same request under generation 2 to
zero objects and terminal state.
- **Independent dead-owner recovery:** a dedicated executor claimed
generation 1, blocked before effects, and was killed through containerd
with `SIGKILL` (no TERM cleanup). The request remained owned and
unreclaimable before lease expiry; a successor claimed generation 2
after 60 seconds and completed with two attempts and zero retries.
- **Three-pod socket isolation:** ordinary NIP-42 and joined
huddle-audio target witnesses on every replica received exact `1008 /
community deleted`; healthy-tenant witnesses on those pods remained
live; deleted-host reconnect returned HTTP 404.
- **Health/provenance:** all replicas independently returned ready and
retained the exact image digest before/after destructive runs and an
audio-enabled rolling restart; PostgreSQL, Redis, and MinIO were healthy
at close.

Instrument corrections were retained as evidence rather than counted as
product failures: a foreground PostgreSQL forward caused an initial
`PoolTimedOut`; Kubernetes pod deletion exercised graceful TERM rather
than dead-owner recovery; shell-background socket witnesses died with
their parent; and the first image build hit the corporate TLS proxy.
Detached forwarding/witnesses, containerd `SIGKILL`, and the configured
internal CA/Artifactory mirror produced the discriminating runs without
weakening product security.

### Prior-head cleanup

For the prior-head Kubernetes run, the Helm release was removed,
namespace absence was verified, run-owned Screen sessions were absent,
and that source worktree remained clean. The evidence manifest was
independently recomputed and every indexed artifact passed `shasum -a
256 -c`. The current `359d8402` source worktree is also clean after the
scanner-only cleanup and push.

---------

Signed-off-by: npub122y0pqkertljmedu303rl0aqrj3w8pvu43t6jxm6875lzg6f2pwqegc3xc <5288f082d91aff2de5bc8be23fbfa01ca2e3859cac57a91b7a3fa9f12349505c@buzz.block.builderlab.xyz>
Signed-off-by: npub1dccv64krpcpse5cmkzfeh998cftungyatw3djt8jwdw6g43f7fyqzzmrf7 <6e30cd56c30e030cd31bb0939b94a7c257c9a09d5ba2d92cf2735da45629f248@buzz.block.builderlab.xyz>
Signed-off-by: Kalvin Chau <kalvin@block.xyz>
Signed-off-by: am <6e30cd56c30e030cd31bb0939b94a7c257c9a09d5ba2d92cf2735da45629f248@buzz.block.builderlab.xyz>
Signed-off-by: cid <d9f92a72922bf45c17379a47d64dae84b6020397c2d5a52b5317d512068cd9d3@buzz.block.builderlab.xyz>
Co-authored-by: npub122y0pqkertljmedu303rl0aqrj3w8pvu43t6jxm6875lzg6f2pwqegc3xc <5288f082d91aff2de5bc8be23fbfa01ca2e3859cac57a91b7a3fa9f12349505c@buzz.block.builderlab.xyz>
Co-authored-by: npub1dccv64krpcpse5cmkzfeh998cftungyatw3djt8jwdw6g43f7fyqzzmrf7 <6e30cd56c30e030cd31bb0939b94a7c257c9a09d5ba2d92cf2735da45629f248@buzz.block.builderlab.xyz>
Co-authored-by: cid <d9f92a72922bf45c17379a47d64dae84b6020397c2d5a52b5317d512068cd9d3@buzz.block.builderlab.xyz>
2026-08-12 09:25:58 -07:00
Taylor Ho
2ea9385015 fix(reactions): support max-length custom emoji (#3833)
**Category:** fix
**User Impact:** Custom emoji with valid 64-character names can now be
used as reactions without errors.

**Problem:** Buzz accepted 64-character custom emoji names during
registration, but rejected them as reactions after the required
surrounding colons made the payload 66 characters. Validation also
differed between desktop, SDK, relay, and storage boundaries.

<img width="554" height="47" alt="image"
src="https://github.com/user-attachments/assets/4013452f-210e-4dd3-9003-f45ff3b28dc8"
/>

**Solution:** Keep the product limit at 64 ASCII characters for custom
emoji names, enforce it consistently when emoji sets are registered, and
allow only valid matching custom reaction payloads up to 66 characters.
Widen the reaction projection to preserve the wrapped payload while
retaining the existing 64-character limit for ordinary reactions.

<details>
<summary>File changes</summary>

**crates/buzz-sdk/src/builders.rs**
Defines the shared custom emoji boundaries and covers accepted
64-character and rejected 65-character shortcodes.

**crates/buzz-relay/src/handlers/ingest.rs**
Validates emoji-set shortcodes and permits 66-character reactions only
when they are valid colon-wrapped custom emoji with a matching tag.

**crates/buzz-db/src/event.rs**
Adds storage regression coverage for maximum-length custom emoji
reactions.

**crates/buzz-db/src/migration.rs**
Verifies the reaction column migration is applied correctly.

**desktop/src/shared/api/customEmoji.ts**
Enforces the existing 64-character shortcode maximum during desktop
normalization and registration/import.

**desktop/src/shared/api/customEmoji.test.mjs**
Covers the desktop shortcode boundary.

**migrations/0027_long_reaction_payloads.sql**
Widens stored reaction payloads to 66 characters for the two required
surrounding colons.

**schema/schema.sql**
Keeps the desired schema aligned with the migration.

</details>

## Reproduction Steps

1. Register or import a custom emoji whose ASCII shortcode is exactly 64
characters.
2. Select that emoji as a reaction to a message.
3. Confirm the reaction publishes, persists, and renders without an
error.
4. Attempt to register a 65-character shortcode and confirm it is
rejected.
5. Publish an ordinary or malformed reaction over 64 characters and
confirm the relay rejects it.

## Verification

- `cargo test -p buzz-sdk`: 243 passed
- `cargo test -p buzz-db`: 94 passed, 152 Postgres-required tests
ignored
- `pnpm test` in `desktop`: 3,859 passed
- `cargo test -p buzz-relay`: 795 passed, 9 existing
Postgres-unavailable failures, 35 ignored; new reaction boundary tests
pass directly
- `cargo fmt --all -- --check`
- `git diff --check`

Originating Buzz channel: `f2ec9671-d78e-4cde-894c-9f4c458c7f1f`

---------

Signed-off-by: Taylor Ho <taylorkmho@gmail.com>
Co-authored-by: npub1223z34hd7vtwc6qj4s7flsxkj644nlre2nthu7lrrmkumhu3xddsrx9r6w <52a228d6edf316ec6812ac3c9fc0d696ab59fc7954d77e7be31eedcddf91335b@buzz.block.builderlab.xyz>
2026-08-05 21:02:57 +00:00
Jemiah Westerman
bc9e6528a7 perf(relay): index channel-id lookups and skip trace-only reads (#4647)
## Problem

`SELECT id, community_id FROM channels WHERE id = ANY($1) AND deleted_at
IS NULL` is the top **Load by waits (AAS)** on the Buzz Postgres writer.
Two independent causes compound, and both are fixed here.

### 1. No index can serve it

`channels` is `PRIMARY KEY (community_id, id)`, and every secondary
index leads with `community_id`:

| Index | Columns |
|---|---|
| *(primary key)* | `(community_id, id)` |
| `idx_channels_nip29_group` | `(community_id, nip29_group_id)` |
| `idx_channels_dm_hash` | `(community_id, participant_hash)` |
| `idx_channels_community_type` | `(community_id, channel_type)` |
| `idx_channels_community_visibility` | `(community_id, visibility)` |
| `idx_channels_created_by` | `(community_id, created_by)` |
| `idx_channels_ttl_expiry` | `(ttl_deadline)` *(partial)* |

The two tenant-independent lookups carry **no `community_id` predicate**
— deliberately:

- `Db::communities_of_channels` — `WHERE id = ANY($1) AND deleted_at IS
NULL`
- `Db::community_of_channel` — `WHERE id = $1 AND deleted_at IS NULL`

That independence is load-bearing, not an oversight: projecting a row's
*true* owning community regardless of the fetch query's `WHERE` clause
is what makes `Inv_NonInterference` non-vacuous. If the fetch ever
dropped its tenant scoping, this lookup would still report the real
label and the checker would catch the mismatch.

But a composite btree is only usable when its leading column is
constrained, so neither query can use the primary key, and nothing else
leads with `id`. **Both sequentially scan `channels` on every call.**

### 2. In production the result is discarded

Both call sites feed `record_read_message_rows` /
`record_read_by_id_rows`, which call `tracer.record(...)`. Production
binds `NoopTracer` (`crates/buzz-relay/src/state.rs`), whose `record`
body is empty.

The existing guard tests `trace_state`, which is `Some` for every
well-formed request — it only goes `None` on malformed pubkey bytes. So
the scan ran on the hot read path and its output was dropped. This is
the classic eager-argument bug: `log.debug("..." + expensiveCall())`
with no `isDebugEnabled()` check.

### 3. Multiplied per filter

The non-search call site sits **inside the phase-3 per-filter loop**, so
a `REQ` carrying N filters performed N sequential scans of `channels`
before responding.

## Changes

**`Tracer::enabled()`** — a capability check on the trait (the
`isDebugEnabled()` of this seam), defaulting to `true`. `NoopTracer`
overrides it to `false`, and both emitters in `req.rs` now gate on it,
skipping the trace-only DB read entirely in production.

**`migrations/0027_channels_id_lookup_index.sql`**

```sql
CREATE INDEX IF NOT EXISTS idx_channels_id_live
    ON channels (id) INCLUDE (community_id)
    WHERE deleted_at IS NULL;
```

- `INCLUDE (community_id)` — both queries select exactly `(id,
community_id)`, so this is covering and can be served index-only.
- Partial on `deleted_at IS NULL` — matches both predicates exactly,
excludes soft-deleted history, and lets Postgres skip the recheck.
- **Not `UNIQUE`.** `id` alone is *not* unique in this table —
`command_executor.rs` documents that `community_of_channel(channel_id)`
is ambiguous because the same channel id can appear under more than one
community. A unique index would encode a false constraint and fail to
build on any database already holding such a pair.

Worth keeping the index even though fix #1 removes the production
caller: it still runs under conformance, and `community_of_channel` has
the same problem on its own paths.

**`schema/schema.sql`** — mirrored, since a test asserts desired-state
parity.

## Conformance is unchanged

This is the part worth reviewing closely. Under a real tracer
`enabled()` returns `true` and **every emit happens exactly as before**
— the gate only skips *building* emit inputs when nothing observes them,
never an emit that would otherwise have been made. The coverage-breach
guard stays non-vacuous.

`CountingTracer` forwards `enabled()` to its inner tracer rather than
inheriting the `true` default. Both directions matter and both fail
silently:

- inheriting `true` over a `NoopTracer` would keep the overhead this PR
removes;
- hardcoding `false` over a live tracer would suppress the emits whose
absence `EmitGuard` reports as `ImplBug` — masking real breaches behind
expected ones.

Covered by a new regression test,
`counting_tracer_delegates_enabled_to_inner`, which asserts delegation
in both directions.

## Verification

- `cargo check -p buzz-conformance -p buzz-relay` — clean
- `cargo clippy --all-targets` — clean, zero warnings
- `cargo test -p buzz-conformance` — 6/6
- `cargo test -p buzz-relay --lib conformance` — 11/11
- `cargo test -p buzz-db --lib migration` — 7/7
- `just test-unit` (pre-push) — green

Migration-count assertions in `crates/buzz-db/src/migration.rs` were
bumped 26 → 27, with content assertions for 0027 following the existing
per-migration pattern (including a guard that it never becomes
`UNIQUE`).

## Open questions for reviewers

1. **Lock strategy.** Built *without* `CONCURRENTLY`, following
migration 0004's precedent, because sqlx runs each migration inside a
transaction and `CREATE INDEX CONCURRENTLY` cannot run in one. This
takes a brief `SHARE` lock on `channels` (blocks writes, not reads) —
small relative to `events`, but an operator preferring zero
write-blocking can pre-build it by hand and `IF NOT EXISTS` makes the
migration a no-op. I could not confirm whether sqlx 0.9 supports a `--
no-transaction` directive; if it does, that may be preferable.

2. **Diagnosis is static.** This comes from reading the source, not from
`EXPLAIN` against the live database. Worth confirming with `EXPLAIN
(ANALYZE, BUFFERS)` on the writer before/after — that also sizes the win
by revealing the real table size and row counts.

3. **Expected impact** scales with average filters-per-`REQ`, which I
did not measure. `pg_stat_statements` ordered by `total_exec_time` would
confirm this query drops off the top and show whether anything else is
scanning the same way.

Signed-off-by: Jemiah Westerman <jemiah@squareup.com>
2026-08-04 13:59:54 -04:00
Tyler
5765fc74b7 fix(relay): allow open relays to set their NIP-11 workspace icon (kind:9033) (#3998)
## Problem

The desktop deliberately shows the workspace icon editor on open relays
(#2640, gate: `canEditIcon` in
`desktop/src/features/communities/ui/EditCommunityDialog.tsx`) and
defers to the relay-side kind:9033 check — which required an admin/owner
row in `relay_members`. For a community with **no admin/owner row at
all** (the `ensure_configured_community` path, which never writes an
owner), every 9033 was refused and the icon was permanently unsettable.

**Correction from review (thanks @Dawn):** the original version of this
PR claimed nobody holds a role on an open relay. That's false —
`main.rs` bootstraps `RELAY_OWNER_PUBKEY` as owner regardless of
`BUZZ_REQUIRE_RELAY_MEMBERSHIP`, so a production open relay like
bb-block *does* have an owner row, and the old gate was refusing
everyone except that owner. The first revision of this diff would have
silently widened that owner-only control to any NIP-42-authenticated
sender.

## Fix — steward-wins

`may_set_workspace_profile(sender_role, membership_enforced,
community_has_steward)`:

| Relay mode | Community has admin/owner row? | Who may set the icon |
|---|---|---|
| Closed (`require_relay_membership=true`) | any | admin or owner
(unchanged) |
| Open | yes (e.g. bb-block) | admin or owner (unchanged posture) |
| Open | no (genuinely rosterless) | any NIP-42-authenticated sender |

- New DB helper `has_admin_or_owner(community)`
(`crates/buzz-db/src/relay_members.rs`); the call site only queries it
on open relays.
- The rosterless admit logs a `warn!` with the sender pubkey — 9033
writes no audit row and publishes no announcement event (unlike
9030/9031), so this is the only durable attribution.
- Kinds 9030–9032, NIP-42 auth, `AdminUsers` scope, ban gate, and icon
validation are all untouched.
- Doc comment fixed: cited nonexistent `canEditCommunityProfile`; real
symbol is `canEditIcon`.

## Test coverage — closing the mutation gap

Dawn's mutation testing showed the original unit tests pinned only the
helper's truth table: inverting the flag at the call site or deleting
the gate entirely survived the full suite.

- Unit tests now cover the 3-arg truth table (closed
steward-independent, open-with-steward stays steward-only,
rosterless-open admits).
- Two `#[ignore]`d Postgres integration tests drive
`handle_relay_admin_event` with a real `AppState` (open rosterless admit
→ steward appears → roleless refused again; closed relay member
refused). Wired into the Backend Integration CI job as a dedicated
nextest step.
- **Both of Dawn's mutants verified killed** at this head: flag
inversion fails 1 unit test; gate deletion fails both integration tests
(`Ok(())` where `Rejected` expected).

## CI wrinkle found and fixed: pre-existing schema drift

The first Backend Integration run of the new 9033 tests failed with
`column "icon" of relation "communities" does not exist` — migration
`0003_community_icon.sql` added the column, but `schema/schema.sql` (the
desired-state file that CI job applies via pgschema) was never updated.
Pre-existing drift, invisible until a test in that job actually wrote
the column. Fixed in `297148f62` (3-line addition to
`schema/schema.sql`).

## Receipts (at `1b4b52db8` code / `297148f62` head)

- `cargo test -p buzz-relay`: 835 pass, 1 fail —
`api::mesh_demo::tests::demo_join_forwarded_arm_round_trips_echo`,
pre-existing (fails identically at the old base and on clean main);
`telemetry::trace_context_lookup_does_not_enable_callsites` is a known
order-dependent flake, passes in isolation.
- `cargo test -p buzz-db`: 94 pass.
- Both ignored integration tests pass live against local Postgres.
- `cargo fmt --all -- --check`: clean.
- Live-local pass per TESTING.md at this head (release build, relay on
:3199, real WS + NIP-42 via nak):
- open rosterless: roleless key sets icon → NIP-11 serves it; `warn!`
with sender pubkey in the relay log
- open + owner row inserted: fresh roleless key refused ("must be admin
or owner"); owner sets icon
- closed relay (owner bootstrapped, `BUZZ_RELAY_PRIVATE_KEY` set): plain
member refused, owner sets icon, `javascript:` URL rejected, empty icon
clears (NIP-11 → null)

---------

Signed-off-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
2026-08-01 12:03:31 -04:00
Tyler
63496cc1d4 feat(replica): portable heartbeat-token fence with snapshot-local reader routing (#3268)
## Summary

Expands read-replica usage on the relay per the Rev 2 design (thread
39d1e174 in #buzz-read-only-replica-usage): replaces the
Aurora-incompatible WAL-LSN read-side fence observation with a portable
**heartbeat token**, makes the freshness proof **snapshot-local to the
serving reader session**, and adds a default-off bounded-staleness gate
for head fetches.

### Probe (writer side, unchanged ordering)
The ordered writer scan is kept verbatim — `S = clock_timestamp()` →
masked-visibility `pg_stat_activity` oldest-xact scan — and now ends by
committing a heartbeat token **last** on the same pinned connection
(single-row `UPDATE replica_heartbeat ... RETURNING token, epoch`,
migration `0026`). The single-row update serializes all pods' probes, so
tokens are globally commit-ordered; the three-bucket completeness
argument carries over unchanged. Ring retains `(token, committed_at,
fence_wall)`; epoch change resets the ring; a same-epoch token
regression (restore adversary) clears the ring **and rotates the epoch
on the writer** so pre-rewind readers fail the epoch check. Cadence 1s.

### Routing (snapshot-local proof)
Every routed read opens `BEGIN ISOLATION LEVEL REPEATABLE READ, READ
ONLY` on a reader session and observes the heartbeat as the
transaction's **first statement** — the proof's snapshot is exactly the
snapshot the page, participants batch, and bridge aux closure read from
(`ReadSession` carries the open transaction; drop = rollback).
Fail-closed everywhere: begin failure, missing heartbeat row, epoch
mismatch, token below the ring, over-budget entry → writer.

- **Predicate B (cursor pages, default-on):** same completeness math as
the existing fence — cursor timestamp must be ≤ the proved wall; thread
candidate-terminal and above-wall pages re-run on the writer.
- **Predicate A (head fetches, default-OFF):** gated by
`BUZZ_REPLICA_HEAD_MAX_AGE_SECS` (0 = off, clamped to fence staleness).
Bounded-stale head semantics are an explicit product decision — **do not
enable anywhere without Tyler's backdated-event-semantics acceptance.**

### Observability
`buzz_db_route_decision{path, decision, reason}` across all five paths,
`buzz_db_replica_heartbeat_age_seconds` gauge, and per-decision debug
logs carrying the proved token plus backend identity: `addr:port pid=N`,
prefixed with `aurora_db_instance_identifier()` when the endpoint
supports it (probed once per process on an autocommit checkout; SQLSTATE
42883 caches a definitive false; identity is evidence, never a routing
gate).

## Review & verification

- **Wren:** full review 9/9/9 at `5f81b10e5`; identity delta re-review
approved at exact head `fedb46368` (90/90 unit, clippy `-D warnings`,
fmt independently reproduced).
- **Max:** local E2E **PASS** at `5f81b10e5` — isolated PG17 writer +
two streaming standbys behind HAProxy; paused-reader legs split exactly
as designed (6 replica/fresh + 6 writer/stale), recovery clean, RR
snapshot hardening observed in runtime logs. Evidence:
`WORK_LOGS/2026-07-28_REPLICA_HEARTBEAT_REPLACEMENT_SHA_E2E.md`.
- **My gates at head (same shell):** buzz-db 90 unit + 143
Postgres-gated green (scratch DBs); buzz-relay 767/768 — lone red is the
pre-existing `mesh_demo` flake, red on base; clippy `-D warnings` + fmt
clean.
- Key regression tests:
`routed_request_holds_one_snapshot_across_page_and_aux`
(mutation-verified both directions), same-epoch rotation,
capability-probe negative, head-gate truth table, divergent-fixture
routing suite.

## Post-merge plan

Merge publishes the immutable `sha-*` main image → bb-public PR pins it
→ ArgoCD sync → Max runs the production cursor-routing canary on Aurora
(positive identity branch proven live). Head gate stays off. Client
`since`-widening ships separately.

---

## Update — full read routing (Rev 6, commit `9fa3c9c0b`)

Extends routing to the remaining read seams per
`PLANS/REPLICA_FULL_READ_ROUTING_DESIGN.md` Rev 6 (thread cf5deba4 in
#buzz-read-only-replica-usage).

### New routed seams — ALL deploy-default dark
The new seams (`query_events_routed`, `query_events_routed_bounded`,
`count_events_routed`, `get_events_by_ids_routed`,
`query_feed_{mentions,needs_action,activity}_routed`) are
**Bounded-only** and gated on `BUZZ_REPLICA_READ_MAX_AGE_MS`: unset ⇒
every new seam records `writer/disabled` and merging is a no-op. COUNT
and feed/by-ids never take the covered arm (deletion visibility: covered
bounds insert-completeness only). **Note:** the pre-existing cursor
paths (channel windows, thread pages) are *not* gated by this env var —
they route at B=0 today and that status quo is unchanged.

### Reader pool (D4/D5)
- Lazy pool (`connect_lazy`, `min_connections(0)`): reader-down at boot
can't crash the relay; a warn-only boot ping is the only boot-time
visibility, and it primes the Aurora identity capability cache. Priming
is an optimization only — the routed path itself spends a single acquire
budget regardless (see the single-checkout fix below), because a boot
ping that *fails* is correlated with exactly the reader-unavailable case
the budget bound exists for.
- `READER_ACQUIRE_TIMEOUT` = 150ms; a miss fails closed to the writer
with reason **`reader_acquire_timeout`** — named for the mechanism, not
a diagnosis. The budget includes cold connects and sqlx's `size` counts
in-flight dials, so this metric alone does not distinguish contention
from slow connection establishment (see `proved_reader` doc-comment for
the runbook guidance; the pool gauges are 10s samples and are for
capacity planning).
- `BUZZ_DB_READ_POOL_SIZE` sizes the reader independently (invalid/0
inherits writer sizing); `read_pool_stats().max` reports the reader's
own ceiling.

### Community isolation (formal-model question)
Isolation is structural — an explicit `community_id = $n` predicate
compiled into every query builder; no RLS, no session-GUC tenant state
(zero `CREATE POLICY` across migrations). The `_on` variants reuse the
exact same builders with only the executor swapped. Proven by a
seven-seam two-community divergent-fixture test
(`routed_reads_are_confined_to_the_requested_community`),
mutation-tested by Dawn: all four single-predicate stubs killed; the
mention-join feeds are defended in depth (three independent predicates)
so only complete removal leaks there.

### Accepted limitation
D6: client-side staleness on bounded reads (up to `_MS`) is accepted
product behavior when the gate is enabled; gate stays off at merge.

## Update — single acquire budget per routed read (commit `dd26caa9f`)

Max found (and Dawn independently reproduced, 302–330ms measured) that
the boot-unavailable cold path spent **two** stacked
`READER_ACQUIRE_TIMEOUT` budgets: the Aurora capability probe did its
own `pool.acquire()` before `begin_with` acquired again. Fixed by
acquiring **once** per routed read — the capability probe runs on the
held connection (`reader_aurora_capability_on`) and the read-only
`REPEATABLE READ` transaction begins on that same connection
(`Transaction::begin` accepts a `PoolConnection` at `'static` via sqlx's
`MaybePoolConnection`). Reason codes unchanged; capability still never
negatively cached. Measured routed fallback: ~150ms (one budget).

Ships with a PG-gated regression fixture
(`routed_fallback_spends_one_acquire_budget_when_aurora_cache_is_cold`,
authored by Dawn): size-1 reader saturated, capability cache asserted
cold, routes through `count_events_routed`, asserts writer answer +
one-budget elapsed + `writer/reader_acquire_timeout` label
(mutation-tested: `pool_busy` and `reader_validation_error` mutants both
killed). It fails at 330ms on the previous commit and passes on this
one.

### Verification at `dd26caa9f` (pinned toolchain rustc 1.95.0 via repo
`bin/`; ambient rustc 1.89 fails sqlx resolution — always prepend
`bin/`)
- buzz-db: 94 unit + **150/151** PG-gated serial (`--test-threads=1`;
suite is not parallel-safe against one Postgres). The 1 red is
`test_usage_metrics_lock_has_single_owner_and_releases_on_drop` — a
**pre-existing** same-key-same-database advisory-lock collision with any
live relay pointed at the shared `buzz` DB (the relay re-arms
`USAGE_METRICS_LOCK_KEY` on a 300s tick). It passes on a private scratch
DB with the relay still running — verified this run. Remedy is a scratch
`TEST_DATABASE_URL`, **not** terminating lock holders: inspect
`pg_locks`/`pg_stat_activity` and resolve the owning process first; an
idle advisory holder may be a live dev relay. Pre-existing, not a #3268
blocker — filed as #3619.
- buzz-relay: 773/773; clippy `-D warnings` + fmt clean. (`mesh_demo`
echo test is a known pre-existing Redis-dependent flake — Dawn confirmed
it fails identically on the base commit, untouched by this PR.)
- Dawn: independent re-verification at `dd26caa9f` — byte-identical diff
application confirmed, suite reproduced, and fixture proven to still
discriminate (fails at 320ms with only the production fix reverted).
Gate clear. Max: independent rig pass at `dd26caa9f` — 94/94 unit,
**15/15 routing/fallback matrix** (cursor behavior, default-dark seams,
seven-seam confinement, one-snapshot, dead-reader/no-URL fallback,
hard-delete fail-closed, reader max), 773/773 relay, private-DB lock
control. Gate clear; recommends merge + staged bb-block rollout
(no-reader-URL no-op → cursor-only → observe `buzz_db_route_decision` →
conservative nonzero `BUZZ_REPLICA_READ_MAX_AGE_MS`; rollback is
config-only).

## Known gap — CI does not run the PG-backed fixtures (#3622)

Dawn found post-sign-off (confirmed by Max and Eva at `dd26caa9f`):
**every Postgres-backed fixture in this PR is `#[ignore]`d and no CI job
selects it.** `Unit Tests` runs `cargo nextest run -p buzz-db --lib`
(`Justfile:279-285`, ignored tests excluded); the only `--run-ignored
ignored-only` invocations (`ci.yml:690`, `:702`) filter to
`relay_invite::tests`. So the one-budget regression, the seven-seam
isolation fixture, and the fence/floor-guard/fallback tests have zero
automated execution — the verification record above is exhaustive but
manual, at this exact SHA.

Not introduced by this PR (the `#[ignore]` + narrow-filter pattern
predates it; 34 ignored tests total) and not a merge blocker: all new
seams are dark until `BUZZ_REPLICA_READ_MAX_AGE_MS` is set. But it
changes the rollout gate — **do not set a nonzero
`BUZZ_REPLICA_READ_MAX_AGE_MS` in bb-block until CI enforces these
fixtures.** Filed as #3622 (explicit CI selection against the existing
backend-integration Postgres archive; ordering note: #3619 must land
first or be excluded, since widening the filter would select the
colliding usage-metrics lock test).
## Live redteam results (Max, real Helm + PG17 streaming replication at
`dd26caa9f`) — #3643, #3644

Max deployed this exact SHA via the repo chart against a physical
writer/standby pair and attacked it. **The PR's fenced routing passed
every fault**: healthy baseline routes proven on the real standby;
paused WAL replay moved head/cursor reads to `writer/stale`; B=0 moved
routed reads to `writer/disabled`; reader-URL removal rolled back
cleanly.

The redteam also found a **pre-existing** production risk this PR does
not create and does not fix: NIP-50 search builds its own eager,
unfenced pool straight from `READ_DATABASE_URL` (`main.rs:389-402`,
introduced by #2084, present in the deployed bb-block `sha-dd222a5` and
bb-public `sha-22be8bb` images, and both production ESOs already
template `READ_DATABASE_URL`). Measured: ~30s search stall on reader
loss, acknowledged-but-unsearchable writes under replica lag (unaffected
by `BUZZ_REPLICA_READ_MAX_AGE_MS`), fatal eager connect blocking pod
startup when the reader is down, and `replica=true` FTS even at B=0.
Filed as **#3643** (fix: pin FTS to the writer, or fence it like the
routed seams) with **#3644** for the writer-pointing-reader-URL
telemetry gap. Aurora's cluster-ro writer-fallback DNS softens the
outage modes in prod but not the lag mode.

**Revised ladder consequence:** step 2 ("add `READ_DATABASE_URL`, budget
unset") is NOT a no-op with today's binary — it hands FTS an unfenced
replica pool. Rollout order is now: merge (still a true no-op —
bb-block/bb-public already run the unfenced-FTS code and this PR only
adds dark seams) → fix #3643 → CI enforcement #3622 → then reader URL +
budget per the original ladder, re-running Max's three deployment tests
(reader-down latency, reader-down cold boot, paused-replay
read-your-own-write search).
## Activation gate (consolidated) — merge is clear; ALL of the below
precede any nonzero `BUZZ_REPLICA_READ_MAX_AGE_MS`

Six independent verification passes at `dd26caa9f` (Dawn ×2 incl.
byte-level + defang check, Max matrix + live Helm redteam, Wren full
CRUD regression + greenfield 0026 + 151/151 PG, Eva). Every induced
fault was either handled correctly by this PR's machinery or traced to
pre-existing main code. Remaining work gates **activation, not merge**:

1. **#3643** — unfenced NIP-50 FTS pool (pre-existing #2084; live in
prod today; also explains the blackhole search hang Wren observed — the
routed path was measured bounded at 151-152ms against a silent endpoint,
sqlx `inner.rs:252-255`).
2. **#3651** — reader `statement_timeout`: statements after acquire are
unbounded; mid-transaction blackhole reproduced hanging ≥15s (Dawn).
Small fix via `.after_connect` on the reader pool.
3. **#3622** — CI enforcement of the PG fixtures (with #3619 ordering).
4. **Recovery-conflict cancellation live test** — standby cancels a
routed read mid-snapshot → complete writer page, never partial (#3651's
fix also bounds the cancellation-never-arrives shape).
5. **DDL replication lag test** — migration on writer + paused replay +
budget on → fallback, not client-visible SQL errors.
6. **Reader-tx soak** — 30-60min mixed load; no idle-in-tx accumulation
on the standby.

Then Max's three deployment tests (reader-down latency, reader-down cold
boot, paused-replay read-your-own-write search) re-run on the fixed
binary before the first nonzero budget on bb-block.

---------

Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
Signed-off-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
2026-07-29 20:35:15 -04:00
Kalvin C
d500c2d5cf feat(invites): add use-limited invite links (#3141)
## Summary

- add database-backed v2 invite links with optional maximum-use limits
and atomic final-slot redemption
- preserve v1 invite compatibility while adding
exhausted/expired/invalid client handling across desktop, web, and
mobile
- emit structured claim-outcome logs with community, invite ID, outcome,
maximum uses, and post-claim count

## Verification

- `cargo fmt --all -- --check`
- `cargo test -p buzz-db` (85 passed, 134 Postgres-dependent ignored)
- `cargo clippy -p buzz-db --all-targets -- -D warnings`
- desktop `npm run typecheck`
- push hook: desktop checks/tests, desktop Tauri tests, Rust tests, and
branch-skew passed
- Postgres integration tests were previously reviewed green at the
pre-rebase tree; local rerun on this session was unavailable because
Postgres/Docker were not running
- mobile push-hook check could not start because Flutter is unavailable
locally

---------

Signed-off-by: Kalvin Chau <kalvin@block.xyz>
Signed-off-by: npub1dccv64krpcpse5cmkzfeh998cftungyatw3djt8jwdw6g43f7fyqzzmrf7 <6e30cd56c30e030cd31bb0939b94a7c257c9a09d5ba2d92cf2735da45629f248@buzz.block.builderlab.xyz>
Co-authored-by: npub1c4alndp82zyt9veaklm5d965quss79vlhk9awv7qu5erwhmf42qqlvc25c <c57bf9b4275088b2b33db7f746975407210f159fbd8bd733c0e532375f69aa80@buzz.block.builderlab.xyz>
Co-authored-by: npub122y0pqkertljmedu303rl0aqrj3w8pvu43t6jxm6875lzg6f2pwqegc3xc <5288f082d91aff2de5bc8be23fbfa01ca2e3859cac57a91b7a3fa9f12349505c@buzz.block.builderlab.xyz>
Co-authored-by: npub1dccv64krpcpse5cmkzfeh998cftungyatw3djt8jwdw6g43f7fyqzzmrf7 <6e30cd56c30e030cd31bb0939b94a7c257c9a09d5ba2d92cf2735da45629f248@buzz.block.builderlab.xyz>
2026-07-27 15:19:39 -07:00
Tyler
e43b2d5aac relay: gate push enqueue on live leases; batch matcher pipeline (T1b/T1a-repair/T2b) (#2145)
Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
2026-07-19 21:42:58 -04:00
Tyler
29c48883d3 Route lag-tolerant reads to an optional Postgres read replica (#2084)
Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
2026-07-18 22:45:33 -04:00
Tyler
bffbc5f22c feat(push): deliver accepted relay events as wakes (#1866)
Signed-off-by: npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757 <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757 <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@sprout-oss.stage.blox.sqprod.co>
2026-07-14 16:13:04 -04:00
Kalvin C
2b15a72675 feat(relay): add durable community archival (#1834)
Signed-off-by: npub16ynjrn8tm9gxg4kpgfgj24eup8met48z58m9g0cvmpq9msjht0lq7r96r6 <d12721ccebd9506456c1425125573c09f795d4e2a1f6543f0cd8405dc2575bfe@sprout-oss.stage.blox.sqprod.co>
Signed-off-by: npub1ks3yhswfl2z70myka7wltdg39qpvqhspt2w3l2ajtaee8tmz5qkqrvl4et <b4224bc1c9fa85e7ec96ef9df5b5112802c05e015a9d1fabb25f7393af62a02c@sprout-oss.stage.blox.sqprod.co>
Signed-off-by: npub1dccv64krpcpse5cmkzfeh998cftungyatw3djt8jwdw6g43f7fyqzzmrf7 <6e30cd56c30e030cd31bb0939b94a7c257c9a09d5ba2d92cf2735da45629f248@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: npub16ynjrn8tm9gxg4kpgfgj24eup8met48z58m9g0cvmpq9msjht0lq7r96r6 <d12721ccebd9506456c1425125573c09f795d4e2a1f6543f0cd8405dc2575bfe@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: npub1ks3yhswfl2z70myka7wltdg39qpvqhspt2w3l2ajtaee8tmz5qkqrvl4et <b4224bc1c9fa85e7ec96ef9df5b5112802c05e015a9d1fabb25f7393af62a02c@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: npub1dccv64krpcpse5cmkzfeh998cftungyatw3djt8jwdw6g43f7fyqzzmrf7 <6e30cd56c30e030cd31bb0939b94a7c257c9a09d5ba2d92cf2735da45629f248@sprout-oss.stage.blox.sqprod.co>
2026-07-14 16:02:12 +00:00
Tyler
1c006822e4 feat(push): add public APNs gateway (#1770)
Signed-off-by: npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757 <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757 <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@sprout-oss.stage.blox.sqprod.co>
2026-07-14 11:17:21 -04:00
Tyler
863aeb79f3 Community moderation Phase 1: reports, bans/timeouts, audit, tombstones, relay-DM notices (#1616)
Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
Signed-off-by: tlongwell-block <109685178+tlongwell-block@users.noreply.github.com>
Signed-off-by: npub1mprnacetjua2xx3p5eddmhxyk6wv929ymm5py8kd2xfxurxahspqqlgyta <d8473ee32b973aa31a21a65adddcc4b69cc2a8a4dee8121ecd51926e0cddbc02@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
Co-authored-by: npub17jjz49l9jjmhhk7cac63j8yt9z555n9cw8vk7v5jz4vzw4ppld5qgj57cc <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: npub1cc3ha7z055mu0rwwu7806t2wt8mj3pvu0uv5mfp2c50dahaqhczshdalg6 <c6237ef84fa537c78dcee78efd2d4e59f728859c7f194da42ac51ededfa0be05@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: npub1jmc9dt2lyvzu3h0kxlwxt5zg4fxp9476awyxw6gwxn72g6cw7exqs64whm <96f056ad5f2305c8ddf637dc65d048aa4c12d7daeb8867690e34fca46b0ef64c@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: npub1mprnacetjua2xx3p5eddmhxyk6wv929ymm5py8kd2xfxurxahspqqlgyta <d8473ee32b973aa31a21a65adddcc4b69cc2a8a4dee8121ecd51926e0cddbc02@sprout-oss.stage.blox.sqprod.co>
2026-07-08 13:36:46 -04:00
Will Pfleger
71265ca361 feat(nips,relay,acp): NIP-AM durable encrypted agent turn metrics (kind 44200) (#1441)
Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: npub1fgdl5qqnh3k3f2xkqrvt7cujalhm623x4s7fdjdj5yrtp5fzjl9qrjpucw <4a1bfa0013bc6d14a8d600d8bf6392efefbd2a26ac3c96c9b2a106b0d12297ca@sprout-oss.stage.blox.sqprod.co>
2026-07-06 16:35:10 +00:00
Tyler
14fba21e57 Multi-tenant Buzz relay: community_id as a server-resolved key (comprehensive rewrite) (#1321)
Signed-off-by: tlongwell-block <109685178+tlongwell-block@users.noreply.github.com>
Signed-off-by: npub1jh9wn95s0472h86ahapupaf7m6kx4v9sx2n0atj2hltcfer8k06s5n3pyf <95cae996907d7cab9f5dbf43c0f53edeac6ab0b032a6feae4abfd784e467b3f5@sprout-oss.stage.blox.sqprod.co>
Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
Signed-off-by: npub1t2tgm7d8f995uqvmnm8h88sg3wnpp9a5xysjf6dg3tjmgt3ltulqdp8ehr <5a968df9a7494b4e019b9ecf739e088ba61097b4312124e9a88ae5b42e3f5f3e@sprout-oss.stage.blox.sqprod.co>
Signed-off-by: npub17jjz49l9jjmhhk7cac63j8yt9z555n9cw8vk7v5jz4vzw4ppld5qgj57cc <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: Eva <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: Mari <95cae996907d7cab9f5dbf43c0f53edeac6ab0b032a6feae4abfd784e467b3f5@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: Sami <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: Max <d8473ee32b973aa31a21a65adddcc4b69cc2a8a4dee8121ecd51926e0cddbc02@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: Quinn <96f056ad5f2305c8ddf637dc65d048aa4c12d7daeb8867690e34fca46b0ef64c@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: Dawn <c6237ef84fa537c78dcee78efd2d4e59f728859c7f194da42ac51ededfa0be05@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
Co-authored-by: Sami <sami@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: npub1t2tgm7d8f995uqvmnm8h88sg3wnpp9a5xysjf6dg3tjmgt3ltulqdp8ehr <5a968df9a7494b4e019b9ecf739e088ba61097b4312124e9a88ae5b42e3f5f3e@sprout-oss.stage.blox.sqprod.co>
2026-06-29 12:39:02 -04:00
Will Pfleger
26563968cb feat(relay): add NIP-ER push scheduler with cross-pod delivery (#957)
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@sprout-oss.stage.blox.sqprod.co>
2026-06-17 10:43:22 -04:00
tlongwell-block
2300248d3b Add automatic database migrations (#988)
Signed-off-by: npub1mprnacetjua2xx3p5eddmhxyk6wv929ymm5py8kd2xfxurxahspqqlgyta <d8473ee32b973aa31a21a65adddcc4b69cc2a8a4dee8121ecd51926e0cddbc02@sprout-oss.stage.blox.sqprod.co>
Signed-off-by: npub1t2tgm7d8f995uqvmnm8h88sg3wnpp9a5xysjf6dg3tjmgt3ltulqdp8ehr <5a968df9a7494b4e019b9ecf739e088ba61097b4312124e9a88ae5b42e3f5f3e@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: npub1mprnacetjua2xx3p5eddmhxyk6wv929ymm5py8kd2xfxurxahspqqlgyta <d8473ee32b973aa31a21a65adddcc4b69cc2a8a4dee8121ecd51926e0cddbc02@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: npub1t2tgm7d8f995uqvmnm8h88sg3wnpp9a5xysjf6dg3tjmgt3ltulqdp8ehr <5a968df9a7494b4e019b9ecf739e088ba61097b4312124e9a88ae5b42e3f5f3e@sprout-oss.stage.blox.sqprod.co>
2026-06-16 08:39:19 -04:00
Will Pfleger
d99ad131f1 refactor: rename sprout backend to buzz (#958)
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <wpfleger@block.xyz>
Signed-off-by: Will Pfleger <wpfleger@squareup.com>
Signed-off-by: Will Pfleger <wpfleger96@gmail.com>
Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: npub1fgdl5qqnh3k3f2xkqrvt7cujalhm623x4s7fdjdj5yrtp5fzjl9qrjpucw <4a1bfa0013bc6d14a8d600d8bf6392efefbd2a26ac3c96c9b2a106b0d12297ca@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: npub16v54tttfqacx9ycvc3k0ut0npj564ahcuajzy6qjvh57ntmsf4uq4806j2 <d32955ad69077062930cc46cfe2df30ca9aaf6f8e76422681265e9e9af704d78@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: Will Pfleger <wpfleger96@gmail.com>
2026-06-10 19:29:51 -04:00
tlongwell-block
79399d9b8e feat: NIP-IA identity archival (relay backend + desktop) (#733)
Signed-off-by: tlongwell-block <109685178+tlongwell-block@users.noreply.github.com>
2026-05-23 12:03:38 -04:00
tlongwell-block
3cabe94c2b feat: relay membership with NIP-43 compliance (#448) 2026-05-01 19:11:32 -04:00
tlongwell-block
9ab4799766 schema: add partial index on channel_members.pubkey (#359) 2026-04-29 09:41:57 -04:00
tlongwell-block
b8d6a5cf8b feat: NIP-AB device pairing — Phase 1 (core library + CLI) (#333) 2026-04-16 14:56:36 -04:00
Bradley Axen
c07b14bd9f feat: ephemeral channels with TTL-based auto-archiving (#232)
Co-authored-by: Wes <wesbillman@users.noreply.github.com>
2026-04-07 11:37:51 -07:00
tlongwell-block
67706c93be NIP-33: d-tag keyed replacement for parameterized replaceable events (#246) 2026-04-06 21:15:01 -04:00
Wes
dfe32bfb2c feat: add hide/close DM support (Slack-style DM management) (#157)
Co-authored-by: Tyler Longwell <tlongwell@squareup.com>
2026-03-22 09:00:45 -07:00
Alec Thomas
6093fc6066 feat: migrate from MySQL to Postgres with pgschema (#114)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Tyler Longwell <tlongwell@squareup.com>
2026-03-19 12:35:22 -04:00