mirror of
https://github.com/block/buzz.git
synced 2026-10-30 05:51:46 +00:00
Pinky, an AI agent, updated this description on Wes's behalf after
taking over the startup investigation.
**Category:** fix
**User Impact:** An EVENT refused by WebSocket admission or handler
saturation receives a correlated `OK(event_id, false, reason)` instead
of an uncorrelated NOTICE, so the client can settle that refusal without
waiting for its publish timeout. Rate-limited refusals also arm client
backoff. This fixes a protocol failure mechanism; it does not establish
that every startup send will succeed or that the reported Desktop
startup incident is fully resolved.
**Problem:** Startup opens several live subscriptions and publishes at
once, and the relay's WebSocket admission gate is a fixed 5-second
window (`ws_admission_budget` = `human_ws_events_per_sec * 5`). If that
shared per-principal quota is exhausted, `enforce_ws_admission`
previously rejected an EVENT with a bare `["NOTICE", reason]`. Quota
pressure is a possible trigger, not proof of the original incident's
complete cause.
A NOTICE carries no event id. Both clients settle a pending publish
*only* from an `OK` keyed by event id (desktop `pendingEvents`, mobile
`_pendingEvents`), so nothing settled — and `handle_text_message`
returns early, so no `OK` ever followed either. The send **could not
fail**; it could only time out at `PUBLISH_TIMEOUT_MS` = 25s. That
explains how this rejection mechanism can produce a roughly 25-second
timeout; attributing the original report to it still requires the actual
startup/send workflow.
The handler-semaphore saturation path had the identical defect, and that
one needs no quota burst to fire.
**Solution:** NIP-01 gives each request type its own acknowledgement
channel, and a rejection is only actionable on the same one. Reject a
REQ with `CLOSED`, an EVENT with `OK(id, false, reason)`, and fall back
to `NOTICE` only where no per-request correlation exists. COUNT refusals
now also use `CLOSED(query_id, reason)` per NIP-45, covering both quota
admission and handler saturation (added in
`cd12c93804b87a24b61075dfd171dc471a0a527f`).
Reason strings are unchanged, so the `rate-limited:` prefix and `retry
in {N}s` hint that existing client gates parse keep working (desktop
`parseRateLimitHint`, mobile `RelayRateLimitGate`, buzz-acp
`set_rate_limit_gate`). Only the frame *type* changes, so
`docs/multi-tenant-relay.md` L7 stays satisfied.
Two notes on how this landed, both worth a reviewer's attention:
1. **A survived mutation became a design change.**
`send_admission_result` originally took a `RejectionTarget` parameter,
and reverting the *second* call site (the per-minute message quota)
survived the whole suite — with Redis unreachable the first quota check
short-circuits, so that line is unreachable in test. Rather than test
around it, the parameter is gone: the target is derived from the frame,
so no call site can name the wrong channel.
2. **The relay fix would have caused a client regression on its own.**
Gate arming lived only in the NOTICE branch. Once rejections arrive as
`OK:false`, `handleOk` failed the send without ever backing off — the
client would retry straight into the same quota. Desktop and Mobile now
arm on a `rate-limited:` OK rejection. ACP was subsequently fixed in
`3b06dd32493596ec650f20abf8805791c50fdc24`: it arms the gate and
re-parks only the refused observer frame, preserving other in-flight
frames. Desktop gets `activateRateLimitIfSignalled` as the single owner
of that prefix test, called from both `handleOk` and the NOTICE branch.
<details>
<summary>File changes</summary>
**crates/buzz-relay/src/rejection.rs** (new)
Owns the admission-rejection concern: `RejectionTarget`,
`rejection_target_for`, `request_rejection_message`,
`send_admission_result`, and `enforce_ws_admission`, moved out of
`connection.rs`. Six tests, two of which drive the real
`enforce_ws_admission` against a real `AppState`.
**crates/buzz-relay/src/connection.rs**
Fix the EVENT handler-semaphore rejection to correlate to the event id;
delegate admission to the new module. Add two tests that drive the real
`handle_text_message` with every handler permit held. Down from 1319 to
1116 lines.
**crates/buzz-relay/src/state.rs**
Widen the existing `test_state` helper to `pub(crate)` so the rejection
tests reuse it rather than adding a ninth copy of `AppState`
construction.
**desktop/src/shared/api/relayRateLimitGate.ts**
Add `activateRateLimitIfSignalled` — one owner for the `rate-limited:`
prefix test, since three inbound frame types now carry it.
**desktop/src/shared/api/relayClientSession.ts**
Arm the gate on a rate-limited OK rejection; route the NOTICE branch
through the same helper. Net zero lines, which keeps this
already-oversized file within the differential ratchet.
**desktop/src/shared/api/relayClientPublishRejection.test.mjs** (new)
Four tests against the real `RelayClient`: a rate-limited OK settles the
pending publish and arms the gate; an ordinary rejection does not arm
it; an accepted OK still resolves.
**mobile/lib/shared/relay/relay_session.dart**
Arm the gate in `_handleOk` for a rate-limited rejection.
**mobile/test/shared/relay/relay_session_test.dart**
Two tests driving the real `publish` + `debugHandleMessage` path.
</details>
<details>
<summary>Validation</summary>
**Mutation-tested — 5 mutations, all now killed.** Each production call
site was reverted to the defective behaviour to confirm a test fails.
This caught two false-negative tests:
| # | Mutation | Result |
|---|----------|--------|
| 1 | `rejection_target_for`: EVENT → `Connection` | 4 tests fail |
| 2 | EVENT handler-semaphore call site → bare NOTICE | **survived at
first** |
| 3 | per-minute quota call site → `Connection` | **survived**; fixed by
removing the parameter |
| 4 | desktop `handleOk` gate arming removed | 1 test fails |
| 5 | mobile `_handleOk` gate arming removed | 1 test fails |
Mutation 2 is the lesson: my first saturation test called
`request_rejection_message` directly, so reverting the real call site
inside the `match` arm left it green. It now drives
`handle_text_message` itself and dies on that mutation.
- `cargo test -p buzz-relay` — 928 passed, 1 failed:
`api::mesh_demo::tests::demo_join_forwarded_arm_round_trips_echo`,
**pre-existing**, reproduced with all changes stashed at `4dd4d73de`.
- `cd desktop && npm test` — 5721 passed, 0 failed (full suite).
- `cd mobile && flutter test` — 1876 passed, 0 failed (full suite).
- `just fmt-check`, `just clippy`, `just desktop-check`, `just
mobile-check`, `just file-size-check` — clean. Desktop's 5 biome
warnings are pre-existing (reproduced with changes stashed).
- All 9 pre-push lanes green, including `rust-tests` and
`desktop-tauri-checks`.
**Not verified:** not reproduced end-to-end against a live relay under a
forced quota burst. The causal chain is source-proven and
mutation-proven at the frame level; the ~25s attribution follows from
`PUBLISH_TIMEOUT_MS` but is not directly measured. A packaged-build
click-through would close that gap.
</details>
Related work: #6957 bounds Desktop HTTP event submission, but safe
retained-operation recovery after exhausted/ambiguous outcomes remains
unfinished. #6998 is the separately reviewable Desktop
readiness/duplicate-subscription slice. Neither is claimed to complete
native before/after startup-send validation.
Diagnosis note: `RESEARCH/DESKTOP_STARTUP_SEND_STALL_2026_08_27.md`
(Brain's workspace).
## Current review disposition (2026-08-28)
The [review on
`cd12c938`](https://github.com/block/buzz/pull/6961#pullrequestreview-5052902510)
identified ACP's missing rate-limited-OK handling. Commit
`3b06dd32493596ec650f20abf8805791c50fdc24` fixes gate arming, re-parking
the specifically refused observer frame, and the stale NOTICE comment.
Two regressions drive the real frame dispatcher. See [the implementation
and validation
response](https://github.com/block/buzz/pull/6961#issuecomment-5455032054).
The Mobile generation-check inline thread is resolved: its `async
publish` returns a failed Future when superseded; it does not throw
synchronously at invocation. No further production change was indicated
by that comment.
The validation counts above describe the original slice, not a new
rerun. At `3b06dd324`, the current GitHub check rollup has successful
completed test/build checks (non-applicable jobs skipped). The
security-review comment still requires review for the current base/head
range; do not read a green authorization job as a completed security
review. Approval and merge remain human decisions.
---------
Signed-off-by: Brain <1a02c72794dcd0f07058a353bc3a81f4028b8c77c92c87fce6d5c8b85970a20b@buzz.block.builderlab.xyz>
Signed-off-by: Wes <wesbillman@users.noreply.github.com>
Co-authored-by: Brain <1a02c72794dcd0f07058a353bc3a81f4028b8c77c92c87fce6d5c8b85970a20b@buzz.block.builderlab.xyz>
Co-authored-by: Carl <9d00794d3df50972eb8b615511783cab12a77a8fd5dd5edd58073ec73b54bd8b@buzz.block.builderlab.xyz>
342 lines
9.1 KiB
TypeScript
342 lines
9.1 KiB
TypeScript
import { Channel, invoke } from "@tauri-apps/api/core";
|
|
|
|
import { createAuthEvent } from "@/shared/api/tauri";
|
|
import type { RelayEvent } from "@/shared/api/types";
|
|
import {
|
|
getTextPayload,
|
|
sortEvents,
|
|
toRelayFrames,
|
|
type RelaySubscriptionFilter,
|
|
} from "@/shared/api/relayClientShared";
|
|
import { closeWebSocket } from "@/shared/api/relayWebSocketClose";
|
|
import {
|
|
activateRateLimitIfSignalled,
|
|
waitForRateLimit,
|
|
} from "@/shared/api/relayRateLimitGate";
|
|
import {
|
|
AUTH_TIMEOUT_MS,
|
|
HISTORY_TIMEOUT_MS,
|
|
PUBLISH_TIMEOUT_MS,
|
|
} from "@/shared/api/relayClientTimings";
|
|
|
|
type PendingHistory = {
|
|
events: RelayEvent[];
|
|
resolve: (events: RelayEvent[]) => void;
|
|
reject: (error: Error) => void;
|
|
timeout: number;
|
|
};
|
|
|
|
type PendingPublish = {
|
|
resolve: () => void;
|
|
reject: (error: Error) => void;
|
|
timeout: number;
|
|
};
|
|
|
|
/**
|
|
* Minimal relay session for inactive-community observation (and the rail's
|
|
* cross-relay "mark all as read" publish). It never reads or mutates the
|
|
* active community backend relay URL; callers pass an explicit URL and should
|
|
* disconnect as soon as their polling batch finishes.
|
|
*/
|
|
export class ReadOnlyRelayClient {
|
|
private wsId: number | null = null;
|
|
private onMessageChannel: Channel<unknown> | null = null;
|
|
private connectPromise: Promise<void> | null = null;
|
|
private authRequest: {
|
|
pendingEventId: string;
|
|
resolve: () => void;
|
|
reject: (error: Error) => void;
|
|
timeout: number;
|
|
} | null = null;
|
|
private histories = new Map<string, PendingHistory>();
|
|
private publishes = new Map<string, PendingPublish>();
|
|
private generation = 0;
|
|
|
|
private readonly relayUrl: string;
|
|
|
|
constructor(relayUrl: string) {
|
|
this.relayUrl = relayUrl;
|
|
}
|
|
|
|
async connect(): Promise<void> {
|
|
if (this.wsId !== null) return;
|
|
if (this.connectPromise) return this.connectPromise;
|
|
|
|
const promise = this.openConnection();
|
|
this.connectPromise = promise;
|
|
try {
|
|
await promise;
|
|
} finally {
|
|
if (this.connectPromise === promise) {
|
|
this.connectPromise = null;
|
|
}
|
|
}
|
|
}
|
|
|
|
disconnect(): void {
|
|
const error = new Error("Read-only relay observer disconnected.");
|
|
this.generation++;
|
|
|
|
if (this.wsId !== null) {
|
|
void closeWebSocket(this.wsId, "observer disconnected");
|
|
this.wsId = null;
|
|
}
|
|
|
|
if (this.authRequest) {
|
|
window.clearTimeout(this.authRequest.timeout);
|
|
this.authRequest.reject(error);
|
|
this.authRequest = null;
|
|
}
|
|
|
|
for (const [subId, pending] of this.histories) {
|
|
window.clearTimeout(pending.timeout);
|
|
pending.reject(error);
|
|
this.histories.delete(subId);
|
|
}
|
|
|
|
for (const [eventId, pending] of this.publishes) {
|
|
window.clearTimeout(pending.timeout);
|
|
pending.reject(error);
|
|
this.publishes.delete(eventId);
|
|
}
|
|
|
|
this.onMessageChannel = null;
|
|
this.connectPromise = null;
|
|
}
|
|
|
|
async fetchEvents(filter: RelaySubscriptionFilter): Promise<RelayEvent[]> {
|
|
await this.connect();
|
|
return this.requestHistory(filter);
|
|
}
|
|
|
|
async publishEvent(event: RelayEvent): Promise<void> {
|
|
await this.connect();
|
|
const generation = this.generation;
|
|
await waitForRateLimit();
|
|
|
|
if (generation !== this.generation || this.wsId === null) {
|
|
throw new Error("Read-only relay socket is not connected.");
|
|
}
|
|
|
|
return new Promise<void>((resolve, reject) => {
|
|
const timeout = window.setTimeout(() => {
|
|
this.publishes.delete(event.id);
|
|
reject(new Error("Timed out publishing to observer relay."));
|
|
}, PUBLISH_TIMEOUT_MS);
|
|
|
|
this.publishes.set(event.id, { resolve, reject, timeout });
|
|
|
|
void this.sendRaw(["EVENT", event]).catch((error) => {
|
|
window.clearTimeout(timeout);
|
|
this.publishes.delete(event.id);
|
|
reject(
|
|
error instanceof Error
|
|
? error
|
|
: new Error("Failed to publish to observer relay."),
|
|
);
|
|
});
|
|
});
|
|
}
|
|
|
|
private async openConnection(): Promise<void> {
|
|
const generation = ++this.generation;
|
|
this.onMessageChannel = new Channel<unknown>((delivery) => {
|
|
for (const message of toRelayFrames(delivery)) {
|
|
void this.handleWsMessage(message, generation);
|
|
}
|
|
});
|
|
|
|
this.wsId = await invoke<number>("plugin:websocket|connect", {
|
|
url: this.relayUrl,
|
|
onMessage: this.onMessageChannel,
|
|
config: {},
|
|
});
|
|
|
|
await new Promise<void>((resolve, reject) => {
|
|
const timeout = window.setTimeout(() => {
|
|
this.authRequest = null;
|
|
this.disconnect();
|
|
reject(new Error("Timed out while authenticating observer relay."));
|
|
}, AUTH_TIMEOUT_MS);
|
|
|
|
this.authRequest = {
|
|
pendingEventId: "",
|
|
resolve,
|
|
reject,
|
|
timeout,
|
|
};
|
|
});
|
|
}
|
|
|
|
private requestHistory(
|
|
filter: RelaySubscriptionFilter,
|
|
): Promise<RelayEvent[]> {
|
|
if (this.wsId === null) {
|
|
return Promise.reject(
|
|
new Error("Read-only relay socket is not connected."),
|
|
);
|
|
}
|
|
|
|
return new Promise<RelayEvent[]>((resolve, reject) => {
|
|
const subId = `observer-history-${crypto.randomUUID()}`;
|
|
const timeout = window.setTimeout(() => {
|
|
this.histories.delete(subId);
|
|
void this.sendRaw(["CLOSE", subId]).catch(() => {});
|
|
reject(new Error("Timed out while loading observer relay history."));
|
|
}, HISTORY_TIMEOUT_MS);
|
|
|
|
this.histories.set(subId, { events: [], resolve, reject, timeout });
|
|
|
|
void this.sendRaw(["REQ", subId, filter]).catch((error) => {
|
|
window.clearTimeout(timeout);
|
|
this.histories.delete(subId);
|
|
reject(
|
|
error instanceof Error
|
|
? error
|
|
: new Error("Failed to request observer relay history."),
|
|
);
|
|
});
|
|
});
|
|
}
|
|
|
|
private async sendRaw(payload: unknown[]): Promise<void> {
|
|
if (this.wsId === null) {
|
|
throw new Error("Read-only relay socket is not connected.");
|
|
}
|
|
|
|
await invoke("plugin:websocket|send", {
|
|
id: this.wsId,
|
|
message: {
|
|
type: "Text",
|
|
data: JSON.stringify(payload),
|
|
},
|
|
});
|
|
}
|
|
|
|
private async handleWsMessage(
|
|
message: unknown,
|
|
generation: number,
|
|
): Promise<void> {
|
|
if (generation !== this.generation) return;
|
|
|
|
if (
|
|
typeof message === "object" &&
|
|
message !== null &&
|
|
"type" in message &&
|
|
(message.type === "Close" || message.type === "Error")
|
|
) {
|
|
this.disconnect();
|
|
return;
|
|
}
|
|
|
|
const payload = getTextPayload(message);
|
|
if (!payload) return;
|
|
|
|
let data: unknown;
|
|
try {
|
|
data = JSON.parse(payload);
|
|
} catch {
|
|
return;
|
|
}
|
|
if (!Array.isArray(data) || data.length === 0) return;
|
|
|
|
const [type, ...rest] = data;
|
|
if (type === "AUTH" && typeof rest[0] === "string") {
|
|
await this.handleAuthChallenge(rest[0], generation);
|
|
return;
|
|
}
|
|
if (type === "EVENT" && typeof rest[0] === "string" && rest[1]) {
|
|
this.histories.get(rest[0])?.events.push(rest[1] as RelayEvent);
|
|
return;
|
|
}
|
|
if (
|
|
type === "OK" &&
|
|
typeof rest[0] === "string" &&
|
|
typeof rest[1] === "boolean"
|
|
) {
|
|
this.handleOk(
|
|
rest[0],
|
|
rest[1],
|
|
typeof rest[2] === "string" ? rest[2] : "",
|
|
);
|
|
return;
|
|
}
|
|
if (type === "EOSE" && typeof rest[0] === "string") {
|
|
this.handleEose(rest[0]);
|
|
}
|
|
}
|
|
|
|
private async handleAuthChallenge(
|
|
challenge: string,
|
|
generation: number,
|
|
): Promise<void> {
|
|
const event = await createAuthEvent({
|
|
challenge,
|
|
relayUrl: this.relayUrl,
|
|
});
|
|
|
|
if (generation !== this.generation || !this.authRequest) return;
|
|
this.authRequest.pendingEventId = event.id;
|
|
await this.sendRaw(["AUTH", event]);
|
|
}
|
|
|
|
private handleOk(eventId: string, success: boolean, message: string): void {
|
|
const publish = this.publishes.get(eventId);
|
|
if (publish) {
|
|
window.clearTimeout(publish.timeout);
|
|
this.publishes.delete(eventId);
|
|
if (success) {
|
|
publish.resolve();
|
|
} else {
|
|
activateRateLimitIfSignalled(message);
|
|
publish.reject(
|
|
new Error(message || "Observer relay rejected the event."),
|
|
);
|
|
}
|
|
return;
|
|
}
|
|
|
|
if (!this.authRequest || this.authRequest.pendingEventId !== eventId) {
|
|
return;
|
|
}
|
|
|
|
window.clearTimeout(this.authRequest.timeout);
|
|
const authRequest = this.authRequest;
|
|
this.authRequest = null;
|
|
|
|
if (success) {
|
|
authRequest.resolve();
|
|
return;
|
|
}
|
|
|
|
authRequest.reject(
|
|
message
|
|
? new Error(message)
|
|
: new Error("Observer relay authentication rejected."),
|
|
);
|
|
this.disconnect();
|
|
}
|
|
|
|
private handleEose(subId: string): void {
|
|
const pending = this.histories.get(subId);
|
|
if (!pending) return;
|
|
|
|
window.clearTimeout(pending.timeout);
|
|
this.histories.delete(subId);
|
|
void this.sendRaw(["CLOSE", subId]).catch(() => {});
|
|
pending.resolve(sortEvents(pending.events));
|
|
}
|
|
}
|
|
|
|
export async function withReadOnlyRelayClient<T>(
|
|
relayUrl: string,
|
|
callback: (client: ReadOnlyRelayClient) => Promise<T>,
|
|
): Promise<T> {
|
|
const client = new ReadOnlyRelayClient(relayUrl);
|
|
try {
|
|
return await callback(client);
|
|
} finally {
|
|
client.disconnect();
|
|
}
|
|
}
|