jerry/buzz
1
0
Fork 0
mirror of https://github.com/block/buzz.git synced 2026-10-27 04:30:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(relay): reject a frame on its own acknowledgement channel (#6961)

Browse Source 
Pinky, an AI agent, updated this description on Wes's behalf after
taking over the startup investigation.

**Category:** fix

**User Impact:** An EVENT refused by WebSocket admission or handler
saturation receives a correlated `OK(event_id, false, reason)` instead
of an uncorrelated NOTICE, so the client can settle that refusal without
waiting for its publish timeout. Rate-limited refusals also arm client
backoff. This fixes a protocol failure mechanism; it does not establish
that every startup send will succeed or that the reported Desktop
startup incident is fully resolved.

**Problem:** Startup opens several live subscriptions and publishes at
once, and the relay's WebSocket admission gate is a fixed 5-second
window (`ws_admission_budget` = `human_ws_events_per_sec * 5`). If that
shared per-principal quota is exhausted, `enforce_ws_admission`
previously rejected an EVENT with a bare `["NOTICE", reason]`. Quota
pressure is a possible trigger, not proof of the original incident's
complete cause.

A NOTICE carries no event id. Both clients settle a pending publish
*only* from an `OK` keyed by event id (desktop `pendingEvents`, mobile
`_pendingEvents`), so nothing settled — and `handle_text_message`
returns early, so no `OK` ever followed either. The send **could not
fail**; it could only time out at `PUBLISH_TIMEOUT_MS` = 25s. That
explains how this rejection mechanism can produce a roughly 25-second
timeout; attributing the original report to it still requires the actual
startup/send workflow.

The handler-semaphore saturation path had the identical defect, and that
one needs no quota burst to fire.

**Solution:** NIP-01 gives each request type its own acknowledgement
channel, and a rejection is only actionable on the same one. Reject a
REQ with `CLOSED`, an EVENT with `OK(id, false, reason)`, and fall back
to `NOTICE` only where no per-request correlation exists. COUNT refusals
now also use `CLOSED(query_id, reason)` per NIP-45, covering both quota
admission and handler saturation (added in
`cd12c93804b87a24b61075dfd171dc471a0a527f`).

Reason strings are unchanged, so the `rate-limited:` prefix and `retry
in {N}s` hint that existing client gates parse keep working (desktop
`parseRateLimitHint`, mobile `RelayRateLimitGate`, buzz-acp
`set_rate_limit_gate`). Only the frame *type* changes, so
`docs/multi-tenant-relay.md` L7 stays satisfied.

Two notes on how this landed, both worth a reviewer's attention:

1. **A survived mutation became a design change.**
`send_admission_result` originally took a `RejectionTarget` parameter,
and reverting the *second* call site (the per-minute message quota)
survived the whole suite — with Redis unreachable the first quota check
short-circuits, so that line is unreachable in test. Rather than test
around it, the parameter is gone: the target is derived from the frame,
so no call site can name the wrong channel.

2. **The relay fix would have caused a client regression on its own.**
Gate arming lived only in the NOTICE branch. Once rejections arrive as
`OK:false`, `handleOk` failed the send without ever backing off — the
client would retry straight into the same quota. Desktop and Mobile now
arm on a `rate-limited:` OK rejection. ACP was subsequently fixed in
`3b06dd32493596ec650f20abf8805791c50fdc24`: it arms the gate and
re-parks only the refused observer frame, preserving other in-flight
frames. Desktop gets `activateRateLimitIfSignalled` as the single owner
of that prefix test, called from both `handleOk` and the NOTICE branch.

<details>
<summary>File changes</summary>

**crates/buzz-relay/src/rejection.rs** (new)
Owns the admission-rejection concern: `RejectionTarget`,
`rejection_target_for`, `request_rejection_message`,
`send_admission_result`, and `enforce_ws_admission`, moved out of
`connection.rs`. Six tests, two of which drive the real
`enforce_ws_admission` against a real `AppState`.

**crates/buzz-relay/src/connection.rs**
Fix the EVENT handler-semaphore rejection to correlate to the event id;
delegate admission to the new module. Add two tests that drive the real
`handle_text_message` with every handler permit held. Down from 1319 to
1116 lines.

**crates/buzz-relay/src/state.rs**
Widen the existing `test_state` helper to `pub(crate)` so the rejection
tests reuse it rather than adding a ninth copy of `AppState`
construction.

**desktop/src/shared/api/relayRateLimitGate.ts**
Add `activateRateLimitIfSignalled` — one owner for the `rate-limited:`
prefix test, since three inbound frame types now carry it.

**desktop/src/shared/api/relayClientSession.ts**
Arm the gate on a rate-limited OK rejection; route the NOTICE branch
through the same helper. Net zero lines, which keeps this
already-oversized file within the differential ratchet.

**desktop/src/shared/api/relayClientPublishRejection.test.mjs** (new)
Four tests against the real `RelayClient`: a rate-limited OK settles the
pending publish and arms the gate; an ordinary rejection does not arm
it; an accepted OK still resolves.

**mobile/lib/shared/relay/relay_session.dart**
Arm the gate in `_handleOk` for a rate-limited rejection.

**mobile/test/shared/relay/relay_session_test.dart**
Two tests driving the real `publish` + `debugHandleMessage` path.

</details>

<details>
<summary>Validation</summary>

**Mutation-tested — 5 mutations, all now killed.** Each production call
site was reverted to the defective behaviour to confirm a test fails.
This caught two false-negative tests:

| # | Mutation | Result |
|---|----------|--------|
| 1 | `rejection_target_for`: EVENT → `Connection` | 4 tests fail |
| 2 | EVENT handler-semaphore call site → bare NOTICE | **survived at
first** |
| 3 | per-minute quota call site → `Connection` | **survived**; fixed by
removing the parameter |
| 4 | desktop `handleOk` gate arming removed | 1 test fails |
| 5 | mobile `_handleOk` gate arming removed | 1 test fails |

Mutation 2 is the lesson: my first saturation test called
`request_rejection_message` directly, so reverting the real call site
inside the `match` arm left it green. It now drives
`handle_text_message` itself and dies on that mutation.

- `cargo test -p buzz-relay` — 928 passed, 1 failed:
`api::mesh_demo::tests::demo_join_forwarded_arm_round_trips_echo`,
**pre-existing**, reproduced with all changes stashed at `4dd4d73de`.
- `cd desktop && npm test` — 5721 passed, 0 failed (full suite).
- `cd mobile && flutter test` — 1876 passed, 0 failed (full suite).
- `just fmt-check`, `just clippy`, `just desktop-check`, `just
mobile-check`, `just file-size-check` — clean. Desktop's 5 biome
warnings are pre-existing (reproduced with changes stashed).
- All 9 pre-push lanes green, including `rust-tests` and
`desktop-tauri-checks`.

**Not verified:** not reproduced end-to-end against a live relay under a
forced quota burst. The causal chain is source-proven and
mutation-proven at the frame level; the ~25s attribution follows from
`PUBLISH_TIMEOUT_MS` but is not directly measured. A packaged-build
click-through would close that gap.

</details>

Related work: #6957 bounds Desktop HTTP event submission, but safe
retained-operation recovery after exhausted/ambiguous outcomes remains
unfinished. #6998 is the separately reviewable Desktop
readiness/duplicate-subscription slice. Neither is claimed to complete
native before/after startup-send validation.

Diagnosis note: `RESEARCH/DESKTOP_STARTUP_SEND_STALL_2026_08_27.md`
(Brain's workspace).

## Current review disposition (2026-08-28)

The [review on
`cd12c938`](https://github.com/block/buzz/pull/6961#pullrequestreview-5052902510)
identified ACP's missing rate-limited-OK handling. Commit
`3b06dd32493596ec650f20abf8805791c50fdc24` fixes gate arming, re-parking
the specifically refused observer frame, and the stale NOTICE comment.
Two regressions drive the real frame dispatcher. See [the implementation
and validation
response](https://github.com/block/buzz/pull/6961#issuecomment-5455032054).

The Mobile generation-check inline thread is resolved: its `async
publish` returns a failed Future when superseded; it does not throw
synchronously at invocation. No further production change was indicated
by that comment.

The validation counts above describe the original slice, not a new
rerun. At `3b06dd324`, the current GitHub check rollup has successful
completed test/build checks (non-applicable jobs skipped). The
security-review comment still requires review for the current base/head
range; do not read a green authorization job as a completed security
review. Approval and merge remain human decisions.

---------

Signed-off-by: Brain <1a02c72794dcd0f07058a353bc3a81f4028b8c77c92c87fce6d5c8b85970a20b@buzz.block.builderlab.xyz>
Signed-off-by: Wes <wesbillman@users.noreply.github.com>
Co-authored-by: Brain <1a02c72794dcd0f07058a353bc3a81f4028b8c77c92c87fce6d5c8b85970a20b@buzz.block.builderlab.xyz>
Co-authored-by: Carl <9d00794d3df50972eb8b615511783cab12a77a8fd5dd5edd58073ec73b54bd8b@buzz.block.builderlab.xyz>
This commit is contained in:
Wes
2026-08-31 14:43:31 -06:00
committed by GitHub
parent 93237b4a74
commit 2f3dd850db
13 changed files with 1448 additions and 175 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
mobile/lib/shared/relay/relay_session.dart
View File

@@ -311,7 +311,13 @@ class RelaySessionNotifier extends Notifier<SessionState> {
Future<NostrEvent> publish(
NostrEvent event, {
Duration timeout = const Duration(seconds: 8),
}) {
}) async {
final generation = _connectionGeneration;
if (_rateLimitGate.isActive) await _rateLimitGate.wait();
if (!_isActiveConnection(generation) || !_socketConnected) {
throw StateError('Relay session is not connected');
}
final completer = Completer<NostrEvent>();
final timer = Timer(timeout, () {
@@ -824,6 +830,13 @@ class RelaySessionNotifier extends Notifier<SessionState> {
);
}
} else {
// Back-pressure now arrives here rather than as a NOTICE: the relay
// rejects an over-quota EVENT on the OK channel so this pending publish
// can be settled at all. Without arming the gate the send would fail
// without ever backing off.
if (message.startsWith('rate-limited:')) {
_rateLimitGate.activate(parseRateLimitRetrySeconds(message));
}
if (!pending.completer.isCompleted) {
pending.completer.completeError(
Exception(message.isNotEmpty ? message : 'Event rejected'),

148
mobile/test/shared/relay/relay_session_test.dart
View File

@@ -1382,6 +1382,150 @@ void main() {
expect(closedMessages, ['restricted: no longer valid']);
unsubscribe();
});
// The relay rejects an over-quota EVENT on the OK channel rather than with a
// bare NOTICE, because a NOTICE carries no event id and `_pendingEvents` is
// keyed by one — nothing settled, so the publish could only time out. The
// gate arming that used to depend on the NOTICE has to happen here too.
test(
'a rate-limited OK rejection fails the publish and arms the gate',
() async {
final gateTimers = <_ManualTimer>[];
final gate = RelayRateLimitGate(
now: () => DateTime(2026),
timerFactory: (duration, callback) {
final timer = _ManualTimer(duration, callback);
gateTimers.add(timer);
return timer;
},
);
final session = RelaySessionNotifier(rateLimitGate: gate);
session.debugAttachSocketForTest(_RecordingRelaySocket());
final publish = session.publish(_event());
session.debugHandleMessage([
'OK',
'event-1',
false,
'rate-limited: quota exceeded; retry in 4s',
]);
await expectLater(publish, throwsA(isA<Exception>()));
expect(
gate.isActive,
isTrue,
reason:
'back-pressure now arrives on the OK channel — without arming here '
'the client fails the send and retries into the same quota',
);
expect(gateTimers.single.duration, const Duration(seconds: 4));
},
);
test(
'publish waits out the rate-limit gate before timeout registration and send',
() async {
final gateTimers = <_ManualTimer>[];
final gate = RelayRateLimitGate(
now: () => DateTime(2026),
timerFactory: (duration, callback) {
final timer = _ManualTimer(duration, callback);
gateTimers.add(timer);
return timer;
},
);
final socket = _RecordingRelaySocket();
final session = RelaySessionNotifier(rateLimitGate: gate);
session.debugAttachSocketForTest(socket);
final firstPublish = session.publish(_event(id: 'event-a'));
session.debugHandleMessage([
'OK',
'event-a',
false,
'rate-limited: quota exceeded; retry in 4s',
]);
await expectLater(firstPublish, throwsA(isA<Exception>()));
var secondSettled = false;
final secondPublish = session.publish(
_event(id: 'event-b'),
timeout: Duration.zero,
);
unawaited(secondPublish.whenComplete(() => secondSettled = true));
await Future<void>.delayed(Duration.zero);
expect(
socket.messages.where((message) => message.first == 'EVENT'),
hasLength(1),
reason: 'the next EVENT must remain unsent while the gate is active',
);
expect(
secondSettled,
isFalse,
reason:
'the publish timeout must not start until after the gate expires',
);
gateTimers.single.fire();
await Future<void>.microtask(() {});
final events = socket.messages
.where((message) => message.first == 'EVENT')
.toList();
expect(events, hasLength(2));
expect((events.last[1] as Map<String, dynamic>)['id'], 'event-b');
session.debugHandleMessage(['OK', 'event-b', true, '']);
expect((await secondPublish).id, 'event-b');
},
);
test(
'a gated publish is cancelled if the connection changes while waiting',
() async {
final gateTimers = <_ManualTimer>[];
final gate = RelayRateLimitGate(
now: () => DateTime(2026),
timerFactory: (duration, callback) {
final timer = _ManualTimer(duration, callback);
gateTimers.add(timer);
return timer;
},
);
final socket = _RecordingRelaySocket();
final session = RelaySessionNotifier(rateLimitGate: gate);
session.debugAttachSocketForTest(socket);
gate.activate(4);
final publish = session.publish(_event(id: 'event-b'));
session.debugSupersedeConnection();
gateTimers.single.fire();
await expectLater(publish, throwsA(isA<StateError>()));
expect(socket.messages, isEmpty);
},
);
test('an ordinary OK rejection does not arm the gate', () async {
final gate = RelayRateLimitGate(now: () => DateTime(2026));
final session = RelaySessionNotifier(rateLimitGate: gate);
session.debugAttachSocketForTest(_RecordingRelaySocket());
final publish = session.publish(_event());
session.debugHandleMessage([
'OK',
'event-1',
false,
'invalid: bad signature',
]);
await expectLater(publish, throwsA(isA<Exception>()));
expect(
gate.isActive,
isFalse,
reason: 'only `rate-limited:` rejections signal back-pressure',
);
});
}
class _ControlledHttpClient extends http.BaseClient {
@@ -1524,9 +1668,9 @@ class _FakeRelayConfigNotifier extends RelayConfigNotifier {
RelayConfig build() => RelayConfig(baseUrl: _baseUrl, nsec: _nsec);
}
NostrEvent _event({int createdAt = 20}) {
NostrEvent _event({int createdAt = 20, String id = 'event-1'}) {
return NostrEvent(
id: 'event-1',
id: id,
pubkey: 'alice',
createdAt: createdAt,
kind: EventKind.streamMessageV2,