mirror of
https://github.com/block/buzz.git
synced 2026-10-27 04:30:52 +00:00
## Summary Mobile does background work nobody sees, and it waits on round trips it doesn't need. This PR fixes four things: - **Chats you've left let go.** A chat with no listener for 30 s drops its live and typing subscriptions. Coming back shows the cached timeline right away, then resubscribes and catches up. - **No requests before AUTH.** The relay rejects any REQ that arrives before NIP-42 AUTH completes. History requests are now held until the socket authenticates, and they flush after the rate-limit gate. Live subscriptions skip their REQ, and the replay on connect sends it. `auth-required` is terminal for a live subscription, so a pre-AUTH live REQ could be dropped for good. - **Resume reconciles the channel list once.** `resumed` is published before the session hears about it. So the channel list now decides one microtask later: if the session is reconnecting, the connect does the reconcile. - **The 60 s backstop only runs in the foreground on a connected session.** After an OS suspension, the missed tick fires at unfreeze, about 450 ms before the app hears `resumed`, while the session still looks connected. Without this, that tick triggered a second full sync. Scope note: while the app is in the foreground, the backstop still re-downloads membership every 60 s. That matches desktop, which refetches every 60 s while focused. To drop it, mobile would need to subscribe to the live membership notices (44100/44101), as desktop does. That would be a follow-up. ### Related issue N/A. This is follow-up work from the mobile perf dive. None found. ## Before and after Simulator setup: iPhone 18 Pro simulator, debug build, a local relay behind a proxy that adds ~120 ms RTT, and a seeded account with 120 channels. I scripted one identical run against each build: 1. Open 5 channels. 2. Visit 20 more channels. 3. Wait 40 s. 4. Stay idle in the foreground for 3 min. 5. Go to the background for 3 min. 6. Resume. `main` is `ca3be5501`; the branch is `d77c4e86f`. The last amend changed only test names. Since then, chat history waits for the live EOSE again (see the note below), so channel open is back to `main`'s order. | | `main` | this PR | |---|---|---| | Live subscriptions after visiting 20 chats | 61 | 11 | | Resume: membership reads | 2 (244 kB) | 1 (122 kB) | | Resume: websocket REQs | 36 | 31 | | Resume: REQs sent before AUTH | 9 | 0 | | Resume: REQs rejected as `auth-required` | 2 | 0 | | Idle foreground, 3 min | 3 membership reads, 12 REQs | same | Notes: - **Channel open:** an earlier revision requested history without waiting for EOSE, which cut the median open from 297 ms to 197 ms. Review showed it can lose a message. The relay registers the live REQ only after its access checks, so a message committed in between reached neither the window read nor the subscription. A timestamp overlap only covers messages signed within it. Keeping that speedup safely would mean a second window read on every open, so it's left out. - **Idle foreground is unchanged by design.** See the scope note above. - **Background traffic:** the simulator suspends the app shortly after it leaves the foreground, so both builds sent almost nothing during the 3 minutes in the background. The backstop fix shows up at unfreeze instead, as the missing second membership read. - **Resume inbound events:** in this one run, events received over the websocket on resume were 247 kB on `main` and 338 kB on the branch. I haven't attributed the difference. ## Test These extend the existing journeys, plus one new journey: - **`channel_messages_journey_test`:** - The window read waits for the live REQ's admission, and a message signed a minute earlier but committed before admission shows up. - After the listener leaves, the chat keeps following for 29 s and stops at 31 s. - Coming back shows the cached rows immediately, then catches up on a message sent while away. - **`relay_session_test`:** a stale-connected resume sends nothing before AUTH. Then exactly one history REQ and one live REQ go out after AUTH. - **New `channels_sync_journey_test`:** drives the real lifecycle, session and channel notifier against `FakeRelay`: - A backgrounding just before a backstop tick sends nothing. - A resume after the session paused syncs once. - An OS-suspension resume with AUTH held syncs once, after AUTH, and sends no REQ before it. - **`FakeRelay`:** gained `holdReq`, `holdAuth`, `before_id` pagination and an injectable clock. Each guard was mutation-checked, and every mutation failed its test: - live REQ sent before AUTH - history sent before AUTH - resume refresh not deferred - backstop without the lifecycle check - history read alongside the live REQ - a 30-day detach delay Follow-up, not in this PR: a history REQ already in flight on a socket that gets replaced still waits for its 8 s timeout. Princess Donut found this in review; it predates this change. ## Validation - `flutter analyze`: clean - Full mobile suite at `678617b`: 3,696 passed - Court review (Princess Donut on the relay session, Mongo on chat subscriptions, Mordecai on the channel list plus a challenge pass): no blocking findings. The rate-limit flush ordering came from that review. **Human test:** Wes ran `da5077d` on the simulator with his real account and confirmed it works. `460dc35` then fixed a review blocker: a chat rebuilt while detached stayed detached. A retest of that path is pending: leave a chat for more than 30 s, background the app for more than 5 s, resume, then reopen the chat. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Wes <wesbillman@users.noreply.github.com> Co-authored-by: Carl <32a2e2c9d428ee08902cab75d956da2c1d235a22d4766b0dd4138bf6e2e5db1d@buzz.block.builderlab.xyz> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
452 lines
14 KiB
Dart
452 lines
14 KiB
Dart
import 'dart:async';
|
|
import 'dart:convert';
|
|
|
|
import 'package:buzz/shared/auth/auth_provider.dart';
|
|
import 'package:buzz/shared/relay/relay.dart';
|
|
import 'package:flutter_test/flutter_test.dart';
|
|
import 'package:hooks_riverpod/misc.dart';
|
|
import 'package:http/http.dart' as http;
|
|
import 'package:http/testing.dart';
|
|
import 'package:nostr/nostr.dart' as nostr;
|
|
|
|
/// An in-memory relay behind the real [RelaySessionNotifier].
|
|
///
|
|
/// Only the process boundary is fake: the websocket (through the session's
|
|
/// socket factory) and the HTTP bridge (through its HTTP client). The session
|
|
/// still connects from `build()`, matches REQ/EOSE/OK/CLOSED, batches events,
|
|
/// backs off and replays live subscriptions. Replies arrive on microtasks, so
|
|
/// tests drive timers with `fakeAsync` or `tester.pump`, never real sleeps.
|
|
///
|
|
/// It is not a relay emulator: authorization, NIP-42 and JSON framing stay
|
|
/// with the relay and socket tests. A request it cannot answer faithfully is
|
|
/// recorded and fails the test at teardown, because app code often catches
|
|
/// relay errors and falls back.
|
|
class FakeRelay {
|
|
/// [now] is the session's wall clock, so a test can model OS suspension:
|
|
/// time passes while no timer runs.
|
|
FakeRelay({DateTime Function()? now}) : _now = now {
|
|
addTearDown(() {
|
|
expect(unanswerable, isEmpty, reason: 'requests FakeRelay cannot answer');
|
|
});
|
|
}
|
|
|
|
final keys = nostr.Keys.generate();
|
|
final stored = <NostrEvent>[];
|
|
final published = <NostrEvent>[];
|
|
final reqs = <List<dynamic>>[];
|
|
final queries = <List<Map<String, dynamic>>>[];
|
|
final sockets = <FakeRelaySocket>[];
|
|
final unanswerable = <String>[];
|
|
final _rejections = <({bool Function(NostrEvent) match, String message})>[];
|
|
Completer<void>? _okHold;
|
|
Completer<void>? _echoHold;
|
|
Completer<void>? _reqHold;
|
|
Completer<void>? _authHold;
|
|
final DateTime Function()? _now;
|
|
|
|
String get pubkey => keys.public;
|
|
|
|
late final session = RelaySessionNotifier(
|
|
socketFactory: _openSocket,
|
|
httpClient: MockClient(_handleHttp),
|
|
now: _now,
|
|
);
|
|
|
|
List<Override> get overrides => [
|
|
relaySessionProvider.overrideWith(() => session),
|
|
relayConfigProvider.overrideWith(() => _FixedRelayConfig(keys.nsec)),
|
|
authProvider.overrideWith(_SignedInAuth.new),
|
|
];
|
|
|
|
/// Stores history without notifying live subscriptions.
|
|
void seed(Iterable<NostrEvent> events) => stored.addAll(events);
|
|
|
|
/// Another client published [event]: store it and fan it out.
|
|
void deliver(NostrEvent event) {
|
|
stored.add(event);
|
|
_fanOut(event);
|
|
}
|
|
|
|
/// Signs an event as [author] (default: a fresh member, not the app user).
|
|
NostrEvent sign({
|
|
required int kind,
|
|
required List<List<String>> tags,
|
|
String content = '',
|
|
int? createdAt,
|
|
nostr.Keys? author,
|
|
}) {
|
|
final event = nostr.Event.from(
|
|
kind: kind,
|
|
tags: tags,
|
|
content: content,
|
|
createdAt: createdAt,
|
|
secretKey: (author ?? nostr.Keys.generate()).secret,
|
|
verify: false,
|
|
);
|
|
return NostrEvent.fromJson(event.toMap());
|
|
}
|
|
|
|
/// The next published event that [match]es is answered `OK false` and is
|
|
/// neither stored nor fanned out.
|
|
void rejectNext(String message, {bool Function(NostrEvent)? match}) =>
|
|
_rejections.add((match: match ?? (_) => true, message: message));
|
|
|
|
/// Withholds OK for accepted events until the completer completes.
|
|
Completer<void> holdOk() => _okHold = Completer<void>();
|
|
|
|
/// Withholds live fan-out of accepted events until the completer completes.
|
|
/// The relay dispatches after commit, so echo and OK have no fixed order.
|
|
Completer<void> holdEcho() => _echoHold = Completer<void>();
|
|
|
|
/// Holds new REQs until the completer completes, as a relay's access checks
|
|
/// would: they read stored events, start live fan-out and reach EOSE only
|
|
/// once admitted. Events committed meanwhile reach them only through replay.
|
|
Completer<void> holdReq() => _reqHold = Completer<void>();
|
|
|
|
/// Keeps the next connection unauthenticated until the completer completes.
|
|
/// Like the relay, it answers a REQ sent before AUTH with CLOSED.
|
|
Completer<void> holdAuth() => _authHold = Completer<void>();
|
|
|
|
/// REQs the relay rejected because they arrived before AUTH.
|
|
final reqsBeforeAuth = <List<dynamic>>[];
|
|
|
|
/// Filters of the live subscriptions open on the current connection.
|
|
Iterable<List<Map<String, dynamic>>> get liveFilters =>
|
|
sockets.isEmpty ? const [] : sockets.last._live.values;
|
|
|
|
/// Drops the current connection, as a network loss would. Its live
|
|
/// subscriptions and queued replies die with it; stored history survives.
|
|
void drop([Object? error]) => sockets.last.drop(error);
|
|
|
|
RelaySocket _openSocket({
|
|
required String wsUrl,
|
|
required String? nsec,
|
|
required void Function(List<dynamic> message) onMessage,
|
|
required void Function() onConnected,
|
|
required void Function(Object? error) onDisconnected,
|
|
}) {
|
|
final socket = FakeRelaySocket._(
|
|
this,
|
|
onMessage: onMessage,
|
|
onConnected: onConnected,
|
|
onDisconnected: onDisconnected,
|
|
);
|
|
sockets.add(socket);
|
|
return socket;
|
|
}
|
|
|
|
void _receive(FakeRelaySocket socket, List<dynamic> message) {
|
|
switch (message.first) {
|
|
case 'REQ':
|
|
final subId = message[1] as String;
|
|
if (!socket._authenticated) {
|
|
reqsBeforeAuth.add(message);
|
|
scheduleMicrotask(
|
|
() => socket._reply([
|
|
'CLOSED',
|
|
subId,
|
|
'auth-required: not authenticated',
|
|
]),
|
|
);
|
|
return;
|
|
}
|
|
reqs.add(message);
|
|
final filters = message.skip(2).cast<Map<String, dynamic>>().toList();
|
|
void admit() {
|
|
if (!socket._authenticated) return;
|
|
final history = _query(filters);
|
|
if (history == null) return;
|
|
socket._live[subId] = filters;
|
|
scheduleMicrotask(() {
|
|
for (final event in history) {
|
|
socket._reply(['EVENT', subId, event.toJson()]);
|
|
}
|
|
socket._reply(['EOSE', subId]);
|
|
});
|
|
}
|
|
|
|
final hold = _reqHold;
|
|
hold == null ? admit() : hold.future.then((_) => admit());
|
|
case 'CLOSE':
|
|
socket._live.remove(message[1]);
|
|
case 'EVENT':
|
|
_publish(
|
|
socket,
|
|
NostrEvent.fromJson(message[1] as Map<String, dynamic>),
|
|
);
|
|
default:
|
|
unanswerable.add('${message.first} frame');
|
|
}
|
|
}
|
|
|
|
void _publish(FakeRelaySocket socket, NostrEvent event) {
|
|
published.add(event);
|
|
final index = _rejections.indexWhere((r) => r.match(event));
|
|
if (index >= 0) {
|
|
final message = _rejections.removeAt(index).message;
|
|
scheduleMicrotask(() => socket._reply(['OK', event.id, false, message]));
|
|
return;
|
|
}
|
|
stored.add(event);
|
|
final okHold = _okHold?.future;
|
|
final echoHold = _echoHold?.future;
|
|
Future<void> ok() async {
|
|
await okHold;
|
|
socket._reply(['OK', event.id, true, '']);
|
|
}
|
|
|
|
Future<void> echo() async {
|
|
await echoHold;
|
|
_fanOut(event);
|
|
}
|
|
|
|
scheduleMicrotask(ok);
|
|
scheduleMicrotask(echo);
|
|
}
|
|
|
|
void _fanOut(NostrEvent event) {
|
|
for (final socket in sockets) {
|
|
for (final MapEntry(key: subId, value: filters) in socket._live.entries) {
|
|
if (filters.any((f) => _matches(f, event, live: true))) {
|
|
socket._reply(['EVENT', subId, event.toJson()]);
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
/// Stored events for [filters], or null after recording why it cannot
|
|
/// answer. Each filter applies its own limit before the union, as the
|
|
/// relay does.
|
|
List<NostrEvent>? _query(List<Map<String, dynamic>> filters) {
|
|
if (!_answerable(filters)) return null;
|
|
final result = <String, NostrEvent>{};
|
|
for (final filter in filters) {
|
|
final limit = filter['limit'] as int?;
|
|
final matches = stored.where((e) => _matches(filter, e)).toList()
|
|
..sort(_newestFirst);
|
|
for (final event in limit == null ? matches : matches.take(limit)) {
|
|
result[event.id] = event;
|
|
}
|
|
}
|
|
return result.values.toList()..sort(_newestFirst);
|
|
}
|
|
|
|
/// Whether [_matches] implements every key in [filters]; records why not.
|
|
/// Checked before matching, so an empty store cannot hide it.
|
|
bool _answerable(List<Map<String, dynamic>> filters) {
|
|
final unknown = {
|
|
for (final filter in filters)
|
|
...filter.keys.where(
|
|
(key) => !_filterKeys.contains(key) && !key.startsWith('#'),
|
|
),
|
|
};
|
|
if (unknown.isEmpty) return true;
|
|
unanswerable.add('filter keys ${unknown.toList()}');
|
|
return false;
|
|
}
|
|
|
|
/// NIP-01 matching for a filter [_answerable] accepted.
|
|
bool _matches(
|
|
Map<String, dynamic> filter,
|
|
NostrEvent event, {
|
|
bool live = false,
|
|
}) {
|
|
if (!live && filter['limit'] == 0) return false;
|
|
bool allows(String key, Object value) =>
|
|
(filter[key] as List?)?.contains(value) ?? true;
|
|
if (!allows('ids', event.id) ||
|
|
!allows('authors', event.pubkey) ||
|
|
!allows('kinds', event.kind)) {
|
|
return false;
|
|
}
|
|
final since = filter['since'] as int?;
|
|
final until = filter['until'] as int?;
|
|
if (since != null && event.createdAt < since) return false;
|
|
if (until != null && event.createdAt > until) return false;
|
|
// Buzz pagination: strictly after the cursor in `created_at DESC, id ASC`.
|
|
final beforeId = filter['before_id'] as String?;
|
|
if (beforeId != null &&
|
|
event.createdAt == until &&
|
|
event.id.compareTo(beforeId) <= 0) {
|
|
return false;
|
|
}
|
|
for (final MapEntry(:key, :value) in filter.entries) {
|
|
if (!key.startsWith('#')) continue;
|
|
final name = key.substring(1);
|
|
final wanted = (value as List).cast<String>();
|
|
if (!event.tags.any(
|
|
(tag) => tag.length > 1 && tag[0] == name && wanted.contains(tag[1]),
|
|
)) {
|
|
return false;
|
|
}
|
|
}
|
|
return true;
|
|
}
|
|
|
|
Future<http.Response> _handleHttp(http.Request request) async {
|
|
if (request.method != 'POST' ||
|
|
request.url.path != '/query' ||
|
|
request.headers['Authorization'] == null) {
|
|
unanswerable.add('${request.method} ${request.url.path}');
|
|
return http.Response('unanswerable', 501);
|
|
}
|
|
final filters = (jsonDecode(request.body) as List)
|
|
.cast<Map<String, dynamic>>();
|
|
queries.add(filters);
|
|
final events = filters.length == 1 && filters.single['top_level'] == true
|
|
? _channelWindow(filters.single)
|
|
: _query(filters);
|
|
if (events == null) return http.Response('unanswerable', 501);
|
|
return http.Response(jsonEncode([for (final e in events) e.toJson()]), 200);
|
|
}
|
|
|
|
/// The NIP-CW top-level channel window: one page of top-level rows plus its
|
|
/// kind:39006 bounds. Thread summaries and aux events are not synthesized
|
|
/// yet; a journey that needs them adds them here.
|
|
List<NostrEvent>? _channelWindow(Map<String, dynamic> filter) {
|
|
final channel = ((filter['#h'] as List).single as String).toLowerCase();
|
|
final beforeId = filter['before_id'] as String?;
|
|
final until = filter['until'] as int?;
|
|
final limit = filter['limit'] as int;
|
|
final base = {
|
|
for (final MapEntry(:key, :value) in filter.entries)
|
|
if (!_windowKeys.contains(key)) key: value,
|
|
};
|
|
if (!_answerable([base])) return null;
|
|
final rows = <NostrEvent>[];
|
|
for (final event in stored) {
|
|
if (!_matches(base, event) || event.threadReference.parentId != null) {
|
|
continue;
|
|
}
|
|
// Rows strictly after the cursor in `created_at DESC, id ASC` order.
|
|
if (until != null &&
|
|
beforeId != null &&
|
|
!(event.createdAt < until ||
|
|
(event.createdAt == until && event.id.compareTo(beforeId) > 0))) {
|
|
continue;
|
|
}
|
|
rows.add(event);
|
|
}
|
|
rows.sort(_newestFirst);
|
|
final page = rows.take(limit).toList();
|
|
final hasMore = rows.length > limit;
|
|
final cursor = until == null || beforeId == null
|
|
? 'head'
|
|
: '$until:${beforeId.toLowerCase()}';
|
|
return [
|
|
...page,
|
|
NostrEvent(
|
|
id: 'bounds-$channel-$cursor',
|
|
pubkey: 'relay',
|
|
createdAt: 0,
|
|
kind: EventKind.channelWindowBounds,
|
|
tags: [
|
|
['d', '$channel:$cursor'],
|
|
],
|
|
content: jsonEncode({
|
|
'has_more': hasMore,
|
|
'next_cursor': hasMore
|
|
? {'created_at': page.last.createdAt, 'id': page.last.id}
|
|
: null,
|
|
}),
|
|
sig: 'sig',
|
|
),
|
|
];
|
|
}
|
|
}
|
|
|
|
/// The socket the session talks to; [FakeRelay] owns both ends.
|
|
class FakeRelaySocket extends RelaySocket {
|
|
FakeRelaySocket._(
|
|
this._relay, {
|
|
required super.onMessage,
|
|
required super.onConnected,
|
|
required super.onDisconnected,
|
|
}) : _onMessage = onMessage,
|
|
_onConnected = onConnected,
|
|
_onDisconnected = onDisconnected,
|
|
super(wsUrl: 'wss://relay.test', nsec: null);
|
|
|
|
final FakeRelay _relay;
|
|
final void Function(List<dynamic> message) _onMessage;
|
|
final void Function() _onConnected;
|
|
final void Function(Object? error) _onDisconnected;
|
|
final _live = <String, List<Map<String, dynamic>>>{};
|
|
bool _open = false;
|
|
bool _authenticated = false;
|
|
|
|
@override
|
|
Future<void> connect() async {
|
|
_open = true;
|
|
final hold = _relay._authHold;
|
|
_relay._authHold = null;
|
|
if (hold != null) await hold.future;
|
|
if (!_open) return;
|
|
_authenticated = true;
|
|
_onConnected();
|
|
}
|
|
|
|
@override
|
|
void send(List<dynamic> payload) {
|
|
if (_open) _relay._receive(this, payload);
|
|
}
|
|
|
|
@override
|
|
Future<void> disconnect() async => dispose();
|
|
|
|
@override
|
|
void dispose() {
|
|
_open = false;
|
|
_authenticated = false;
|
|
_live.clear();
|
|
}
|
|
|
|
void drop(Object? error) {
|
|
if (!_open) return;
|
|
dispose();
|
|
_onDisconnected(error);
|
|
}
|
|
|
|
void _reply(List<dynamic> message) {
|
|
if (_open) _onMessage(message);
|
|
}
|
|
}
|
|
|
|
const _filterKeys = {
|
|
'ids',
|
|
'authors',
|
|
'kinds',
|
|
'since',
|
|
'until',
|
|
'limit',
|
|
'before_id',
|
|
};
|
|
const _windowKeys = {
|
|
'top_level',
|
|
'include_summaries',
|
|
'include_aux',
|
|
'before_id',
|
|
'until',
|
|
'limit',
|
|
};
|
|
|
|
int _newestFirst(NostrEvent a, NostrEvent b) => a.createdAt != b.createdAt
|
|
? b.createdAt.compareTo(a.createdAt)
|
|
: a.id.compareTo(b.id);
|
|
|
|
class _FixedRelayConfig extends RelayConfigNotifier {
|
|
_FixedRelayConfig(this._nsec);
|
|
|
|
final String _nsec;
|
|
|
|
@override
|
|
RelayConfig build() =>
|
|
RelayConfig(baseUrl: 'https://relay.test', nsec: _nsec);
|
|
}
|
|
|
|
class _SignedInAuth extends AuthNotifier {
|
|
@override
|
|
Future<AuthState> build() async =>
|
|
const AuthState(status: AuthStatus.authenticated);
|
|
}
|