mirror of
https://github.com/block/buzz.git
synced 2026-10-27 04:30:52 +00:00
Pinky is opening this PR on Wes’s behalf. ## Summary Reduce two separately measured mobile delays without changing the relay API or removing rich message rendering: - Publish the finite channel-list snapshot without waiting for live subscription setup. - Batch active channel-list subscriptions into sorted, deterministic chunks of at most 128 explicit channel IDs, retaining unchanged chunks. - Install replacement chunks before retiring old coverage. Retain old chunks across thrown replacement failures; filter callbacks to the current relay/identity and still-desired channels; clean up retired/in-flight work across disconnect and disposal. - Scope the custom-emoji Markdown matcher to known shortcodes actually referenced in the rendered content, rather than embedding the whole community palette in every message’s regex. Preserve unknown literals, shared-colon token boundaries, event-tag URL priority, content edits, and code literals. - Honor explicit zero retry hints without inventing a ten-second session-wide gate, while preserving the ordinary live-subscription retry backoff and any already-active gate. ## Matched performance results Medians of three before and three after process-cold launches, alternated on the same authenticated iPhone 17 Pro / iOS 26.5 simulator. Before is mobile source at `e76c81968b65b0755b83efdd59dc3375c59ddf40`; after is this production patch before two documentation-only comment fixes. First channel-list frame: 11.617s → 3.179s · 73% lower latency Live setup duration: 8.475s → 0.185s · 98% lower latency Channel-open first message-list frame: 2.754s → 1.230s · 55% lower latency Message data ready → first frame: 1.977s → 0.286s · 86% lower latency Channel-open reveal complete: 2.845s → 1.394s · 51% lower latency Channel-open data readiness: 0.770s → 0.944s · 23% higher latency The gain is client-side orchestration/rendering, not a claim that the relay became faster. First channel-list frame ranges were 10.835–11.788s before and 2.872–3.395s after; channel-open first-frame ranges were 1.560–2.906s before and 1.149–1.317s after. ### Measurement boundaries - Debug simulator builds, CPU sampling disabled, bounded timestamp probes enabled identically. These are not release/physical-device measurements. - Startup clock starts at Dart `main`; build/install/native pre-main time is excluded. Auth/preferences and OS/disk caches are retained between new processes. - Same account scale: 113 active channels. Latest-message events varied slightly with live activity (1543–1546). - Channel-open uses the same initial 50-row history window, 97 query events, and 67 provider events. The 2306-entry emoji palette is explicitly loaded before navigation on both sides; palette preparation is excluded from the channel-open clock and happens after the startup frame measurement. - Both diagnostic builds temporarily disabled unused avatar segmentation to work around the existing Google ML Kit arm64-simulator slice limitation. The workaround, dependency/native changes, auto-navigation, and all probes are excluded from this PR. ## Validation - Full mobile package suite: `flutter test` — 1890 passed. - `just mobile-check` — 506 files unchanged; analyzer clean. - `just file-size-check` — policy tests and all client ratchets passed. - `git diff --check` — passed. - New lifecycle regressions cover front-sorting insertion across a chunk boundary while replacement readiness is paused, failure retention/departed-channel filtering, retired generation + disconnect cleanup, disposal, chunk limits, unchanged-set reuse, and scope switches. - Emoji unit/widget coverage includes a 2500-unused-emoji palette, unknown tokens, case matching at the component level, shared-colon boundaries, rich text, event URL priority, and content edits. - Fresh-frame source review traced the subscription queue/fences, callback scopes, duplicate-event paths, matcher/wiring, and retry scheduling. - At committed/pushed head `13a83b628c8411c5885e6f76a250ba87accf6067`, all normal pre-push hooks passed: `mobile-checks` (formatter, analyzer, and the full 1890-test mobile suite), `file-size-check`, `branch-skew`, and `push-head-scope`. The commit hook formatted 506 files with no changes. Runtime measurements preceded only the two documentation-comment fixes; no runtime source changed afterward. ## Limits / follow-ups - `RelaySession.subscribe` still settles under its existing EOSE/fallback/retryable-CLOSED contract. “Setup completed” is not an unconditional EOSE or live-delivery guarantee. This PR does not add status-aware replacement ownership. - The channel-message provider still awaits subscribe before fetching history; that separate serialization is not removed here. - Oversized Huddle queries and the separate history batching path above 128 active channels remain follow-ups, as do pre-existing read-state initialization/size warnings. - Palette-only widget refresh and upstream Markdown uppercase-dispatch behavior are not changed. - A clean source build still has the existing Google ML Kit arm64-simulator issue; the profiling workaround is not a proposed product fix. Originating Buzz conversation: buzz://message?channel=793b0522-7995-4375-b1a6-fd94a96fa21d&id=6ba88afdec78ab2cfb6728afcd4a6d10f29e6aa33ff0f62f45d6750381e4d789 --------- Signed-off-by: Wes <wesbillman@users.noreply.github.com> Co-authored-by: Pinky <5f5ab050ec58ae208332edd544ebf705221e24c1b86d82a6ca07038a7a8f6ac9@buzz.block.builderlab.xyz> Co-authored-by: Brain <1a02c72794dcd0f07058a353bc3a81f4028b8c77c92c87fce6d5c8b85970a20b@buzz.block.builderlab.xyz> Co-authored-by: Carl <9d00794d3df50972eb8b615511783cab12a77a8fd5dd5edd58073ec73b54bd8b@buzz.block.builderlab.xyz>
275 lines
9.6 KiB
Dart
275 lines
9.6 KiB
Dart
part of 'channels_provider.dart';
|
|
|
|
// Mirrors MAX_EXPLICIT_CHANNEL_VALUES in the relay REQ handler. A larger live
|
|
// set must use multiple subscriptions or the relay rejects the whole REQ.
|
|
const _maxLiveChannelsPerSubscription = 128;
|
|
|
|
/// Owns live channel subscription reconciliation and teardown.
|
|
///
|
|
/// Keeping this extension in a part preserves access to the notifier's private
|
|
/// lifecycle state while keeping `channels_provider.dart` below the mobile
|
|
/// file-size ratchet.
|
|
extension _ChannelsNotifierLiveSubscriptions on ChannelsNotifier {
|
|
/// Subscribe to live events in deterministic chunks that stay within the
|
|
/// relay's aggregate explicit-`#h` limit. Also starts a 60s WS backstop poll
|
|
/// to reconcile membership changes without downloading the global directory.
|
|
Future<void> _subscribeLive(
|
|
List<Channel> channels,
|
|
_ChannelRefreshFence fence,
|
|
) {
|
|
final channelIds = {
|
|
for (final channel in channels)
|
|
if (channel.isMember && !channel.isArchived) channel.id,
|
|
};
|
|
final relayBaseUrl = _lifecycleRef.read(relayConfigProvider).baseUrl;
|
|
_desiredLiveChannelIds = channelIds;
|
|
// A changed filter may be admitted, but an unchanged terminal rejection
|
|
// cannot recover just because a timer or manual refresh fetched it again.
|
|
// Drop obsolete keys so this quarantine stays bounded by desired chunks.
|
|
final desiredKeys = chunkChannelIdsForLiveSubscriptions(
|
|
channelIds,
|
|
).map(_liveChunkKey).toSet();
|
|
_terminallyClosedLiveChunks.retainAll(desiredKeys);
|
|
final subscriptionVersion = ++_subscriptionVersion;
|
|
|
|
final sync = _liveSubscriptionQueue.then(
|
|
(_) => _syncLiveSubscriptions(
|
|
relayBaseUrl,
|
|
subscriptionVersion,
|
|
channels,
|
|
fence,
|
|
),
|
|
);
|
|
_liveSubscriptionQueue = sync
|
|
.whenComplete(() {
|
|
// Reconcile even if a retired generation exits before its successor
|
|
// can run (for example, a disconnect while subscribe is pending).
|
|
if (_lifecycleRef.mounted) _removeUndesiredLiveChunks();
|
|
})
|
|
.catchError((Object error, StackTrace stack) {
|
|
if (error is! _StaleChannelRefresh) {
|
|
debugPrint(
|
|
'[ChannelsNotifier] live subscription sync failed: $error\n$stack',
|
|
);
|
|
}
|
|
});
|
|
return _liveSubscriptionQueue;
|
|
}
|
|
|
|
Future<void> _syncLiveSubscriptions(
|
|
String relayBaseUrl,
|
|
int subscriptionVersion,
|
|
List<Channel> channels,
|
|
_ChannelRefreshFence fence,
|
|
) async {
|
|
if (!_lifecycleRef.mounted) return;
|
|
fence.ensureCurrent();
|
|
if (_lifecycleRef.read(relaySessionProvider).status !=
|
|
SessionStatus.connected ||
|
|
subscriptionVersion != _subscriptionVersion) {
|
|
return;
|
|
}
|
|
|
|
if (_subscriptionRelayBaseUrl != relayBaseUrl) {
|
|
_clearRetainedLiveChunks();
|
|
_subscriptionRelayBaseUrl = relayBaseUrl;
|
|
}
|
|
if (_lifecycleRef.read(relayConfigProvider).baseUrl != relayBaseUrl) return;
|
|
|
|
final desiredChunks = chunkChannelIdsForLiveSubscriptions(
|
|
_desiredLiveChannelIds,
|
|
);
|
|
// Keep existing coverage while replacements are installed. Cleanup runs
|
|
// when this generation settles, and retains old coverage on failures.
|
|
|
|
final session = _lifecycleRef.read(relaySessionProvider.notifier);
|
|
for (final chunk in desiredChunks) {
|
|
final chunkKey = _liveChunkKey(chunk);
|
|
if (_liveSubscriptionsByChunk.containsKey(chunkKey) ||
|
|
_terminallyClosedLiveChunks.contains(chunkKey)) {
|
|
continue;
|
|
}
|
|
if (_lifecycleRef.read(relaySessionProvider).status !=
|
|
SessionStatus.connected) {
|
|
return;
|
|
}
|
|
final generation = ++_nextLiveChunkGeneration;
|
|
final subscription = _LiveChunkSubscription(generation);
|
|
_liveSubscriptionsByChunk[chunkKey] = subscription;
|
|
try {
|
|
final unsubscribe = await session.subscribe(
|
|
NostrFilter(
|
|
kinds: EventKind.channelEventKinds,
|
|
tags: {'#h': chunk},
|
|
limit: 0,
|
|
),
|
|
(event) {
|
|
// Superseded chunks can overlap during replacement or recovery.
|
|
// Deliver only the current scope's still-desired channels; duplicate
|
|
// events are already idempotent in the unread/timestamp stores.
|
|
if (!_lifecycleRef.mounted ||
|
|
_refreshCoordinator.currentScope() != fence.scope ||
|
|
!_desiredLiveChannelIds.contains(event.channelId)) {
|
|
return;
|
|
}
|
|
_handleLiveEvent(event);
|
|
},
|
|
onClosed: (message) =>
|
|
_handleLiveChunkClosed(chunkKey, generation, message),
|
|
);
|
|
subscription.unsubscribe = unsubscribe;
|
|
if (!_lifecycleRef.mounted || !fence.isCurrent) {
|
|
_liveSubscriptionsByChunk.remove(chunkKey);
|
|
unsubscribe();
|
|
if (!fence.isCurrent) throw const _StaleChannelRefresh();
|
|
return;
|
|
}
|
|
if (subscriptionVersion != _subscriptionVersion ||
|
|
_lifecycleRef.read(relaySessionProvider).status !=
|
|
SessionStatus.connected ||
|
|
_lifecycleRef.read(relayConfigProvider).baseUrl != relayBaseUrl ||
|
|
_subscriptionRelayBaseUrl != relayBaseUrl ||
|
|
!chunk.every(_desiredLiveChannelIds.contains)) {
|
|
_liveSubscriptionsByChunk.remove(chunkKey);
|
|
unsubscribe();
|
|
return;
|
|
}
|
|
if (_liveSubscriptionsByChunk[chunkKey] != subscription) {
|
|
unsubscribe();
|
|
continue;
|
|
}
|
|
} on _StaleChannelRefresh {
|
|
rethrow;
|
|
} catch (error) {
|
|
if (_liveSubscriptionsByChunk[chunkKey] == subscription) {
|
|
_liveSubscriptionsByChunk.remove(chunkKey);
|
|
}
|
|
if (!_lifecycleRef.mounted) return;
|
|
debugPrint(
|
|
'[ChannelsNotifier] live subscription failed for '
|
|
'${chunk.length} channels: $error',
|
|
);
|
|
}
|
|
}
|
|
|
|
if (!_lifecycleRef.mounted ||
|
|
_lifecycleRef.read(relaySessionProvider).status !=
|
|
SessionStatus.connected ||
|
|
subscriptionVersion != _subscriptionVersion) {
|
|
return;
|
|
}
|
|
|
|
fence.ensureCurrent();
|
|
unawaited(_catchUpUnreadEvents(channels, fence, subscriptionVersion));
|
|
|
|
_backstopTimer?.cancel();
|
|
_backstopTimer = Timer.periodic(
|
|
ChannelsNotifier._backstopInterval,
|
|
(_) => _backstopRefresh(),
|
|
);
|
|
}
|
|
|
|
void _handleLiveChunkClosed(String chunkKey, int generation, String message) {
|
|
if (!_lifecycleRef.mounted) return;
|
|
final subscription = _liveSubscriptionsByChunk[chunkKey];
|
|
if (subscription == null || subscription.generation != generation) return;
|
|
_liveSubscriptionsByChunk.remove(chunkKey);
|
|
// RelaySession calls onClosed only for terminal admission failures. Its
|
|
// transient/rate-limit retries have their own backoff. Do not turn a
|
|
// terminal rejection into a fresh membership/history/REQ loop here.
|
|
if (chunkChannelIdsForLiveSubscriptions(
|
|
_desiredLiveChannelIds,
|
|
).any((chunk) => _liveChunkKey(chunk) == chunkKey)) {
|
|
_terminallyClosedLiveChunks.add(chunkKey);
|
|
}
|
|
debugPrint(
|
|
'[ChannelsNotifier] live subscription closed by relay: $message',
|
|
);
|
|
}
|
|
|
|
void _removeUndesiredLiveChunks() {
|
|
final desiredChunks = chunkChannelIdsForLiveSubscriptions(
|
|
_desiredLiveChannelIds,
|
|
);
|
|
final desiredKeys = desiredChunks.map(_liveChunkKey).toSet();
|
|
final retainedKeys = <String>{...desiredKeys};
|
|
final uncoveredIds = <String>{};
|
|
for (final chunk in desiredChunks) {
|
|
final key = _liveChunkKey(chunk);
|
|
if (!_liveSubscriptionsByChunk.containsKey(key)) {
|
|
uncoveredIds.addAll(chunk);
|
|
}
|
|
}
|
|
final obsolete = _liveSubscriptionsByChunk.entries
|
|
.where((entry) => !desiredKeys.contains(entry.key))
|
|
.toList();
|
|
while (uncoveredIds.isNotEmpty) {
|
|
MapEntry<String, _LiveChunkSubscription>? best;
|
|
var bestCoverage = 0;
|
|
for (final entry in obsolete) {
|
|
if (retainedKeys.contains(entry.key)) continue;
|
|
final coverage = entry.key
|
|
.split('\u0000')
|
|
.where(uncoveredIds.contains)
|
|
.length;
|
|
if (coverage > bestCoverage) {
|
|
best = entry;
|
|
bestCoverage = coverage;
|
|
}
|
|
}
|
|
if (best == null) break;
|
|
retainedKeys.add(best.key);
|
|
uncoveredIds.removeAll(best.key.split('\u0000'));
|
|
}
|
|
for (final entry in _liveSubscriptionsByChunk.entries.toList()) {
|
|
if (retainedKeys.contains(entry.key)) continue;
|
|
_liveSubscriptionsByChunk.remove(entry.key);
|
|
entry.value.unsubscribe?.call();
|
|
}
|
|
}
|
|
|
|
void _clearRetainedLiveChunks() {
|
|
for (final subscription in _liveSubscriptionsByChunk.values) {
|
|
subscription.unsubscribe?.call();
|
|
}
|
|
_liveSubscriptionsByChunk.clear();
|
|
}
|
|
|
|
void _clearLiveSubscriptions() {
|
|
_subscriptionVersion++;
|
|
_desiredLiveChannelIds = const {};
|
|
_terminallyClosedLiveChunks.clear();
|
|
_clearRetainedLiveChunks();
|
|
_subscriptionRelayBaseUrl = null;
|
|
_backstopTimer?.cancel();
|
|
_backstopTimer = null;
|
|
}
|
|
}
|
|
|
|
/// Returns deterministic channel-ID chunks within the relay's live REQ cap.
|
|
List<List<String>> chunkChannelIdsForLiveSubscriptions(
|
|
Iterable<String> channelIds,
|
|
) {
|
|
final sortedIds = channelIds.toList()..sort();
|
|
return [
|
|
for (
|
|
var start = 0;
|
|
start < sortedIds.length;
|
|
start += _maxLiveChannelsPerSubscription
|
|
)
|
|
sortedIds.sublist(
|
|
start,
|
|
min(start + _maxLiveChannelsPerSubscription, sortedIds.length),
|
|
),
|
|
];
|
|
}
|
|
|
|
String _liveChunkKey(List<String> channelIds) => channelIds.join('\u0000');
|
|
|
|
class _LiveChunkSubscription {
|
|
_LiveChunkSubscription(this.generation);
|
|
|
|
final int generation;
|
|
void Function()? unsubscribe;
|
|
}
|