jerry/buzz
1
0
Fork 0
mirror of https://github.com/block/buzz.git synced 2026-11-14 13:21:02 +00:00
Code Issues Packages Projects Releases Wiki Activity

perf(mobile): download the identity archive list only when it changed (#9445)

Browse Source 
Every reconnect refetches the community's identity archive list (kind
13535) so the agent pickers stay current. On Block's Builderlab that
list is about 228 KB, and it is downloaded again on every resume even
though it rarely changes.

The provider now remembers the last verified list per community and
relay authority, and asks the relay only for a newer one (`since` the
known list's `created_at` plus one). When nothing newer exists it reuses
the known list, as the same set instance so watchers (mention
candidates, search, the Agents page, channel details, the members sheet)
don't rebuild. A newer list replaces it after the same verification as
before. A list that fails any check, including its signature, yields no
archived identities and drops the remembered one.

### Validation

Provider tests: a refresh with an unchanged list sends `since`, keeps
the known set and leaves the hidden set's listeners untouched; a newer
list replaces it and becomes the new baseline; every invalid list (wrong
signer, missing `-` marker, bad signature) starting from a warm cache
drops the remembered list, so a following empty answer hides nobody.
Each new step fails with its fix reverted. The existing verification,
community-switch and deadline tests pass with the relay fake now
honoring `since`. Full mobile test suite (4803 passed) and `flutter
analyze` pass.

On an iPhone 12 mini signed in to Block's Builderlab (instrumented
profile build), a resume after 60 s in the background downloaded the 228
KB list each time before this change.

### Regressions

**Accepted tradeoff: a same-second list can be missed.** The cursor
assumes each list the relay publishes is stamped strictly later than the
previous one. The relay nearly always does this
(`publish_nipia_archival_list` stamps past the current head and
republishes on drift), but two lists published within one second can
share a `created_at`, and NIP-IA then keeps the one with the lower id.
If the app already holds the other one, it misses the replacement until
a later list arrives (the next archive or unarchive) or the app
restarts. Meanwhile a just-archived or just-unarchived agent can show
wrongly in pickers and search, never in history. We accept this for the
bandwidth and resume-time win; a relay follow-up that stamps each list
under the replacement lock and makes the rule a NIP-IA MUST removes the
case.

The same bound applies if a relay's database is restored to an older
list: the app keeps its newer one until the relay publishes a list newer
than it, or the app restarts.

A failed or timed-out fetch still hides nobody, as before.

---------

Signed-off-by: Tom Brow <tomb@squareup.com>
Co-authored-by: Tom Brow <tomb@squareup.com>
This commit is contained in:
Tom Brow
2026-11-13 14:40:06 -08:00
committed by GitHub
parent 9e40bf5d79
commit 5e52fa9831
2 changed files with 148 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files

59
mobile/lib/shared/identity_archives/identity_archives.dart
View File

@@ -6,6 +6,7 @@ import 'package:buzz/shared/crypto/event_signing.dart';
import 'package:hooks_riverpod/hooks_riverpod.dart';
import 'package:http/http.dart' as http;
import 'package:buzz/shared/http/app_http_client.dart';
import 'package:nostr/nostr.dart' as nostr;
import '../relay/relay.dart';
import '../me/me_sync.dart';
@@ -16,6 +17,14 @@ final relayInformationClientProvider = Provider<http.Client>((ref) {
return client;
});
/// The last verified archive list per community and authority, so a refresh
/// asks the relay only for a newer list instead of downloading it again.
class _ArchiveCache {
final entries = <(String, String), ({int createdAt, Set<String> archived})>{};
}
final _archiveCacheProvider = Provider<_ArchiveCache>((_) => _ArchiveCache());
/// The identities this relay's NIP-IA archive list (kind 13535) names, never
/// the viewer: an archive stays visible to its subject. Empty when the list
/// cannot be read. This is presentation, never ownership or admission.
@@ -61,12 +70,37 @@ final identityArchivesProvider = FutureProvider<Set<String>>((ref) async {
!RegExp(r'^[0-9a-f]{64}$').hasMatch(authority)) {
return {};
}
final cache = ref.read(_archiveCacheProvider).entries;
final key = (sync.community, authority);
final known = cache[key];
// Ask only for a list newer than the known one. This assumes each list is
// stamped strictly later than the previous one, which the relay does
// except when two lists are published in the same second; then the
// second can be missed until a later list arrives or the app restarts.
final events = await sync.session.fetchHistory(
NostrFilter(kinds: const [13535], authors: [authority], limit: 1),
NostrFilter(
kinds: const [13535],
authors: [authority],
since: known == null ? null : known.createdAt + 1,
limit: 1,
),
timeout: const Duration(seconds: 3),
);
sync.check();
return (await _parseArchives(events, authority))..remove(sync.pubkey);
// Hand back the known instance when it does not name the viewer, so an
// unchanged list does not rebuild every watcher.
Set<String> visible(Set<String> archived) => archived.contains(sync.pubkey)
? ({...archived}..remove(sync.pubkey))
: archived;
if (known != null && events.isEmpty) return visible(known.archived);
final archived = await _parseArchives(events, authority);
if (events.length != 1 || archived == null) {
cache.remove(key);
return {};
}
final shared = Set<String>.unmodifiable(archived);
cache[key] = (createdAt: events.single.createdAt, archived: shared);
return visible(shared);
} catch (_) {
return {};
} finally {
@@ -98,11 +132,14 @@ final hiddenArchivedIdentitiesProvider = Provider<Set<String>>((ref) {
return known.$2;
});
Future<Set<String>> _parseArchives(List<NostrEvent> events, String authority) =>
Isolate.run(() => _archived(events, authority));
/// The archived identities, or null when [events] is not one valid list.
Future<Set<String>?> _parseArchives(
List<NostrEvent> events,
String authority,
) => Isolate.run(() => _archived(events, authority));
Set<String> _archived(List<NostrEvent> events, String authority) {
if (events.length != 1) return {};
Set<String>? _archived(List<NostrEvent> events, String authority) {
if (events.length != 1) return null;
final event = events.single;
final markers = event.tags.where((tag) => tag.isNotEmpty && tag.first == '-');
if (event.kind != 13535 ||
@@ -110,11 +147,15 @@ Set<String> _archived(List<NostrEvent> events, String authority) {
event.content.isNotEmpty ||
markers.length != 1 ||
markers.single.length != 1) {
return {};
return null;
}
final encoded = jsonEncode(event.toJson());
if (utf8.encode(encoded).length > 2 * 1024 * 1024) return {};
verifiedNostrEventFromJson(encoded);
if (utf8.encode(encoded).length > 2 * 1024 * 1024) return null;
try {
verifiedNostrEventFromJson(encoded);
} on nostr.EventValidationException {
return null;
}
return {
for (final tag in event.tags)
if (tag.length >= 2 &&

116
mobile/test/shared/identity_archives/identity_archives_test.dart
View File

@@ -18,12 +18,14 @@ void main() {
final authority = nostr.Keys.generate();
final impostor = nostr.Keys.generate();
final target = 'a' * 64;
var createdAt = 1700000000;
NostrEvent snapshot(nostr.Keys signer, {bool marker = true}) =>
NostrEvent.fromJson(
nostr.Event.from(
kind: 13535,
content: '',
secretKey: signer.secret,
createdAt: createdAt++,
tags: [
if (marker) ['-'],
['p', target],
@@ -54,12 +56,22 @@ void main() {
addTearDown(client.close);
expect(await container.read(identityArchivesProvider.future), {target});
expect(relay.author, authority.public);
for (final invalid in [
snapshot(impostor),
snapshot(authority, marker: false),
NostrEvent.fromJson(snapshot(authority).toJson()..['sig'] = '0' * 128),
for (final invalid in <NostrEvent Function()>[
() => snapshot(impostor),
() => snapshot(authority, marker: false),
() => NostrEvent.fromJson(
snapshot(authority).toJson()..['sig'] = '0' * 128,
),
]) {
relay.event = invalid;
// Each case starts from a warm cache, so it must also drop the known
// list: once the relay has nothing newer, nobody stays hidden.
relay.event = snapshot(authority);
container.invalidate(identityArchivesProvider);
expect(await container.read(identityArchivesProvider.future), {target});
relay.event = invalid();
container.invalidate(identityArchivesProvider);
expect(await container.read(identityArchivesProvider.future), isEmpty);
relay.event = null;
container.invalidate(identityArchivesProvider);
expect(await container.read(identityArchivesProvider.future), isEmpty);
}
@@ -77,17 +89,19 @@ void main() {
final authority = nostr.Keys.generate();
final archivedInA = 'a' * 64;
final archivedInB = 'b' * 64;
NostrEvent snapshot(List<String> keys) => NostrEvent.fromJson(
nostr.Event.from(
kind: 13535,
content: '',
secretKey: authority.secret,
tags: [
['-'],
for (final key in keys) ['p', key],
],
).toMap(),
);
NostrEvent snapshot(List<String> keys, {int? createdAt}) =>
NostrEvent.fromJson(
nostr.Event.from(
kind: 13535,
content: '',
secretKey: authority.secret,
createdAt: createdAt,
tags: [
['-'],
for (final key in keys) ['p', key],
],
).toMap(),
);
// Community A's list also names the viewer, who must stay visible.
final relay = _ArchiveRelay(snapshot([archivedInA, viewer.public]));
final communityB = Completer<void>();
@@ -110,7 +124,10 @@ void main() {
);
addTearDown(container.dispose);
addTearDown(client.close);
container.listen(hiddenArchivedIdentitiesProvider, (_, _) {});
var hiddenChanges = 0;
container.listen(hiddenArchivedIdentitiesProvider, (_, _) {
hiddenChanges++;
});
Future<Set<String>> settled() async {
await container.read(identityArchivesProvider.future);
await pumpEventQueue();
@@ -122,6 +139,16 @@ void main() {
container.invalidate(identityArchivesProvider);
expect(container.read(hiddenArchivedIdentitiesProvider), {archivedInA});
expect(await settled(), {archivedInA});
// An unchanged list leaves the hidden set alone for every watcher.
relay.event = snapshot([
archivedInA,
], createdAt: relay.event!.createdAt + 10);
container.invalidate(identityArchivesProvider);
expect(await settled(), {archivedInA});
final changes = hiddenChanges;
container.invalidate(identityArchivesProvider);
expect(await settled(), {archivedInA});
expect(hiddenChanges, changes);
// Switch community and account: A's list no longer applies, so nobody
// is hidden until B's list arrives.
@@ -136,6 +163,55 @@ void main() {
},
);
test('a refresh downloads the list again only when it changed', () async {
final owner = nostr.Keys.generate();
final authority = nostr.Keys.generate();
NostrEvent snapshot(String target, int createdAt) => NostrEvent.fromJson(
nostr.Event.from(
kind: 13535,
content: '',
secretKey: authority.secret,
createdAt: createdAt,
tags: [
['-'],
['p', target],
],
).toMap(),
);
final relay = _ArchiveRelay(snapshot('a' * 64, 1700000000));
final client = MockClient(
(_) async => http.Response(jsonEncode({'self': authority.public}), 200),
);
final container = ProviderContainer(
overrides: [
meSyncProvider.overrideWithValue(
MeSync(
relay,
RelayConfig(baseUrl: 'https://relay.test', nsec: owner.nsec),
),
),
relayInformationClientProvider.overrideWithValue(client),
],
);
addTearDown(container.dispose);
addTearDown(client.close);
expect(await container.read(identityArchivesProvider.future), {'a' * 64});
expect(relay.since, isNull);
// Unchanged: the relay has nothing newer, and the known list stands.
container.invalidate(identityArchivesProvider);
expect(await container.read(identityArchivesProvider.future), {'a' * 64});
expect(relay.since, 1700000001);
// Replaced: the newer list wins and becomes the new baseline.
relay.event = snapshot('b' * 64, 1700000100);
container.invalidate(identityArchivesProvider);
expect(await container.read(identityArchivesProvider.future), {'b' * 64});
container.invalidate(identityArchivesProvider);
expect(await container.read(identityArchivesProvider.future), {'b' * 64});
expect(relay.since, 1700000101);
});
test('slow metadata has a total deadline and cancels its stream', () async {
final owner = nostr.Keys.generate();
final relay = _ArchiveRelay(null);
@@ -226,6 +302,7 @@ class _ArchiveRelay extends RelaySessionNotifier {
_ArchiveRelay(this.event);
NostrEvent? event;
String? author;
int? since;
@override
Future<List<NostrEvent>> fetchHistory(
NostrFilter filter, {
@@ -233,7 +310,10 @@ class _ArchiveRelay extends RelaySessionNotifier {
}) async {
expect(filter.kinds, [13535]);
author = filter.authors!.single;
return [?event];
since = filter.since;
final event = this.event;
if (event == null || event.createdAt < (filter.since ?? 0)) return [];
return [event];
}
}