mirror of
https://github.com/block/buzz.git
synced 2026-11-08 10:21:46 +00:00
perf(loadgen): spread seed writes across channel creators (#9206)
## What `buzz-loadgen seed` signed every channel create, member add, and thread root with the community owner. In private channels without human members, the owner also signed all the history. The relay limits HTTP calls per pubkey (default 300 a minute), so a community with thousands of channels can't be seeded against a relay that keeps its default limits. The seeder doesn't retry a 429, so the seed fails partway, and a torn-down `run_id` can't be reused. Take one community with 20,000 private channels: the owner alone signs more than 40,000 writes, which is over two hours of its quota. Human authors had the same problem on a smaller scale. A human in hundreds of consecutive private channels signed hundreds of history posts within seconds. Now the community owner creates its first 8 channels, and each further block of 8 gets its own creator identity (`Role::Creator`, derived from `run_id`). A creator signs its channels' kind:9007 and kind:9000 events and their thread roots, and `run` looks roots up by the creator. That keeps every creator well under one minute of quota, whatever the community's channel count. History and replies still rotate across humans: a private channel's own human members, or its creator when it has none. Because only humans look roots up, the creator signs nothing but roots in their channels. No amount of history or replies can push a root out of `run`'s newest-100 lookup. Human signers can still bunch up, for example a human in hundreds of consecutive channels or a deep thread. To cover that, a seed write the relay throttles now waits out the `retry in <n>s` window the relay names and tries again, up to 10 times, before it counts as failed. Compatibility: a community with up to 8 channels is seeded as before, apart from the retries. An existing fixture with that shape keeps working with `run`, including its root lookups. ## What got simpler Nothing got simpler. This adds a creator role and a retry loop, which is the smallest change that makes large fixtures seedable against default limits. ## Validation - `cargo fmt --check`, `cargo clippy -p buzz-test-client --all-targets -D warnings`, and `cargo test -p buzz-test-client --lib loadgen`, run on a Linux workstation at the head commit. - `no_channel_creator_signs_more_seed_writes_than_one_minute_of_its_quota` seeds a private community of 400 channels against a fake relay. The community has wide channels, a tier of humans in 280 channels each, and runs at 2x. The test requires every creator to sign fewer than 300 writes. With the owner signing everything, it fails: the owner signs all 7,212 writes. - `throttled_writes_wait_out_the_quota_and_a_lasting_quota_fails_the_seed`: when every write gets two 429s it is retried and the seed passes. When the quota never clears, each write is tried 11 times and the seed fails with the quota error. - `private_channels_add_every_member_before_anything_is_posted` now uses 20 channels. It checks that every create, member add and root is signed by its channel's creator, that the owner keeps channels 0–7, and that every other post is by a human member. - `root_lookups_search_the_channel_creators_posts` checks that `run` searches for roots by the owner in a community's first block and by the block's creator after it, across two communities. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Luke Tornquist <tornquist@squareup.com> Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -168,9 +168,24 @@ that requires membership rejects both `seed` and `run`. With
|
||||
every member: the humans and agents assigned to each channel up to the
|
||||
largest ramp step, and the hot-thread root and authors in its channel. It
|
||||
sends kind:9000 events of up to 100 members, `seed_concurrency` at a time,
|
||||
signed by the community owner, before posting any history. A rerun adds
|
||||
signed by the channel's creator, before posting any history. A rerun adds
|
||||
nobody new, and any member the relay refuses fails the seed. Seeded history
|
||||
in a private channel is posted by its own members.
|
||||
in a private channel is posted by its own human members, or by its creator
|
||||
when it has none.
|
||||
|
||||
The relay limits HTTP calls per pubkey, so `seed` spreads its writes across
|
||||
signers. The community owner creates its first 8 channels and each further
|
||||
block of 8 has its own creator identity, derived from `run_id`. A creator
|
||||
signs its channels' creation, member adds and thread roots (`run` looks roots
|
||||
up by it), so no creator nears the quota however many channels the community
|
||||
has. History and replies rotate across humans. A write the relay still
|
||||
throttles waits out the window the relay names and is retried, up to 10
|
||||
times, before it counts as failed. Each rejected attempt of a seed post counts as throttled,
|
||||
and the post's latency covers only its accepted attempt.
|
||||
|
||||
A private fixture with more than 8 channels per community that was seeded
|
||||
before channels had creators cannot be reseeded, and `run` finds none of its
|
||||
roots past the first 8 channels. Seed it again under a fresh `run_id`.
|
||||
|
||||
## Desktop clients
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
# `desktop.mode = "legacy"` to compare the client before its volume fixes.
|
||||
# cargo run -p buzz-test-client --bin buzz-loadgen -- seed perf/scenarios/local-desktop.toml
|
||||
# cargo run -p buzz-test-client --bin buzz-loadgen -- run perf/scenarios/local-desktop.toml
|
||||
run_id = "local-desktop"
|
||||
run_id = "local-desktop-2"
|
||||
|
||||
[target]
|
||||
ws_url = "ws://localhost:3000"
|
||||
|
||||
Reference in New Issue
Block a user