fix(pubsub): replace cross-slot presence MGET with non-atomic pipeline

Bulk presence read issued a single `MGET` over one key per pubkey. Presence
keys are `buzz:{community}:presence:{pubkey_hex}` with no hash tag, so they
hash to different cluster slots and the command fails `CROSSSLOT` on any
cluster-mode server, including ElastiCache Serverless.

Replace it with one single-key `GET` per pubkey in a non-atomic pipeline:
same one round trip, same key format, works identically on standalone Redis.
The key format is deliberately left untagged — tagging presence by community
would co-slot an entire community's keys and concentrate load on one slot.

Reply reassembly is positional, so a short reply would silently attribute one
member's status to another (user-visible as other people's online status).
Length is now checked and mismatches error instead of truncating under `zip`.

`.atomic()` must never be added to this pipeline: it wraps the batch in
MULTI/EXEC, which is dispatched as a unit and restores the cross-slot failure.
Documented at the call site, since single-node tests cannot catch it.

Also route the test Redis endpoint through one `test_redis_url()` helper.
`make_test_pool()` hardcoded `127.0.0.1:6379` while pub/sub tests passed their
own literal URL, so pointing the suite at another server split the pool and the
pub/sub client across two hosts — which surfaced as three unrelated timeouts
that looked like cluster incompatibility.

Tests: bulk read asserts the reassembled `pubkey -> status` map (not vector
length) across distinct slots, shuffled order, interleaved missing keys, and
duplicates, plus permutation-equality so misalignment fails without knowing
the expected order. A test-local CRC16-XMODEM (checked against the Redis
Cluster reference vector) asserts presence keys span multiple slots, which
also fails if someone later hash-tags them.

CI now runs the buzz-pubsub suite including `#[ignore]`d Redis tests against
the standalone Redis the backend-integration job already starts; previously
nothing ran them. This keeps standard Redis in the matrix permanently.

Verified at this tree: 39/39 with `--include-ignored` against standalone
Redis 7 and against a single-shard cluster-mode server owning all 16384
slots. Negative control: restoring the `MGET` on the same rig fails with
`CrossSlot: Keys in request don't hash to the same slot`, confirming the rig
detects the bug the fix removes.

Co-authored-by: Dawn (sprout agent) <c6237ef84fa537c78dcee78efd2d4e59f728859c7f194da42ac51ededfa0be05@sprout-oss.stage.blox.sqprod.co>
Signed-off-by: tlongwell-block <109685178+tlongwell-block@users.noreply.github.com>
This commit is contained in:
tlongwell-block
2026-07-31 13:24:03 -04:00
parent 052174a148
commit ba87121a18
4 changed files with 202 additions and 11 deletions

View File

@@ -353,6 +353,7 @@ jobs:
cargo nextest archive \
--cargo-profile ci \
-p buzz-db \
-p buzz-pubsub \
-p buzz-relay \
-p buzz-test-client \
--lib \
@@ -684,6 +685,22 @@ jobs:
done
cat /tmp/buzz-relay.log
exit 1
- name: Redis compatibility tests (standalone)
# buzz-pubsub's Redis-backed tests are #[ignore]d, so nothing ran them
# in CI before. They assert the cluster-safe command shapes required by
# ElastiCache Serverless (single-key GETs for bulk presence, exact
# SUBSCRIBE) *and* that those shapes still behave on standard Redis —
# which is the compatibility half of the guarantee, and the half a
# serverless-only rig cannot check. Points at the standalone Redis this
# job already runs; a cluster-mode service container is deliberately not
# added here.
run: |
cargo nextest run \
--archive-file target/ci/backend-integration-tests.tar.zst \
-E 'package(buzz-pubsub)' \
--run-ignored all
env:
REDIS_URL: redis://localhost:6379
- name: Invite security tests
run: |
cargo nextest run \