The rollback guard `delivered_at = $stamp` was sound only at second
granularity because `claim_due_reminder` wrote `Utc::now().timestamp()`.
Two claims in the same wall-clock second produced the same i64 stamp, so
a stale rollback from a slow pod could match (and silently clobber) a
live claim freshly written by a fast pod. That's the very ABA hazard the
guard exists to prevent.
Switch the claim write to `Utc::now().timestamp_micros()`. The column
stays `BIGINT` — microsecond-epoch values fit, and no reader interprets
`delivered_at` as a whole-seconds count (every read is `IS NULL` /
`IS NOT NULL`; verified across schema.sql:138, migrations/0003:17,
event.rs:1049). No migration needed.
Tests:
- Add `rollback_under_same_wall_clock_second_does_not_clobber`: two
back-to-back claims (no sleep) with a rollback between, assert the
stamps are distinct, then prove the stale stamp_a rollback against
the live stamp_b row returns false and the row is unchanged. Pins
the actual race Eva [4] flagged — under seconds-granularity stamp_a
== stamp_b on any modern host, the assertion fails, and the stale
rollback silently clobbers.
- Update `rollback_with_matching_stamp_reverts_then_reclaim_wins`
comment that previously cited 1-second resolution as the reason for
not comparing stamps. No longer true.
- Update `rollback_with_stale_stamp_does_not_clobber_live_claim`
comment: stamp_old+1 is now a 1-microsecond delta, not 1 second.
Adversarial: reverted `claim_due_reminder` to `Utc::now().timestamp()`,
ran the new test → loud fail at `assert_ne!(stamp_a, stamp_b)` with
`stamp_a=1782498896, stamp_b=1782498896`. Restored.
Validation:
- cargo fmt --check (buzz-db, buzz-relay): OK
- cargo clippy --tests --no-deps -- -D warnings (buzz-db, buzz-relay): OK
- cargo test -p buzz-db --lib --include-ignored --test-threads=1: 88 pass
- cargo test -p buzz-relay --lib: 369 pass
- cargo check -p buzz-relay: OK
Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
Pre-build §5c gate for the multi-tenant relay rewrite: the NIP-ER
reminder producer at `crates/buzz-relay/src/main.rs:454-487` inherited
the old relay's publish-then-claim shape, with the comment
"duplicate fan-out on the next tick is harmless (subscribers
dedup by event ID)"
That premise is wrong — subscribers do NOT dedup reminder fan-outs by
event id, so a publish that wins fan-out but loses the claim leaks a
duplicate delivery to every connected client. Reorder to:
claim_due_reminder → if Ok(Some(stamp))
publish_event → if Ok(()) we're done
if Err(_) rollback_reminder_claim(..., stamp)
This mirrors the `claim → side-effect` ordering Max + Sami landed for
the workflow cron producer (`df997d7cc` + `910d82228`) and keeps the
DB claim row as the single cross-pod side-effect boundary.
DB API change (single prod caller, `main.rs` only)
claim_due_reminder returns `Result<Option<i64>>` instead of
`Result<bool>`. The `Some(stamp)` carries the `delivered_at` value
the winner wrote, which the rollback uses as its `AND delivered_at
= $stamp` guard. Without that guard, a stale rollback could clobber
another pod's later claim — a silent duplicate-publish window.
rollback_reminder_claim(id, created_at, stamp) -> Result<bool> is
the inverse: stamp-guarded UPDATE that sets delivered_at = NULL.
Returns true on revert, false when the row's delivered_at no longer
equals stamp (caller leaves it alone; bounded duplicate fan-out is
the worst case, never silent data loss).
Tests (`#[ignore = "requires Postgres"]`, follows F1/F4 conventions)
two_concurrent_claims_yield_exactly_one_winner
Two `claim_due_reminder` calls under `tokio::join!` on the same
reminder. Asserts exactly one Some(stamp), one None, and a third
claim still loses. The §5c "two-pod concurrency" acceptance test
Eva called for. Adversarial: drop the `AND delivered_at IS NULL`
guard from the claim UPDATE and both calls win — verified.
rollback_with_matching_stamp_reverts_then_reclaim_wins
Happy path: claim → rollback with matching stamp returns true →
second claim wins (no longer held). Adversarial: replace
`SET delivered_at = NULL` with `SET delivered_at = delivered_at`
(no-op rollback) and the second claim fails to win — verified.
rollback_with_stale_stamp_does_not_clobber_live_claim
ABA guard: claim with stamp_old → manually advance delivered_at
to stamp_new (simulates a later pod's claim) → rollback with
stamp_old returns false WITHOUT reverting; re-read confirms
delivered_at is still stamp_new. Adversarial: drop the
`AND delivered_at = $3` guard and the stale rollback clobbers
the live claim — verified.
Comment fix
The misleading "subscribers dedup by event ID" comment is replaced
with the actual invariant: claim is the side-effect boundary,
rollback is best-effort, on rollback failure bounded duplicate
fan-out is the worst case.
Validation
cargo fmt --check -p buzz-db -p buzz-relay ✅
cargo clippy -p buzz-db --tests --no-deps ✅
cargo clippy -p buzz-relay --tests --no-deps ✅
cargo test -p buzz-db --lib -- --include-ignored
--test-threads=1 ✅ 87 pass
cargo test -p buzz-relay --lib ✅ 369 pass
cargo check -p buzz-relay ✅
Refs §5c reminder scheduler claim-first reorder gate.
Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
