From df3f28a2c44f0bd258cff046717cde3fa026662e Mon Sep 17 00:00:00 2001 From: Will Pfleger Date: Fri, 2 Oct 2026 18:32:17 -0400 Subject: [PATCH] feat(relay): add NIP-FI shadow mode for HTTP, WebSocket and admin (#8034) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Stack: this PR → #8062 🤖 Follows #8028 (merged). Adds `BUZZ_NIP_FI_MODE=shadow`. The relay loads and validates the full enforce configuration (issuers, communities, lifetimes, command issuers), evaluates NIP-FI evidence wherever enforce would decide, and records what enforce would have done, while every request, upgrade and session behaves exactly as in `off`. The one approved exception is admin disconnect: shadow verifies the command and returns `200`, where Off has no verifier and returns `503`. It lets an operator measure the would-deny rate, including Hosts missing from `BUZZ_NIP_FI_COMMUNITIES`, before switching to `enforce`. - **Mode predicates.** `NipFiMode` gains `is_off`, `restricts`, `evaluates`, `enforces`, `denies_unconditionally` and `observes_only`, built on two exhaustive `match` bases (`restricts`, `evaluates`); `is_off` is a `matches!` and the rest derive from the bases, so a new mode cannot silently fall into one side. Every call site uses them, including the Git membership-lookup failure from #8005, which uses `restricts()` so shadow keeps Off's body. A test scans each crate's `src/` and fails on any `NipFiMode::` in production code outside the predicate definitions and the env parser, which catches `==`, `matches!` on any number of lines, `if let` and or-patterns. - **Startup.** Shadow goes through enforce's path, so every enforce startup refusal applies, including the required connection lifetime. Missing-setting errors name the configured mode through one formatter (`mode_requires`); the enforce text is byte-identical. Shadow builds the assertion verifier and claims command replays under its own Redis prefix, both through the production `AppState::new`. - **HTTP and Blossom.** `admit_nip_fi_http` keeps Off's result in shadow and additionally replays enforce in enforce's order: first the router guard's steps (community, then assertion), then the handler's steps (NIP-98, pairing, deny set) on the same NIP-98 proof, through `evaluate_enforce_steps`, which enforce shares. A Host mapped in config but missing from the communities table, with a valid assertion, records an admit, because enforce's guard passes and its 404 is not a NIP-FI denial. A dev-mode `X-Pubkey` identity is marked unsigned and is never an enforce proof, so shadow records it as a `nip98` would-deny. On bridge and Blossom routes, the strict proof check (payload tag, Blossom `Strict`) runs as a separate pure check that consumes no replay entry and records on its own series, so it never counts as an admission verdict. Shadow records exactly one verdict for every identity, community or credential refusal enforce would make, with Off's status, body and challenge unchanged: bridge, workflows, Blossom, Git transport and settings, GIF search and share, and invite minting record the `community` would-deny when the Host is unmapped. Git's missing or malformed credentials still reject before any database work, and shadow records the enforce verdict for that failed proof. The router guard stays transparent in shadow. - **WebSocket, root and audio.** The upgrade check evaluates the attached assertion once. An upgrade enforce would refuse records that would-deny; otherwise the assertion goes into a shadow-only session object and the upgrade returns as in Off. The assertion never reaches the connection, the identity registries, the admission gate, terminal frames or cancellation. Each session records at most one admission verdict. **Pairing** is recorded on both ingresses right after a valid NIP-42 proof, before ordinary relay policy, where enforce pairs: a key mismatch or claimless assertion is a `pairing` denial. **The deny-set check** runs where enforce runs it, after registration: a hit is a `deny_set` denial, and a clean read keeps the session pending and registered. **The admit** is recorded only when the connection is actually admitted: on root at the AUTH commit, after ordinary policy; on audio when the participant join transaction commits, after relay membership, channel membership, the final ban and membership decision, mesh routing and the join's own checks, and before publication. A NIP-42 failure, which comes before pairing, and any refusal between pairing and the admit (ordinary policy, mesh, or a join refusal inside the join transaction) retires the observer when the refusal is decided, before any refusal frame, rollback or cleanup is awaited, so it leaves no record even if the deadline passes during that cleanup. A record-only deadline task, armed at upgrade with enforce's `compute_session_deadline`, records `expired` when enforce would have closed an admitted session. If the deadline passes while the session is still pending, that is its single admission verdict, a denial with `stage=deadline`, and a later AUTH records nothing. A deny for the key (admin or cross-pod) records `revoked` on an admitted session; if it arrives after the deny-set check but before the admit, including a deny refused for capacity, the session keeps a pending-revocation mark and records the admit followed by exactly one `revoked` end, matching enforce's phase-independent close, and the deadline cannot take that end. The phase, the mark and retirement change together under one session lock. Each session records at most one end. The session holds the socket's cancellation token for recording only: once the socket is cancelled, no transition records anything, even while connection references remain (a cancel fence; the token never feeds a close action). Upgrade would-denies carry the ingress route, `ws` or `audio`. - **Admin disconnect.** Shadow verifies the command and records the deny entry, so later admissions record would-denies, but closes no session; the response stays `{"disconnected": true}`. Shadow publishes cross-pod on its own channel, `buzz:nip-fi:shadow-disconnect`, which only shadow pods subscribe to, so no enforce pod (including older builds that know only `buzz:nip-fi:disconnect`) acts on a shadow command. Shadow pods also keep listening on `buzz:nip-fi:disconnect`, so their deny record includes enforce's real disconnects. Shadow claims command replays under `buzz:nip-fi:shadow-command:{hash}`: replays are still rejected within shadow, and a shadow accept never uses up the enforce claim. Enforce and Off keep the same channel, keys and close behavior. - **NIP-11.** Shadow serves the same document as enforce, including `limitation.federated_identity`, because clients only attach evidence to a relay that advertises it. ## Recording | Series | Labels | One increment per | |---|---|---| | `buzz_nip_fi_shadow_total` | `route` (`http`, `ws`, `audio`), `stage`, `outcome`, `community` | decision enforce would make: an HTTP admission, a refused upgrade, a session's NIP-FI AUTH decision, or its deadline passing before AUTH (`stage=deadline`) | | `buzz_nip_fi_shadow_strict_proof_total` | `route` (`bridge`, `blossom`), `outcome` (`pass`, `rejected`), `community` | strict NIP-98 side check | | `buzz_nip_fi_shadow_session_end_total` | `route` (`ws`, `audio`), `reason` (`expired`, `revoked`), `community` | admitted session enforce would have ended (admitted sessions only) | | `buzz_nip_fi_shadow_disconnect_total` | `route` (`admin`, `cross_pod`), `outcome` | disconnect-path event, in place of the real disconnect, lag, capacity and poison counters | `community` is the configured canonical URI or `unmapped`, never the raw `Host`; an unmapped or empty Host records `stage=community, outcome=unavailable`. A shadow pod never moves an enforce disconnect or failure counter. Strict-proof rates use their own `pass + rejected` total, and session-end rates divide by the `admit` count for the same route. There is no per-verdict `info` log; a would-deny writes a `debug` line with no token, claim or issuer. The pre-existing enforce `debug` line for the proven NIP-98 key is shared and also fires in shadow. Admin commands rejected for a bad header, body or proof, a replay, or a dependency error record no shadow verdict; only capacity rejection is counted. ## Not observed - Handler-side path-validation exits: an invalid Git owner or repo name on default-branch, a media path that is not a valid hash, and a Git pack URL with a query string record nothing from the handler. The router guard still records what enforce would refuse at the guard (for example a missing assertion), because enforce refuses those requests before it validates the path. - A refusal that is not a NIP-FI decision, such as a tenant-binding 404 behind a passing guard, or an ordinary WebSocket refusal after pairing (a ban, membership or join refusal before admission). ## Follow-ups - A bounded `route` label per HTTP surface (bridge, Blossom, Git, …) on `buzz_nip_fi_shadow_total`; today every HTTP verdict carries `route=http`. - One shared helper for the ten copied `observe_unbound` recording hooks, stacked on this PR. The shadow recorder and session suites and the root pairing witness run in `just test-unit` and the `scripts/run-tests.sh` fallback; the `AppState::new` shadow witness, which exercises the command verifier against real Redis, runs in the `external_infra_redis` lane. NIP-FI env tests recover a poisoned env lock so one panic cannot cascade. --------- Signed-off-by: Will Pfleger Signed-off-by: Will Pfleger Signed-off-by: Duncan Signed-off-by: Hayt <211b96e6a2b7f45fd4047988976c7bbbeeda0c15f3ae7b32eec20834b5a55118@buzz.block.builderlab.xyz> Co-authored-by: Duncan Co-authored-by: Hayt <211b96e6a2b7f45fd4047988976c7bbbeeda0c15f3ae7b32eec20834b5a55118@buzz.block.builderlab.xyz> --- Justfile | 4 +- crates/buzz-auth/src/nip_fi/command_replay.rs | 14 +- crates/buzz-auth/src/nip_fi/mod.rs | 5 +- crates/buzz-auth/src/nip_fi/startup/mod.rs | 43 +- crates/buzz-auth/src/nip_fi/startup/tests.rs | 40 ++ crates/buzz-pubsub/src/conn_control.rs | 16 +- crates/buzz-pubsub/src/lib.rs | 18 +- .../buzz-pubsub/src/nip_fi_command_replay.rs | 23 +- crates/buzz-relay/Cargo.toml | 1 + crates/buzz-relay/src/api/bridge.rs | 53 +- crates/buzz-relay/src/api/gifs.rs | 1 + crates/buzz-relay/src/api/git/settings.rs | 5 +- crates/buzz-relay/src/api/git/transport.rs | 29 +- crates/buzz-relay/src/api/invites.rs | 3 +- crates/buzz-relay/src/api/media.rs | 34 +- crates/buzz-relay/src/api/nip_fi.rs | 546 +++++++++++++- crates/buzz-relay/src/api/workflows.rs | 5 +- crates/buzz-relay/src/audio/handler.rs | 436 +++++++++++- crates/buzz-relay/src/config.rs | 17 +- crates/buzz-relay/src/connection.rs | 5 +- crates/buzz-relay/src/handlers/auth.rs | 162 +++++ crates/buzz-relay/src/lib.rs | 2 + crates/buzz-relay/src/main.rs | 19 +- crates/buzz-relay/src/nip11.rs | 30 +- crates/buzz-relay/src/nip_fi_config.rs | 243 +++++-- crates/buzz-relay/src/nip_fi_http.rs | 309 ++++++-- crates/buzz-relay/src/nip_fi_session.rs | 10 + crates/buzz-relay/src/nip_fi_shadow.rs | 407 +++++++++++ .../buzz-relay/src/nip_fi_shadow_session.rs | 637 +++++++++++++++++ crates/buzz-relay/src/nip_fi_test_hooks.rs | 11 + crates/buzz-relay/src/nip_fi_upgrade.rs | 147 +++- crates/buzz-relay/src/router.rs | 669 ++++++++++++++++-- crates/buzz-relay/src/state.rs | 37 +- scripts/run-tests.sh | 8 + 34 files changed, 3679 insertions(+), 310 deletions(-) create mode 100644 crates/buzz-relay/src/nip_fi_shadow.rs create mode 100644 crates/buzz-relay/src/nip_fi_shadow_session.rs diff --git a/Justfile b/Justfile index dc02757a02..c448027b68 100644 --- a/Justfile +++ b/Justfile @@ -477,7 +477,7 @@ test-unit: # because they live in the binary target; the nested # `tests::postgres_tests::` stays in the PostgreSQL lane. cargo nextest run -p buzz-relay --lib --bin buzz-relay \ - -E 'test(/^api::admin::/) + test(/^handlers::channel_authz::/) + test(/^handlers::moderation_authz::/) + test(/^handlers::side_effects::tests::/) + test(/^storage_sweep::tests::/) + test(/^nip_fi_core::tests::/) + test(/^nip_fi_http::tests::/) + test(/^nip_fi_config::tests::/) + test(/^readiness::tests::/) + test(/^router::tests::/) + test(/^api::parse_query_tests::/) + test(/^api::git::transport::off_mode_precedence_tests::/) + test(/^audio::join::tests::/) + test(/^audio::handler::tests::/) + test(/^nip_fi_gate::tests::/) + test(/^nip_fi_session::tests::/) + test(=state::tests::neither_a_confirmed_inactive_community_nor_a_failed_lookup_admits_the_socket) + test(=handlers::req::tests::timed_out_historical_read_deregisters_before_closed) + test(=handlers::req::tests::superseded_timeout_leaves_replacement_intact) + test(=handlers::req::tests::search_claim_retires_live_and_yields_to_replacement) + test(=handlers::req::tests::concurrent_claims_and_stale_teardowns_keep_the_last_owner) + test(=handlers::req::tests::timeout_closed_is_emitted_before_a_replacement_can_claim) + test(=handlers::req::tests::revoke_then_replacement_keeps_replacement_whole) + test(=handlers::req::tests::claims_after_connection_cleanup_are_refused) + test(=handlers::req::tests::dropped_terminal_frame_cancels_connection) + test(=handlers::req::tests::revoke_dropped_terminal_frame_cancels_connection) + (kind(bin) & (test(/^tests::/) + test(/^composition_tests::/) + test(/^env_filter_tests::/)) - test(/^tests::postgres_tests::/))' + -E 'test(/^api::admin::/) + test(/^handlers::channel_authz::/) + test(/^handlers::moderation_authz::/) + test(/^handlers::side_effects::tests::/) + test(/^storage_sweep::tests::/) + test(/^nip_fi_core::tests::/) + test(/^nip_fi_http::tests::/) + test(/^nip_fi_config::tests::/) + test(/^readiness::tests::/) + test(/^router::tests::/) + test(/^api::parse_query_tests::/) + test(/^api::git::transport::off_mode_precedence_tests::/) + test(/^audio::join::tests::/) + test(/^audio::handler::tests::/) + test(/^nip_fi_gate::tests::/) + test(/^nip_fi_session::tests::/) + test(/^nip_fi_shadow::tests::/) + test(/^nip_fi_shadow_session::tests::/) + test(=state::tests::neither_a_confirmed_inactive_community_nor_a_failed_lookup_admits_the_socket) + test(=handlers::req::tests::timed_out_historical_read_deregisters_before_closed) + test(=handlers::req::tests::superseded_timeout_leaves_replacement_intact) + test(=handlers::req::tests::search_claim_retires_live_and_yields_to_replacement) + test(=handlers::req::tests::concurrent_claims_and_stale_teardowns_keep_the_last_owner) + test(=handlers::req::tests::timeout_closed_is_emitted_before_a_replacement_can_claim) + test(=handlers::req::tests::revoke_then_replacement_keeps_replacement_whole) + test(=handlers::req::tests::claims_after_connection_cleanup_are_refused) + test(=handlers::req::tests::dropped_terminal_frame_cancels_connection) + test(=handlers::req::tests::revoke_dropped_terminal_frame_cancels_connection) + (kind(bin) & (test(/^tests::/) + test(/^composition_tests::/) + test(/^env_filter_tests::/)) - test(/^tests::postgres_tests::/))' # Note on audio::join::tests scope: the full suite is infra-free (no DB, # no Redis). The infra-free audio/FI regression witnesses — bootstrap # ordering barrier, CommitConfirmed arm, pending-close invisibility, @@ -519,6 +519,8 @@ test-unit: + test(=handlers::auth::tests::fi_root_authorization_denied_rows_emit_identical_frames) + test(=handlers::auth::tests::fi_invalid_nip42_proof_emits_terminal_evidence_rejected) + test(=handlers::auth::tests::handle_auth_pairing_mismatch_runs_full_root_denial_path) + + test(=handlers::auth::tests::shadow_root_auth_records_pairing_denial) + + test(=handlers::auth::tests::shadow_root_invalid_nip42_retires_without_record) + test(=handlers::auth::tests::nip42_denial_class_separates_internal_failure_from_bad_evidence) + test(=handlers::event::tests::fanout_access::owner_only_kinds_keep_only_the_owner) + test(=handlers::event::tests::pubsub_fanout::pubsub_owner_only_kinds_reach_only_the_owner) diff --git a/crates/buzz-auth/src/nip_fi/command_replay.rs b/crates/buzz-auth/src/nip_fi/command_replay.rs index 20234bad3a..b1d7e5b0c7 100644 --- a/crates/buzz-auth/src/nip_fi/command_replay.rs +++ b/crates/buzz-auth/src/nip_fi/command_replay.rs @@ -47,12 +47,24 @@ pub trait CommandReplayGuard: Send + Sync { /// scope. The `nip-fi:command` segment never matches a NIP-98 key, whose /// final segments are always `:nip98:{event_id_hex}`. pub fn command_replay_key(issuer: &str, jti: &str) -> String { + namespaced_command_replay_key(COMMAND_REPLAY_PREFIX, issuer, jti) +} + +/// Key prefix for enforce-mode command replay claims. +pub const COMMAND_REPLAY_PREFIX: &str = "buzz:nip-fi:command"; + +/// Key prefix for shadow-mode claims, so a shadow accept never uses up the +/// enforce claim for the same `(iss, jti)` in a shared store. +pub const SHADOW_COMMAND_REPLAY_PREFIX: &str = "buzz:nip-fi:shadow-command"; + +/// [`command_replay_key`] under an explicit prefix. +pub fn namespaced_command_replay_key(prefix: &str, issuer: &str, jti: &str) -> String { let mut hasher = Sha256::new(); for part in [issuer, jti] { hasher.update((part.len() as u64).to_be_bytes()); hasher.update(part.as_bytes()); } - format!("buzz:nip-fi:command:{}", hex::encode(hasher.finalize())) + format!("{prefix}:{}", hex::encode(hasher.finalize())) } /// Process-local seen-set for tests. Instances shared via `Arc` model pods diff --git a/crates/buzz-auth/src/nip_fi/mod.rs b/crates/buzz-auth/src/nip_fi/mod.rs index e92386c7e6..5eea285adc 100644 --- a/crates/buzz-auth/src/nip_fi/mod.rs +++ b/crates/buzz-auth/src/nip_fi/mod.rs @@ -30,7 +30,10 @@ pub use command::{ }; #[cfg(any(test, feature = "test-utils"))] pub use command_replay::InMemoryCommandReplayGuard; -pub use command_replay::{command_replay_key, CommandReplayGuard}; +pub use command_replay::{ + command_replay_key, namespaced_command_replay_key, CommandReplayGuard, COMMAND_REPLAY_PREFIX, + SHADOW_COMMAND_REPLAY_PREFIX, +}; pub use community::{CommunityBinding, CommunityBindingError}; pub use config::{ AssertionPolicyId, ClientSubjectPosture, FreshnessClass, IssuerPolicy, IssuerPolicyError, diff --git a/crates/buzz-auth/src/nip_fi/startup/mod.rs b/crates/buzz-auth/src/nip_fi/startup/mod.rs index 410c862ec7..967d6c3719 100644 --- a/crates/buzz-auth/src/nip_fi/startup/mod.rs +++ b/crates/buzz-auth/src/nip_fi/startup/mod.rs @@ -23,6 +23,47 @@ pub enum NipFiMode { /// enforce-mode deployment was misconfigured and must fail closed while /// the operator repairs configuration. [FI-INV-14] DenyProtected, + /// Evaluates evidence as `Enforce` would but only records the verdict; + /// admission is identical to `Off`. + Shadow, +} + +impl NipFiMode { + /// NIP-FI is disabled. + pub const fn is_off(self) -> bool { + matches!(self, Self::Off) + } + + /// Admission may deny on NIP-FI grounds; other modes admit as `Off`. + pub const fn restricts(self) -> bool { + match self { + Self::Enforce | Self::DenyProtected => true, + Self::Off | Self::Shadow => false, + } + } + + /// The full enforce configuration is loaded and evidence is evaluated. + pub const fn evaluates(self) -> bool { + match self { + Self::Enforce | Self::Shadow => true, + Self::Off | Self::DenyProtected => false, + } + } + + /// Evidence is evaluated and its verdict is authoritative. + pub const fn enforces(self) -> bool { + self.restricts() && self.evaluates() + } + + /// Every protected request is denied without evaluating evidence. + pub const fn denies_unconditionally(self) -> bool { + self.restricts() && !self.evaluates() + } + + /// Evidence is evaluated only to record its would-be verdict. + pub const fn observes_only(self) -> bool { + self.evaluates() && !self.restricts() + } } /// Every variant corresponds to a concrete, operator-actionable defect. @@ -68,7 +109,7 @@ pub fn validate_nip_fi_config( registry: &IssuerRegistry, jwks_configs: &[IssuerJwksConfig], ) -> Result<(), NipFiStartupError> { - if let NipFiMode::Off | NipFiMode::DenyProtected = mode { + if !mode.evaluates() { return Ok(()); } diff --git a/crates/buzz-auth/src/nip_fi/startup/tests.rs b/crates/buzz-auth/src/nip_fi/startup/tests.rs index 9b1d59b187..9275c117c8 100644 --- a/crates/buzz-auth/src/nip_fi/startup/tests.rs +++ b/crates/buzz-auth/src/nip_fi/startup/tests.rs @@ -178,3 +178,43 @@ fn enforce_duplicate_jwks_issuer_in_configs_rejects() { "duplicate JWKS configs must not pass" ); } + +#[test] +fn shadow_mode_validates_like_enforce() { + let issuer = "https://id.example"; + let mut registry = IssuerRegistry::new(); + assert_eq!( + validate_nip_fi_config(NipFiMode::Shadow, ®istry, &[]).unwrap_err(), + NipFiStartupError::EmptyRegistry + ); + registry.insert(make_offline_policy(issuer)); + assert_eq!( + validate_nip_fi_config(NipFiMode::Shadow, ®istry, &[]).unwrap_err(), + NipFiStartupError::MissingJwksConfig + ); + assert!( + validate_nip_fi_config(NipFiMode::Shadow, ®istry, &[make_jwks_config(issuer)]).is_ok() + ); +} + +#[test] +fn mode_predicates_separate_evaluation_from_authority() { + // (mode, restricts, evaluates): Shadow is the only mode that evaluates + // evidence without the verdict affecting admission. + for (mode, restricts, evaluates) in [ + (NipFiMode::Off, false, false), + (NipFiMode::Enforce, true, true), + (NipFiMode::DenyProtected, true, false), + (NipFiMode::Shadow, false, true), + ] { + assert_eq!(mode.restricts(), restricts, "{mode:?} restricts"); + assert_eq!(mode.evaluates(), evaluates, "{mode:?} evaluates"); + assert_eq!(mode.enforces(), restricts && evaluates, "{mode:?} enforces"); + assert_eq!( + mode.denies_unconditionally(), + restricts && !evaluates, + "{mode:?}" + ); + assert_eq!(mode.observes_only(), evaluates && !restricts, "{mode:?}"); + } +} diff --git a/crates/buzz-pubsub/src/conn_control.rs b/crates/buzz-pubsub/src/conn_control.rs index 535de9f563..0bf88689ea 100644 --- a/crates/buzz-pubsub/src/conn_control.rs +++ b/crates/buzz-pubsub/src/conn_control.rs @@ -42,6 +42,10 @@ pub fn conn_control_channel(ctx: &TenantContext) -> String { /// connected to at that moment is irrelevant. pub const NIP_FI_DISCONNECT_CHANNEL: &str = "buzz:nip-fi:disconnect"; +/// Channel for disconnects accepted in shadow mode. Only shadow pods +/// subscribe, so a shadow command never reaches an enforce pod. +pub const NIP_FI_SHADOW_DISCONNECT_CHANNEL: &str = "buzz:nip-fi:shadow-disconnect"; + /// A NIP-FI admin-disconnect command broadcast cross-pod after the local deny /// entry is inserted. Every pod merges this entry into its own deny map and /// closes any matching sessions (same `max(until)` rule as the local path). @@ -217,17 +221,18 @@ async fn connect_and_subscribe( // ── NIP-FI disconnect subscriber ────────────────────────────────────────────── -/// Subscribes to [`NIP_FI_DISCONNECT_CHANNEL`] and forwards commands to the +/// Subscribes to `channels` and forwards commands to the /// broadcast. Mirrors [`run_conn_control_subscriber`]: reconnect loop with /// exponential backoff. Never returns. pub async fn run_nip_fi_disconnect_subscriber( redis_url: String, broadcast_tx: broadcast::Sender, + channels: &'static [&'static str], ) { let mut backoff_secs = BACKOFF_INITIAL_SECS; loop { - match connect_and_subscribe_nip_fi(&redis_url, &broadcast_tx).await { + match connect_and_subscribe_nip_fi(&redis_url, &broadcast_tx, channels).await { Ok(()) => { backoff_secs = BACKOFF_INITIAL_SECS; tracing::warn!( @@ -249,15 +254,14 @@ pub async fn run_nip_fi_disconnect_subscriber( async fn connect_and_subscribe_nip_fi( redis_url: &str, broadcast_tx: &broadcast::Sender, + channels: &[&str], ) -> Result<(), redis::RedisError> { let client = redis::Client::open(redis_url)?; let mut conn = client.get_async_pubsub().await?; - conn.subscribe(NIP_FI_DISCONNECT_CHANNEL).await?; + conn.subscribe(channels).await?; - tracing::info!( - "Redis NIP-FI disconnect subscriber connected — listening on {NIP_FI_DISCONNECT_CHANNEL}" - ); + tracing::info!("Redis NIP-FI disconnect subscriber connected — listening on {channels:?}"); let mut stream = conn.on_message(); while let Some(msg) = stream.next().await { diff --git a/crates/buzz-pubsub/src/lib.rs b/crates/buzz-pubsub/src/lib.rs index 2bf86598a3..93417e11c3 100644 --- a/crates/buzz-pubsub/src/lib.rs +++ b/crates/buzz-pubsub/src/lib.rs @@ -190,12 +190,17 @@ impl PubSubManager { /// Starts the NIP-FI disconnect subscriber loop with automatic /// reconnection. Runs forever — spawn this in a background task. /// - /// Every pod subscribes to this global channel; on receipt it merges the - /// deny entry and closes matching local sessions. - pub async fn run_nip_fi_disconnect_subscriber(self: Arc) { + /// Subscribes to `channels`: every NIP-FI pod hears the enforce channel, + /// and a shadow pod also hears its own. On receipt the pod merges the + /// deny entry; an enforce pod closes matching local sessions. + pub async fn run_nip_fi_disconnect_subscriber( + self: Arc, + channels: &'static [&'static str], + ) { conn_control::run_nip_fi_disconnect_subscriber( self.redis_url.clone(), self.nip_fi_disconnect_tx.clone(), + channels, ) .await; } @@ -329,20 +334,21 @@ impl PubSubManager { Ok(subscriber_count) } - /// Publish a NIP-FI disconnect command to all pods on the global channel. + /// Publish a NIP-FI disconnect command to all pods listening on `channel`. /// /// Called after the local deny entry is inserted. Remote pods receive this /// and apply the same `max(until)` merge + all-community session close. /// Fire-and-forget: the HTTP response does not wait on delivery. pub async fn publish_nip_fi_disconnect( &self, + channel: &str, command: &NipFiDisconnect, ) -> Result { - use crate::conn_control::{encode_nip_fi_disconnect, NIP_FI_DISCONNECT_CHANNEL}; + use crate::conn_control::encode_nip_fi_disconnect; let mut conn = self.pool.get().await?; let payload = encode_nip_fi_disconnect(command)?; let subscriber_count: i64 = redis::cmd("PUBLISH") - .arg(NIP_FI_DISCONNECT_CHANNEL) + .arg(channel) .arg(&payload) .query_async(&mut conn) .await?; diff --git a/crates/buzz-pubsub/src/nip_fi_command_replay.rs b/crates/buzz-pubsub/src/nip_fi_command_replay.rs index 49232da9b8..d21bd5c55d 100644 --- a/crates/buzz-pubsub/src/nip_fi_command_replay.rs +++ b/crates/buzz-pubsub/src/nip_fi_command_replay.rs @@ -3,9 +3,11 @@ //! Implements [`CommandReplayGuard`] from `buzz-auth` with the same //! `SET NX EX` shape and TTL clamp as [`crate::RedisNip98ReplayGuard`]. +use buzz_auth::nip_fi::{ + namespaced_command_replay_key, COMMAND_REPLAY_PREFIX, SHADOW_COMMAND_REPLAY_PREFIX, +}; use buzz_auth::{ - command_replay_key, error::AuthError, CommandReplayGuard, DEFAULT_REPLAY_TTL_SECS, - MAX_REPLAY_TTL_SECS, + error::AuthError, CommandReplayGuard, DEFAULT_REPLAY_TTL_SECS, MAX_REPLAY_TTL_SECS, }; /// Redis-backed NIP-FI command replay seen-set. @@ -14,12 +16,25 @@ use buzz_auth::{ /// is a first claim and `nil` is a replay. pub struct RedisCommandReplayGuard { pool: deadpool_redis::Pool, + prefix: &'static str, } impl RedisCommandReplayGuard { /// Create a new replay guard backed by the given Redis connection pool. pub fn new(pool: deadpool_redis::Pool) -> Self { - Self { pool } + Self { + pool, + prefix: COMMAND_REPLAY_PREFIX, + } + } + + /// A guard for shadow pods: claims live under + /// `buzz:nip-fi:shadow-command:{hash}`, disjoint from enforce claims. + pub fn shadow(pool: deadpool_redis::Pool) -> Self { + Self { + pool, + prefix: SHADOW_COMMAND_REPLAY_PREFIX, + } } async fn conn(&self) -> Result { @@ -42,7 +57,7 @@ impl CommandReplayGuard for RedisCommandReplayGuard { let ttl = ttl_secs.clamp(DEFAULT_REPLAY_TTL_SECS, MAX_REPLAY_TTL_SECS); let mut conn = self.conn().await?; let result: Option = redis::cmd("SET") - .arg(command_replay_key(issuer, jti)) + .arg(namespaced_command_replay_key(self.prefix, issuer, jti)) .arg("1") .arg("NX") .arg("EX") diff --git a/crates/buzz-relay/Cargo.toml b/crates/buzz-relay/Cargo.toml index db2e287a60..df5b972b0c 100644 --- a/crates/buzz-relay/Cargo.toml +++ b/crates/buzz-relay/Cargo.toml @@ -101,6 +101,7 @@ buzz-core = { workspace = true, features = ["test-utils"] } buzz-auth = { workspace = true, features = ["dev", "test-utils"] } reqwest = { workspace = true } tokio-tungstenite = { workspace = true } +tokio = { workspace = true, features = ["test-util"] } futures = "0.3" flate2 = "1.1.9" opentelemetry_sdk = { workspace = true, features = ["testing"] } diff --git a/crates/buzz-relay/src/api/bridge.rs b/crates/buzz-relay/src/api/bridge.rs index 4c4e010e5b..cd9f931514 100644 --- a/crates/buzz-relay/src/api/bridge.rs +++ b/crates/buzz-relay/src/api/bridge.rs @@ -15,7 +15,7 @@ use axum::{ use base64::Engine; use serde_json::Value; -use buzz_auth::{LimitType, Nip98ReplayGuard, NipFiMode, DEFAULT_REPLAY_TTL_SECS}; +use buzz_auth::{LimitType, Nip98ReplayGuard, DEFAULT_REPLAY_TTL_SECS}; use buzz_core::TenantContext; use crate::handlers::ingest::{IngestAuth, IngestError}; @@ -69,6 +69,17 @@ pub(crate) struct VerifiedBridgeAuth { pub(crate) signed_created_at: Option, } +impl VerifiedBridgeAuth { + /// The admission proof; the dev-mode `X-Pubkey` zero event ID is unsigned. + pub(crate) fn proof(&self, extra: X) -> Nip98Proof { + if self.event_id_bytes == [0; 32] { + Nip98Proof::unsigned(self.pubkey, extra) + } else { + Nip98Proof::new(self.pubkey, extra) + } + } +} + type BridgeAuthResult = Result)>; /// Verify bridge auth: NIP-98 (production) or X-Pubkey (dev mode). @@ -213,7 +224,7 @@ pub(crate) fn make_nip98_closure_for_admission( require_auth_token, require_payload, ) - .map(|auth| Nip98Proof::new(auth.pubkey, (auth.event_id_bytes, auth.signed_created_at))) + .map(|auth| auth.proof((auth.event_id_bytes, auth.signed_created_at))) .map_err(|e| e.into_response()) } } @@ -901,6 +912,7 @@ pub async fn submit_event( let tenant = crate::tenant::bind_community(&state.db, raw_host) .await .map_err(|_| { + crate::nip_fi_shadow::observe_unbound(&state, &headers); api_error( StatusCode::NOT_FOUND, "relay: no community is configured for this host", @@ -912,15 +924,18 @@ pub async fn submit_event( // In NIP-FI enforce/deny-protected mode, a real NIP-98 event is mandatory — // the X-Pubkey dev-mode fallback must never satisfy the pairing requirement. // [NIP-FI.md:594-607, FI-TRACE-HTTP-INGRESS] - let nip_fi_active = !matches!(state.config.nip_fi.mode, NipFiMode::Off); + let nip_fi_active = state.config.nip_fi.mode.restricts(); // POST /events carries an authorization-relevant body (the event determines // resource, effect, and state change), so a payload tag is required in // NIP-FI enforce mode. [NIP-FI.md:619-637] - let nip_fi_enforce = matches!(state.config.nip_fi.mode, NipFiMode::Enforce); + let nip_fi_enforce = state.config.nip_fi.mode.enforces(); // NIP-FI admission: NIP-98 extraction runs inside the closure, followed by // assertion verify → pair → deny-map in fixed order. The proven pubkey is // only available through the returned NipFiAdmission. [FI-TRACE-AUTHORITY-UNIFORM] + crate::nip_fi_shadow::observe_strict_proof(&state, &headers, "bridge", || { + verify_bridge_auth_with_options(&headers, "POST", &url, Some(&body), true, true).map(drop) + }); let admission = admit_nip_fi_http_on_state(&state, &headers, || { verify_bridge_auth_with_options( &headers, @@ -930,7 +945,7 @@ pub async fn submit_event( state.config.require_auth_token || nip_fi_active, nip_fi_enforce, ) - .map(|auth| Nip98Proof::new(auth.pubkey, (auth.event_id_bytes, auth.signed_created_at))) + .map(|auth| auth.proof((auth.event_id_bytes, auth.signed_created_at))) .map_err(|e| e.into_response()) })?; let pubkey = *admission.proven_pubkey(); @@ -1228,6 +1243,7 @@ pub async fn query_events( let tenant = crate::tenant::bind_community(&state.db, raw_host) .await .map_err(|_| { + crate::nip_fi_shadow::observe_unbound(&state, &headers); api_error( StatusCode::NOT_FOUND, "relay: no community is configured for this host", @@ -1238,13 +1254,16 @@ pub async fn query_events( let url = nip98_expected_url(&state.config.relay_url, &tenant, "/query"); // In NIP-FI enforce/deny-protected mode, a real NIP-98 event is mandatory. // [NIP-FI.md:594-607, FI-TRACE-HTTP-INGRESS] - let nip_fi_active = !matches!(state.config.nip_fi.mode, NipFiMode::Off); + let nip_fi_active = state.config.nip_fi.mode.restricts(); // POST /query carries an authorization-relevant body (filter selects the // resources returned), so a payload tag is required in enforce mode. // [NIP-FI.md:619-637] - let nip_fi_enforce = matches!(state.config.nip_fi.mode, NipFiMode::Enforce); + let nip_fi_enforce = state.config.nip_fi.mode.enforces(); // NIP-FI admission. [FI-TRACE-AUTHORITY-UNIFORM] + crate::nip_fi_shadow::observe_strict_proof(&state, &headers, "bridge", || { + verify_bridge_auth_with_options(&headers, "POST", &url, Some(&body), true, true).map(drop) + }); let admission = admit_nip_fi_http_on_state(&state, &headers, || { verify_bridge_auth_with_options( &headers, @@ -1254,7 +1273,7 @@ pub async fn query_events( state.config.require_auth_token || nip_fi_active, nip_fi_enforce, ) - .map(|auth| Nip98Proof::new(auth.pubkey, (auth.event_id_bytes, auth.signed_created_at))) + .map(|auth| auth.proof((auth.event_id_bytes, auth.signed_created_at))) .map_err(|e| e.into_response()) })?; let pubkey = *admission.proven_pubkey(); @@ -1859,6 +1878,7 @@ pub async fn count_events( let tenant = crate::tenant::bind_community(&state.db, raw_host) .await .map_err(|_| { + crate::nip_fi_shadow::observe_unbound(&state, &headers); api_error( StatusCode::NOT_FOUND, "relay: no community is configured for this host", @@ -1869,13 +1889,16 @@ pub async fn count_events( let url = nip98_expected_url(&state.config.relay_url, &tenant, "/count"); // In NIP-FI enforce/deny-protected mode, a real NIP-98 event is mandatory. // [NIP-FI.md:594-607, FI-TRACE-HTTP-INGRESS] - let nip_fi_active = !matches!(state.config.nip_fi.mode, NipFiMode::Off); + let nip_fi_active = state.config.nip_fi.mode.restricts(); // POST /count carries an authorization-relevant body (filter selects what // is counted), so a payload tag is required in enforce mode. // [NIP-FI.md:619-637] - let nip_fi_enforce = matches!(state.config.nip_fi.mode, NipFiMode::Enforce); + let nip_fi_enforce = state.config.nip_fi.mode.enforces(); // NIP-FI admission. [FI-TRACE-AUTHORITY-UNIFORM] + crate::nip_fi_shadow::observe_strict_proof(&state, &headers, "bridge", || { + verify_bridge_auth_with_options(&headers, "POST", &url, Some(&body), true, true).map(drop) + }); let admission = admit_nip_fi_http_on_state(&state, &headers, || { verify_bridge_auth_with_options( &headers, @@ -1885,7 +1908,7 @@ pub async fn count_events( state.config.require_auth_token || nip_fi_active, nip_fi_enforce, ) - .map(|auth| Nip98Proof::new(auth.pubkey, (auth.event_id_bytes, auth.signed_created_at))) + .map(|auth| auth.proof((auth.event_id_bytes, auth.signed_created_at))) .map_err(|e| e.into_response()) })?; let pubkey = *admission.proven_pubkey(); @@ -2674,6 +2697,7 @@ async fn authorize_moderation_read( let tenant = crate::tenant::bind_community(&state.db, raw_host) .await .map_err(|_| { + crate::nip_fi_shadow::observe_unbound(state, headers); api_error( StatusCode::NOT_FOUND, "relay: no community is configured for this host", @@ -2689,9 +2713,12 @@ async fn authorize_moderation_read( // In NIP-FI enforce/deny-protected mode a real NIP-98 event is mandatory — // the X-Pubkey dev-mode fallback must never satisfy the pairing requirement. // [NIP-FI.md:594-607, FI-TRACE-HTTP-INGRESS] - let nip_fi_active = !matches!(state.config.nip_fi.mode, NipFiMode::Off); + let nip_fi_active = state.config.nip_fi.mode.restricts(); // NIP-FI admission. [FI-TRACE-AUTHORITY-UNIFORM] + crate::nip_fi_shadow::observe_strict_proof(state, headers, "bridge", || { + verify_bridge_auth(headers, "GET", &url, None, true).map(drop) + }); let admission = admit_nip_fi_http_on_state(state, headers, || { verify_bridge_auth( headers, @@ -2700,7 +2727,7 @@ async fn authorize_moderation_read( None, state.config.require_auth_token || nip_fi_active, ) - .map(|auth| Nip98Proof::new(auth.pubkey, auth.event_id_bytes)) + .map(|auth| auth.proof(auth.event_id_bytes)) .map_err(|e| e.into_response()) })?; let pubkey = *admission.proven_pubkey(); diff --git a/crates/buzz-relay/src/api/gifs.rs b/crates/buzz-relay/src/api/gifs.rs index 1521f0d993..be3adfb7a9 100644 --- a/crates/buzz-relay/src/api/gifs.rs +++ b/crates/buzz-relay/src/api/gifs.rs @@ -132,6 +132,7 @@ async fn authenticate( let tenant = crate::tenant::bind_community(&state.db, raw_host) .await .map_err(|_| { + crate::nip_fi_shadow::observe_unbound(state, headers); api_error( StatusCode::NOT_FOUND, "relay: no community is configured for this host", diff --git a/crates/buzz-relay/src/api/git/settings.rs b/crates/buzz-relay/src/api/git/settings.rs index ec2250ea08..d79409448b 100644 --- a/crates/buzz-relay/src/api/git/settings.rs +++ b/crates/buzz-relay/src/api/git/settings.rs @@ -173,7 +173,10 @@ async fn authenticate( .unwrap_or(""); let tenant = crate::tenant::bind_community(&state.db, host) .await - .map_err(|_| error(StatusCode::NOT_FOUND, "repository not found"))?; + .map_err(|_| { + crate::nip_fi_shadow::observe_unbound(state, headers); + error(StatusCode::NOT_FOUND, "repository not found") + })?; let url = bridge::nip98_expected_url(&state.config.relay_url, &tenant, path); let method = if body.is_some() { "POST" } else { "GET" }; let require_payload = body.is_some(); diff --git a/crates/buzz-relay/src/api/git/transport.rs b/crates/buzz-relay/src/api/git/transport.rs index 04c1f79a42..b1d1b77e66 100644 --- a/crates/buzz-relay/src/api/git/transport.rs +++ b/crates/buzz-relay/src/api/git/transport.rs @@ -102,8 +102,13 @@ impl axum::extract::FromRequestParts> for GitAuth { // must be verified against the tenant-bound host, not a process-global // domain. let mode = state.config.nip_fi.mode; - if matches!(mode, buzz_auth::NipFiMode::Off) { - parse_git_auth_header(&parts.headers, method)?; + if !mode.restricts() { + if let Err(rejection) = parse_git_auth_header(&parts.headers, method) { + // Off returns `rejection` as is; shadow also records the + // enforce verdict for the failed proof, still with no DB work. + crate::nip_fi_http::observe_failed_proof(state, &parts.headers); + return Err(rejection); + } } // Row zero for Git HTTP: bind the request Host to a server-resolved @@ -118,7 +123,10 @@ impl axum::extract::FromRequestParts> for GitAuth { .unwrap_or(""); let tenant = crate::tenant::bind_community(&state.db, raw_host) .await - .map_err(|_| (StatusCode::NOT_FOUND, "repository not found").into_response())?; + .map_err(|_| { + crate::nip_fi_shadow::observe_unbound(state, &parts.headers); + (StatusCode::NOT_FOUND, "repository not found").into_response() + })?; let expected_url = git_expected_url( &state.config.relay_url, &tenant, @@ -207,12 +215,17 @@ impl axum::extract::FromRequestParts> for GitAuth { let event_auth_tag = crate::handlers::auth::extract_auth_tag_json(&event); let header_auth_tag = crate::api::relay_members::extract_auth_tag_header(&parts.headers); let auth_tag = event_auth_tag.as_deref().or(header_auth_tag); - // A failed policy lookup is 503 (the canonical NIP-FI body outside - // Off mode); only a real refusal is 403. - let unavailable = |legacy: Response| match mode { - buzz_auth::NipFiMode::Off => legacy, - _ => crate::nip_fi_core::http_denial(buzz_auth::DenialClass::AuthorizationUnavailable), + // A failed policy lookup is 503 (the canonical NIP-FI body when the + // mode restricts); only a real refusal is 403. + let unavailable = |legacy: Response| { + if mode.restricts() { + crate::nip_fi_core::http_denial(buzz_auth::DenialClass::AuthorizationUnavailable) + } else { + legacy + } }; + #[cfg(test)] + crate::nip_fi_test_hooks::before_git_membership(tenant.community()).await; match crate::api::relay_members::check_relay_membership( state, tenant.community(), diff --git a/crates/buzz-relay/src/api/invites.rs b/crates/buzz-relay/src/api/invites.rs index 172518fb1b..f1f85f52ef 100644 --- a/crates/buzz-relay/src/api/invites.rs +++ b/crates/buzz-relay/src/api/invites.rs @@ -300,11 +300,12 @@ async fn mint_invite_checked( let tenant = match crate::tenant::bind_community(&state.db, raw_host).await { Ok(t) => t, Err(_) => { + crate::nip_fi_shadow::observe_unbound(&state, &headers); return api_error( StatusCode::NOT_FOUND, "relay: no community is configured for this host", ) - .into_response() + .into_response(); } }; diff --git a/crates/buzz-relay/src/api/media.rs b/crates/buzz-relay/src/api/media.rs index e9f3c3e1bc..fdd16143bf 100644 --- a/crates/buzz-relay/src/api/media.rs +++ b/crates/buzz-relay/src/api/media.rs @@ -49,17 +49,9 @@ impl FromRequestParts> for UploadContext { parts: &mut Parts, state: &Arc, ) -> Result { - let headers = &parts.headers; - // Row zero: bind tenant from the request host. Fail-closed: // unmapped host → 404. - let raw_host = headers - .get(header::HOST) - .and_then(|v| v.to_str().ok()) - .unwrap_or(""); - let tenant = crate::tenant::bind_community(&state.db, raw_host) - .await - .map_err(|_| MediaError::NotFound)?; + let tenant = bind_media_read_tenant(state, &parts.headers).await?; let route_mode = upload_route_mode(parts.uri.path())?; @@ -341,6 +333,12 @@ pub(crate) async fn upload_blob( // as the read path. [FI-TRACE-AUTHORITY-UNIFORM] let strictness = blossom_strictness_from_state(&state); let tenant_host = ctx.tenant.host().to_owned(); + crate::nip_fi_shadow::observe_strict_proof(&state, &headers, "blossom", || { + let event = extract_blossom_auth(&headers).map_err(drop)?; + let strict = BlossomStrictness::Strict; + buzz_media::auth::verify_blossom_auth_event(&event, Some(&tenant_host), strict) + .map_err(drop) + }); let headers_clone = headers.clone(); let admission = match admit_nip_fi_http_on_state(&state, &headers, move || { let auth_event = extract_blossom_auth(&headers_clone).map_err(|e| e.into_response())?; @@ -657,7 +655,10 @@ async fn bind_media_read_tenant( .unwrap_or(""); crate::tenant::bind_community(&state.db, raw_host) .await - .map_err(|_| MediaError::NotFound) + .map_err(|_| { + crate::nip_fi_shadow::observe_unbound(state, headers); + MediaError::NotFound + }) } /// Extract and signature-verify the Blossom auth event for a GET/HEAD read. @@ -816,6 +817,15 @@ pub(crate) async fn get_blob( // In Enforce mode: extraction failure → NIP-FI denial bytes (MissingEvidence/ // EvidenceRejected). In Off mode: MediaError propagates unchanged [FI-INV-15]. use crate::nip_fi_http::admit_nip_fi_http_on_state; + crate::nip_fi_shadow::observe_strict_proof(&state, &req_headers, "blossom", || { + extract_blossom_read_proof( + &req_headers, + &sha256, + &tenant_host, + BlossomStrictness::Strict, + ) + .map(drop) + }); let admission = match admit_nip_fi_http_on_state(&state, &req_headers, move || { extract_blossom_read_proof(&headers_clone, &sha256, &tenant_host, strictness) .map_err(|e| e.into_response()) @@ -1106,6 +1116,10 @@ pub(crate) async fn head_blob( let tenant_host = tenant.host().to_owned(); let headers_clone = headers.clone(); use crate::nip_fi_http::admit_nip_fi_http_on_state; + crate::nip_fi_shadow::observe_strict_proof(&state, &headers, "blossom", || { + extract_blossom_read_proof(&headers, &sha256, &tenant_host, BlossomStrictness::Strict) + .map(drop) + }); let admission = match admit_nip_fi_http_on_state(&state, &headers, move || { extract_blossom_read_proof(&headers_clone, &sha256, &tenant_host, strictness) .map_err(|e| e.into_response()) diff --git a/crates/buzz-relay/src/api/nip_fi.rs b/crates/buzz-relay/src/api/nip_fi.rs index 575508ac32..b3e4267cf6 100644 --- a/crates/buzz-relay/src/api/nip_fi.rs +++ b/crates/buzz-relay/src/api/nip_fi.rs @@ -19,9 +19,10 @@ //! //! ## Environment variables //! -//! The command API is enabled when `BUZZ_NIP_FI_MODE=enforce`. S4 fields are -//! read from the same `BUZZ_NIP_FI_ISSUERS` JSON array as the assertion -//! policy; in enforce mode every issuer entry must carry them: +//! The command API is enabled when `BUZZ_NIP_FI_MODE` is `enforce` or +//! `shadow` (shadow keeps the deny entry but closes no session). S4 fields +//! are read from the same `BUZZ_NIP_FI_ISSUERS` JSON array as the assertion +//! policy; in both modes every issuer entry must carry them: //! //! ```json //! { @@ -136,23 +137,34 @@ pub async fn disconnect( let pubkey_bytes = cmd.target_pubkey.to_bytes(); // Issuer-scoped: the deny entry is keyed by (caller_iss, k), so only // sessions admitted under caller_iss are closed. [FI-TRACE-DENY-SET] - let closed = state - .conn_manager - .disconnect_nip_fi(&cmd.caller_iss, &pubkey_bytes) - + state - .community_connections - .disconnect_nip_fi(&cmd.caller_iss, &pubkey_bytes); + // Shadow keeps the deny entry but closes nothing. + let closed = if !state.config.nip_fi.mode.observes_only() { + state + .conn_manager + .disconnect_nip_fi(&cmd.caller_iss, &pubkey_bytes) + + state + .community_connections + .disconnect_nip_fi(&cmd.caller_iss, &pubkey_bytes) + } else { + state + .nip_fi_shadow_sessions + .would_close(&cmd.caller_iss, &pubkey_bytes); + 0 + }; if closed > 0 { // [FI-TRACE-PRIVACY-NONPUBLIC]: raw `iss` MUST NOT appear in // logs, metrics, or traces. Log only a count. debug!(closed, "nip-fi disconnect: closed sessions"); } - metrics::counter!("buzz_nip_fi_disconnect_total").increment(1); - metrics::counter!( - "buzz_nip_fi_sessions_closed_total", - "reason" => "admin_disconnect" - ) - .increment(closed as u64); + let mode = state.config.nip_fi.mode; + count_disconnect_event(mode, "buzz_nip_fi_disconnect_total", "admin", "accepted", 1); + if !mode.observes_only() { + metrics::counter!( + "buzz_nip_fi_sessions_closed_total", + "reason" => "admin_disconnect" + ) + .increment(closed as u64); + } // Cross-pod propagation: publish to global NIP-FI Redis channel // so remote pods can merge the deny entry and close their sessions. @@ -160,12 +172,18 @@ pub async fn disconnect( { let pubsub = Arc::clone(&state.pubsub); let msg = nip_fi_disconnect_message(&cmd); + let channel = disconnect_publish_channel(state.config.nip_fi.mode); state.nip_fi_publish_tasks.spawn(async move { - if let Err(e) = pubsub.publish_nip_fi_disconnect(&msg).await { + if let Err(e) = pubsub.publish_nip_fi_disconnect(channel, &msg).await { // [FI-TRACE-PRIVACY-NONPUBLIC]: no iss or pubkey in logs tracing::warn!("nip-fi: cross-pod propagation publish failed: {e}"); - metrics::counter!("buzz_nip_fi_disconnect_propagation_failures_total") - .increment(1); + count_disconnect_event( + mode, + "buzz_nip_fi_disconnect_propagation_failures_total", + "admin", + "propagation_failure", + 1, + ); } }); } @@ -175,7 +193,13 @@ pub async fn disconnect( Err(err) => { if err == CommandError::DenySetFull { warn!("nip-fi disconnect: deny set full — command rejected, no sessions closed"); - metrics::counter!("buzz_nip_fi_disconnect_capacity_rejections_total").increment(1); + count_disconnect_event( + state.config.nip_fi.mode, + "buzz_nip_fi_disconnect_capacity_rejections_total", + "admin", + "capacity", + 1, + ); } command_denial(err) } @@ -186,16 +210,16 @@ pub async fn disconnect( /// Per-issuer command configuration parsed from the `BUZZ_NIP_FI_ISSUERS` JSON. /// -/// In enforce mode `maximum_command_age_seconds` and `authorized_principals` -/// are required on every issuer and startup validation rejects any entry -/// without them. The fields remain `Option` for the off and `deny_protected` +/// In enforce and shadow modes `maximum_command_age_seconds` and +/// `authorized_principals` are required on every issuer and startup +/// validation rejects any entry without them. The fields remain `Option` for the off and `deny_protected` /// modes, which do not require them. #[derive(Debug, Default, Clone, serde::Deserialize)] pub struct CommandIssuerEnvConfig { - /// Maximum command JWT age in seconds, in `[1, 60]`. Required in enforce mode. + /// Maximum command JWT age in seconds, in `[1, 60]`. Required in enforce and shadow modes. pub maximum_command_age_seconds: Option, /// Non-empty list of authorized `sub` values, matched exactly - /// (case-sensitive). Required in enforce mode. + /// (case-sensitive). Required in enforce and shadow modes. pub authorized_principals: Option>, /// Hard ceiling on live deny entries for this issuer; must be positive. /// Defaults to [`DEFAULT_DENY_SET_CAPACITY`] when absent. @@ -210,6 +234,58 @@ pub struct NipFiCommandComponents { pub command_verifier: Arc>>>, } +/// Channel this pod publishes accepted disconnects on. Shadow uses its own +/// channel, so no enforce pod (of any build) ever acts on a shadow command. +pub fn disconnect_publish_channel(mode: NipFiMode) -> &'static str { + if mode.observes_only() { + buzz_pubsub::conn_control::NIP_FI_SHADOW_DISCONNECT_CHANNEL + } else { + buzz_pubsub::conn_control::NIP_FI_DISCONNECT_CHANNEL + } +} + +/// Channels this pod receives disconnects on. Shadow also hears enforce's +/// real disconnects so its deny record matches. +pub fn disconnect_subscribe_channels(mode: NipFiMode) -> &'static [&'static str] { + use buzz_pubsub::conn_control::{NIP_FI_DISCONNECT_CHANNEL, NIP_FI_SHADOW_DISCONNECT_CHANNEL}; + if mode.observes_only() { + &[NIP_FI_DISCONNECT_CHANNEL, NIP_FI_SHADOW_DISCONNECT_CHANNEL] + } else { + &[NIP_FI_DISCONNECT_CHANNEL] + } +} + +/// Count a disconnect-path event on its `real` counter, or in shadow on +/// `buzz_nip_fi_shadow_disconnect_total{route, outcome}` instead, so a +/// shadow pod never moves an enforce disconnect metric. +pub fn count_disconnect_event( + mode: NipFiMode, + real: &'static str, + route: &'static str, + outcome: &'static str, + n: u64, +) { + if mode.observes_only() { + let labels = [("route", route), ("outcome", outcome)]; + metrics::counter!("buzz_nip_fi_shadow_disconnect_total", &labels).increment(n); + } else { + metrics::counter!(real).increment(n); + } +} + +/// Command replay guard for this pod. Shadow claims under its own key +/// prefix, so a shadow accept never uses up the enforce claim. +pub fn command_replay_guard( + pool: deadpool_redis::Pool, + mode: NipFiMode, +) -> Arc { + if mode.observes_only() { + Arc::new(buzz_pubsub::RedisCommandReplayGuard::shadow(pool)) + } else { + Arc::new(buzz_pubsub::RedisCommandReplayGuard::new(pool)) + } +} + /// Outcome of applying a cross-pod NIP-FI disconnect message. /// /// Returned by [`apply_nip_fi_disconnect`]; used by the `main.rs` receive loop @@ -317,8 +393,12 @@ pub fn apply_nip_fi_disconnect( use buzz_auth::CrossPodMergeResult; let merge_result = deny_map.merge_cross_pod_deny(&message.issuer, &pubkey, until, now); - // Close sessions for all merge outcomes except UnknownIssuer. + // Close sessions for all merge outcomes except UnknownIssuer, never in shadow. let close_sessions = |reason: &str| { + if state.config.nip_fi.mode.observes_only() { + let sessions = &state.nip_fi_shadow_sessions; + return sessions.would_close(&message.issuer, &message.pubkey_bytes); + } let closed = state .conn_manager .disconnect_nip_fi(&message.issuer, &message.pubkey_bytes) @@ -330,6 +410,13 @@ pub fn apply_nip_fi_disconnect( } }; + let mode = state.config.nip_fi.mode; + // What a capacity or poison failsafe does to targeted sessions here. + let action = if mode.observes_only() { + "would-close recorded (shadow)" + } else { + "targeted sessions closed" + }; match &merge_result { CrossPodMergeResult::Merged => { close_sessions("merged"); @@ -339,17 +426,21 @@ pub fn apply_nip_fi_disconnect( } CrossPodMergeResult::CapacityExceeded => { tracing::warn!( - "nip-fi cross-pod: deny set full for issuer — closing targeted sessions without map entry (capacity miss; issuer re-push is the recovery path)" + action, + "nip-fi cross-pod: deny set full for issuer — no map entry (capacity miss; issuer re-push is the recovery path)" ); close_sessions("capacity-exceeded"); - metrics::counter!("buzz_nip_fi_cross_pod_capacity_exceeded_total").increment(1); + let real = "buzz_nip_fi_cross_pod_capacity_exceeded_total"; + count_disconnect_event(mode, real, "cross_pod", "capacity", 1); } CrossPodMergeResult::ShardPoisoned => { tracing::error!( - "nip-fi cross-pod: issuer shard is poisoned — sessions closed (fail-closed)" + action, + "nip-fi cross-pod: issuer shard is poisoned (fail-closed)" ); close_sessions("poisoned shard failsafe"); - metrics::counter!("buzz_nip_fi_cross_pod_shard_poison_total").increment(1); + let real = "buzz_nip_fi_cross_pod_shard_poison_total"; + count_disconnect_event(mode, real, "cross_pod", "poison", 1); } } @@ -370,7 +461,7 @@ pub fn build_nip_fi_command_components( key_source: Arc>, issuer_command_configs: &[(String, CommandIssuerEnvConfig)], ) -> Result>, String> { - if matches!(mode, NipFiMode::Off) { + if mode.is_off() { return Ok(None); } @@ -385,7 +476,7 @@ pub fn build_nip_fi_command_components( None => { // In enforce mode every issuer must be command-capable; // from_env() already guarantees this, but be defensive here too. - if matches!(mode, NipFiMode::Enforce) { + if mode.evaluates() { return Err(format!( "nip-fi: enforce issuer [index {idx}] has no maximum_command_age_seconds — \ assertion-only issuers are not supported in enforce mode" @@ -425,7 +516,7 @@ pub fn build_nip_fi_command_components( } if command_policies.is_empty() { - if matches!(mode, NipFiMode::Enforce) { + if mode.evaluates() { // Enforce with no command-capable issuers is a misconfiguration: // from_env() guarantees every enforce issuer has command config, so // an empty set here means something was skipped or the configs are wrong. @@ -482,7 +573,7 @@ pub fn install_nip_fi_command_components( command_configs: &[(String, CommandIssuerEnvConfig)], ) -> Result { // Pre-flight: enforce mode with no command configs is always an error. - if matches!(mode, NipFiMode::Enforce) && command_configs.is_empty() { + if mode.evaluates() && command_configs.is_empty() { return Err( "NIP-FI install: enforce mode requires at least one command-capable issuer".to_owned(), ); @@ -1926,6 +2017,32 @@ mod route_integration_tests { ); } + /// Shadow evaluates commands like enforce, so it shares enforce's + /// refusal to start without a command-capable issuer. + #[test] + fn installer_rejects_shadow_without_command_issuers() { + let key_source = Arc::new( + ProductionJwksSource::new( + vec![test_jwks_config()], + buzz_auth::ScriptedJwksFetcher::new([]), + ) + .expect("valid key source"), + ); + let err = super::install_nip_fi_command_components( + &mut None, + &mut None, + buzz_auth::NipFiMode::Shadow, + &IssuerRegistry::new(), + key_source, + &[], + ) + .unwrap_err(); + assert_eq!( + err, + "NIP-FI install: enforce mode requires at least one command-capable issuer" + ); + } + /// Mint a valid ES256 `nip-fi+jwt` assertion for `nostr_pubkey = key_hex`, /// signed by the route-integration-test key pair. /// Used by the startup-oracle test to verify the assertion verifier against @@ -2158,6 +2275,64 @@ mod route_integration_tests { ); } + // Pins: the cross-pod capacity and poison failsafes count on the real + // counters only in enforce; a shadow pod counts them on its own shadow + // disconnect counter, never on an enforce series. + // Mutation: reverting either site to its raw counter adds a real series + // to the shadow run (or drops the shadow series). + #[tokio::test(flavor = "current_thread")] + async fn cross_pod_failsafe_counters_stay_off_enforce_series_in_shadow() { + for (mode, expected) in [ + ( + buzz_auth::NipFiMode::Enforce, + vec![ + "buzz_nip_fi_cross_pod_capacity_exceeded_total".to_owned(), + "buzz_nip_fi_cross_pod_shard_poison_total".to_owned(), + ], + ), + ( + buzz_auth::NipFiMode::Shadow, + vec![ + "buzz_nip_fi_shadow_disconnect_total cross_pod capacity".to_owned(), + "buzz_nip_fi_shadow_disconnect_total cross_pod poison".to_owned(), + ], + ), + ] { + let with_mode = |state: Arc| { + let mut state = (*state).clone(); + Arc::make_mut(&mut state.config).nip_fi.mode = mode; + state + }; + let full = with_mode(cross_pod_state(1).await); + let poisoned = with_mode(cross_pod_state(10).await); + poisoned + .nip_fi_deny_map + .as_deref() + .expect("deny map present") + .poison_shard_for_test(TEST_ISS); + let recorder = metrics_util::debugging::DebuggingRecorder::new(); + let snapshotter = recorder.snapshotter(); + let _guard = metrics::set_default_local_recorder(&recorder); + for state in [&full, &full, &poisoned] { + let key = nostr::Keys::generate().public_key(); + apply_nip_fi_disconnect(state, &cross_pod_message(&key), chrono::Utc::now()); + } + let mut series: Vec = snapshotter + .snapshot() + .into_vec() + .into_iter() + .map(|(key, ..)| { + let key = key.key(); + let labels = key.labels().map(|l| format!(" {}", l.value())); + format!("{}{}", key.name(), labels.collect::()) + }) + .filter(|name| name.starts_with("buzz_nip_fi")) + .collect(); + series.sort(); + assert_eq!(series, expected, "{mode:?}"); + } + } + // ── Cross-pod consumer: rejection and ceiling clamp ────────────────────── // // Only malformed messages are rejected. An `until` beyond the receiving @@ -2793,6 +2968,72 @@ mod route_integration_tests { .await; } + /// Pins: an admin disconnect counts its accept and its capacity rejection + /// on the real counters only in enforce; a shadow pod answers the same + /// way but counts both on its shadow disconnect counter. + /// Mutation: reverting either admin site to its raw counter puts a real + /// series in the shadow run. + #[test] + fn admin_disconnect_counters_stay_off_enforce_series_in_shadow() { + let rt = tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + .expect("current_thread runtime"); + for (mode, expected) in [ + ( + NipFiMode::Enforce, + vec![ + "buzz_nip_fi_disconnect_capacity_rejections_total", + "buzz_nip_fi_disconnect_total", + "buzz_nip_fi_sessions_closed_total admin_disconnect", + ], + ), + ( + NipFiMode::Shadow, + vec![ + "buzz_nip_fi_shadow_disconnect_total admin accepted", + "buzz_nip_fi_shadow_disconnect_total admin capacity", + ], + ), + ] { + let mut config = crate::config::Config::for_test(); + config.nip_fi.mode = mode; + config.nip_fi.registry.insert(test_issuer_policy()); + let state = Arc::new(rt.block_on(build_test_app_state(1, config))); + let recorder = metrics_util::debugging::DebuggingRecorder::new(); + let snapshotter = recorder.snapshotter(); + let statuses = metrics::with_local_recorder(&recorder, || { + rt.block_on(async { + let mut statuses = Vec::new(); + for _ in 0..2 { + let t = target_hex(); + let token = mint_token(&t, 300, serde_json::json!({})); + statuses.push(post_command(&state, &token, &t).await.0); + } + statuses + }) + }); + assert_eq!( + statuses, + [StatusCode::OK, StatusCode::SERVICE_UNAVAILABLE], + "{mode:?}" + ); + let mut series: Vec = snapshotter + .snapshot() + .into_vec() + .into_iter() + .map(|(key, ..)| { + let key = key.key(); + let labels = key.labels().map(|l| format!(" {}", l.value())); + format!("{}{}", key.name(), labels.collect::()) + }) + .filter(|name| name.starts_with("buzz_nip_fi")) + .collect(); + series.sort(); + assert_eq!(series, expected, "{mode:?}"); + } + } + /// `buzz_nip_fi_disconnect_capacity_rejections_total` counts exactly the /// `DenySetFull` rejections and no other command-error arm. /// @@ -3272,6 +3513,245 @@ mod route_integration_tests { Arc::new(buzz_pubsub::RedisCommandReplayGuard::new(pool)) } + async fn pod_in(mode: NipFiMode) -> Arc { + let mut config = crate::config::Config::for_test(); + config.nip_fi.mode = mode; + config.nip_fi.registry.insert(test_issuer_policy()); + let mut state = build_test_app_state(1000, config).await; + state.nip_fi_command_replay = + command_replay_guard(state.redis_pool.clone(), state.config.nip_fi.mode); + Arc::new(state) + } + + /// Shadow startup goes through the same `AppState::new` as production: + /// it builds the assertion verifier, installs a command verifier on + /// the shared key source, and claims commands under the shadow replay + /// prefix, leaving the enforce claim for an enforce pod. + #[tokio::test] + #[ignore = "requires Redis"] + async fn app_state_new_in_shadow_builds_verifiers_and_shadow_replay_guard() { + use crate::nip_fi_config::NipFiRelayConfig; + use buzz_auth::CommandReplayGuard as _; + + let mut config = crate::config::Config::for_test(); + config.redis_url = std::env::var("BUZZ_TEST_REDIS_URL") + .or_else(|_| std::env::var("REDIS_URL")) + .unwrap_or_else(|_| "redis://127.0.0.1:6379".into()); + let mut registry = IssuerRegistry::new(); + registry.insert(test_issuer_policy()); + let command_configs = vec![( + TEST_ISS.to_owned(), + CommandIssuerEnvConfig { + maximum_command_age_seconds: Some(30), + authorized_principals: Some(vec![TEST_SUB.to_owned()]), + deny_set_capacity: Some(100), + }, + )]; + config.nip_fi = NipFiRelayConfig { + mode: NipFiMode::Shadow, + registry: registry.clone(), + jwks_configs: vec![test_jwks_config()], + command_configs: command_configs.clone(), + max_connection_lifetime_secs: 3600, + communities: crate::nip_fi_core::test_support::any_host(TEST_AUD), + }; + let pool = sqlx::PgPool::connect_lazy(&config.database_url).unwrap(); + let db = buzz_db::Db::from_pool(pool.clone()); + let redis_pool = deadpool_redis::Config::from_url(&config.redis_url) + .create_pool(Some(deadpool_redis::Runtime::Tokio1)) + .unwrap(); + let pubsub = Arc::new( + buzz_pubsub::PubSubManager::new(&config.redis_url, redis_pool.clone()) + .await + .unwrap(), + ); + let (mut state, _) = crate::state::AppState::new( + config.clone(), + db.clone(), + redis_pool.clone(), + buzz_audit::AuditService::new(pool.clone()), + pubsub, + buzz_auth::AuthService::new(config.auth.clone()), + buzz_search::SearchService::new(pool), + Arc::new(buzz_workflow::WorkflowEngine::new( + db, + buzz_workflow::WorkflowConfig::default(), + )), + nostr::Keys::generate(), + buzz_media::MediaStorage::new(&config.media).unwrap(), + ); + assert!( + state.nip_fi_verifier.is_some(), + "shadow must build the verifier" + ); + let source = state.nip_fi_jwks_source.clone().expect("shared key source"); + super::super::install_nip_fi_command_components( + &mut state.nip_fi_deny_map, + &mut state.nip_fi_command_verifier, + NipFiMode::Shadow, + ®istry, + Arc::clone(&source), + &command_configs, + ) + .expect("shadow installs the command verifier"); + source.seed_snapshot_for_test(TEST_ISS, test_jwks()).await; + let target = nostr::Keys::generate().public_key(); + let command = mint_token(&target.to_hex(), 300, serde_json::json!({})); + let verifier = state.nip_fi_command_verifier.as_ref().unwrap(); + let verified = + verifier.verify_at(&command, "POST", TEST_PATH, &target, chrono::Utc::now()); + assert!(verified.is_ok(), "shadow verifies commands: {verified:?}"); + + let jti = uuid::Uuid::new_v4().to_string(); + assert!(state + .nip_fi_command_replay + .try_claim(TEST_ISS, &jti, 60) + .await + .unwrap()); + let shadow = buzz_pubsub::RedisCommandReplayGuard::shadow(redis_pool.clone()); + let enforce = buzz_pubsub::RedisCommandReplayGuard::new(redis_pool); + assert!( + !shadow.try_claim(TEST_ISS, &jti, 60).await.unwrap(), + "claim is shadow-prefixed" + ); + assert!( + enforce.try_claim(TEST_ISS, &jti, 60).await.unwrap(), + "enforce claim untouched" + ); + } + + /// Run `state`'s production disconnect subscriber and apply what it + /// hears through the production consumer, like `main.rs`. Yields + /// the target of each applied message. + fn listen(state: &Arc) -> tokio::sync::mpsc::UnboundedReceiver> { + let mut rx = state.pubsub.subscribe_nip_fi_disconnect(); + let channels = disconnect_subscribe_channels(state.config.nip_fi.mode); + tokio::spawn(Arc::clone(&state.pubsub).run_nip_fi_disconnect_subscriber(channels)); + let consumer = Arc::clone(state); + let (seen_tx, seen) = tokio::sync::mpsc::unbounded_channel(); + tokio::spawn(async move { + while let Ok(msg) = rx.recv().await { + apply_nip_fi_disconnect(&consumer, &msg, chrono::Utc::now()); + let _ = seen_tx.send(msg.pubkey_bytes); + } + }); + seen + } + + /// Return once a probe that `prober` publishes has been applied via + /// `seen`: the subscription is live, and anything published earlier + /// on the probe's channel has been applied. + async fn await_probe( + seen: &mut tokio::sync::mpsc::UnboundedReceiver>, + prober: &Arc, + ) { + for _ in 0..50 { + let probe = nostr::Keys::generate().public_key(); + let token = mint_token(&probe.to_hex(), 300, serde_json::json!({})); + post_command(prober, &token, &probe.to_hex()).await; + drain_publishes(prober).await; + let wait = std::time::Duration::from_millis(200); + while let Ok(Some(k)) = tokio::time::timeout(wait, seen.recv()).await { + if k == probe.to_bytes().to_vec() { + return; + } + } + } + panic!("disconnect subscriber never received a probe"); + } + + /// Return once `seen` has applied a message for `key`. + async fn await_target( + seen: &mut tokio::sync::mpsc::UnboundedReceiver>, + key: &nostr::PublicKey, + ) { + let wait = std::time::Duration::from_secs(5); + while let Ok(Some(k)) = tokio::time::timeout(wait, seen.recv()).await { + if k == key.to_bytes().to_vec() { + return; + } + } + panic!("disconnect subscriber never received the target"); + } + + // Pins finding 1 (disconnect half): a shadow disconnect goes out only + // on the shadow channel, so an enforce pod denies and closes nothing. + // Other shadow pods receive it and record the deny; no shadow pod + // closes a session, whether the command came from shadow or enforce. + // Mutation: publishing shadow commands on `NIP_FI_DISCONNECT_CHANNEL` + // closes the enforce session; letting a shadow pod close on a + // received command fails `assert_open` on the shadow pods. + #[tokio::test] + #[ignore = "requires Redis"] + async fn shadow_disconnect_never_reaches_an_enforce_pod() { + let shadow = pod_in(NipFiMode::Shadow).await; + let peer = pod_in(NipFiMode::Shadow).await; + let enforce = pod_in(NipFiMode::Enforce).await; + let mut enforce_seen = listen(&enforce); + let mut shadow_seen = listen(&shadow); + let mut peer_seen = listen(&peer); + await_probe(&mut enforce_seen, &enforce).await; + await_probe(&mut shadow_seen, &enforce).await; + await_probe(&mut peer_seen, &enforce).await; + let key = nostr::Keys::generate().public_key(); + let on_shadow = IssuerSessions::register(&shadow, TEST_ISS, &key); + let on_peer = IssuerSessions::register(&peer, TEST_ISS, &key); + let on_enforce = IssuerSessions::register(&enforce, TEST_ISS, &key); + + let token = mint_token(&key.to_hex(), 300, serde_json::json!({})); + let (status, body) = post_command(&shadow, &token, &key.to_hex()).await; + assert_eq!(status, StatusCode::OK); + assert_eq!(&body[..], br#"{"disconnected": true}"#); + assert!(is_denied(&shadow, &key), "shadow records its deny"); + on_shadow.assert_open("shadow pod"); + drain_publishes(&shadow).await; + // Anything the enforce pod would hear from that publish has + // arrived once a later enforce-channel probe has. + await_probe(&mut enforce_seen, &enforce).await; + assert!(!is_denied(&enforce, &key), "enforce records no shadow deny"); + on_enforce.assert_open("enforce pod"); + await_target(&mut peer_seen, &key).await; + assert!( + is_denied(&peer, &key), + "a second shadow pod records the deny" + ); + on_peer.assert_open("second shadow pod"); + + // An enforce disconnect reaching a shadow pod records the deny + // and closes nothing there. + let target = nostr::Keys::generate().public_key(); + let target_on_shadow = IssuerSessions::register(&shadow, TEST_ISS, &target); + let token = mint_token(&target.to_hex(), 300, serde_json::json!({})); + let (status, _) = post_command(&enforce, &token, &target.to_hex()).await; + assert_eq!(status, StatusCode::OK); + drain_publishes(&enforce).await; + await_target(&mut shadow_seen, &target).await; + assert!(is_denied(&shadow, &target), "shadow records enforce's deny"); + target_on_shadow.assert_open("shadow pod, enforce command"); + } + + // Pins finding 1 (replay half): shadow and enforce share one Redis + // but claim under disjoint prefixes, so a shadow accept leaves the + // command usable by enforce, while shadow still rejects its own + // replay. Mutation: a shadow guard on the enforce prefix makes the + // enforce use a 403. + #[tokio::test] + #[ignore = "requires Redis"] + async fn shadow_command_claim_never_uses_up_enforce_claim() { + let shadow = pod_in(NipFiMode::Shadow).await; + let enforce = pod_in(NipFiMode::Enforce).await; + let key = nostr::Keys::generate().public_key(); + let token = mint_token(&key.to_hex(), 300, serde_json::json!({})); + + let (status, _) = post_command(&shadow, &token, &key.to_hex()).await; + assert_eq!(status, StatusCode::OK, "shadow accepts first use"); + let (status, _) = post_command(&enforce, &token, &key.to_hex()).await; + assert_eq!(status, StatusCode::OK, "enforce still accepts it"); + let (status, body) = post_command(&shadow, &token, &key.to_hex()).await; + assert_eq!(status, StatusCode::FORBIDDEN, "shadow rejects its replay"); + assert_eq!(body.as_ref(), b"authorization denied\n"); + } + #[tokio::test] #[ignore = "requires Redis"] async fn redis_command_replay_on_second_pod_is_denied() { diff --git a/crates/buzz-relay/src/api/workflows.rs b/crates/buzz-relay/src/api/workflows.rs index 5021c1ee19..673de97860 100644 --- a/crates/buzz-relay/src/api/workflows.rs +++ b/crates/buzz-relay/src/api/workflows.rs @@ -17,8 +17,6 @@ use uuid::Uuid; use buzz_core::TenantContext; -use buzz_auth::NipFiMode; - use crate::{ api::{api_error, bridge, internal_error, parse_query_or_400}, nip_fi_http::admit_nip_fi_http_on_state, @@ -58,6 +56,7 @@ async fn authorize_workflow_read( let tenant = crate::tenant::bind_community(&state.db, raw_host) .await .map_err(|_| { + crate::nip_fi_shadow::observe_unbound(state, headers); api_error( StatusCode::NOT_FOUND, "relay: no community is configured for this host", @@ -70,7 +69,7 @@ async fn authorize_workflow_read( // In NIP-FI enforce/deny-protected mode a real NIP-98 event is mandatory — // the X-Pubkey dev-mode fallback must never satisfy the pairing requirement. // [NIP-FI.md:594-607, FI-TRACE-HTTP-INGRESS] - let nip_fi_active = !matches!(state.config.nip_fi.mode, NipFiMode::Off); + let nip_fi_active = state.config.nip_fi.mode.restricts(); // NIP-FI admission. [FI-TRACE-AUTHORITY-UNIFORM] let require_auth = state.config.require_auth_token || nip_fi_active; diff --git a/crates/buzz-relay/src/audio/handler.rs b/crates/buzz-relay/src/audio/handler.rs index 36771cca7e..a9272e8e2a 100644 --- a/crates/buzz-relay/src/audio/handler.rs +++ b/crates/buzz-relay/src/audio/handler.rs @@ -95,15 +95,22 @@ pub async fn ws_audio_handler( // WebSocket handshake. Running pre-lookup means a denied request pays zero // DB cost and the gate is reachable in tests without a live community. // [FI-TRACE-TRANSPORT-CLOSED] [NIP-FI.md §Admission pairing sequence] - let nip_fi_assertion = { - use crate::nip_fi_upgrade::{check_nip_fi_at_upgrade, NipFiUpgradeOutcome}; + let (nip_fi_assertion, shadow_assertion) = { + use crate::nip_fi_upgrade::check_nip_fi_at_upgrade; let mode = state.config.nip_fi.mode; let verifier = state.nip_fi_verifier.as_deref(); let communities = &state.config.nip_fi.communities; - match check_nip_fi_at_upgrade(&headers, communities, verifier, mode) { - NipFiUpgradeOutcome::NotRequired => None, - NipFiUpgradeOutcome::Admitted(assertion) => Some(assertion), - NipFiUpgradeOutcome::Denied(resp) => return resp.into_response(), + match check_nip_fi_at_upgrade( + crate::nip_fi_session::NipFiWsRoute::Audio, + &headers, + communities, + verifier, + mode, + ) + .into_assertions() + { + Ok(assertions) => assertions, + Err(resp) => return resp.into_response(), } }; @@ -151,6 +158,15 @@ pub async fn ws_audio_handler( // so the NIP-FI session partition is rooted at the HTTP handshake, not the // post-community-active-check instant. [FI-TRACE-LEASE-BOUND] let connection_time = chrono::Utc::now(); + let shadow = shadow_assertion.map(|a| { + crate::nip_fi_shadow_session::ShadowSession::start( + &state, + "audio", + &headers, + a, + connection_time, + ) + }); limit_audio_websocket(ws).on_upgrade(move |socket| { handle_audio_connection( socket, @@ -159,6 +175,7 @@ pub async fn ws_audio_handler( channel_id, permit, nip_fi_assertion, + shadow, connection_time, ) }) @@ -199,6 +216,7 @@ fn default_protocol_version() -> u8 { } #[cfg_attr(test, allow(dead_code))] +#[allow(clippy::too_many_arguments)] pub(crate) async fn handle_audio_connection( socket: WebSocket, state: Arc, @@ -206,6 +224,7 @@ pub(crate) async fn handle_audio_connection( channel_id: Uuid, _permit: OwnedSemaphorePermit, nip_fi_assertion: Option, + nip_fi_shadow: Option>, connection_time: chrono::DateTime, ) { let cancel = CancellationToken::new(); @@ -234,6 +253,7 @@ pub(crate) async fn handle_audio_connection( let (pre_terminal_ctrl_tx, pre_terminal_ctrl_rx) = tokio::sync::mpsc::channel::(1); let control = CommunityConnectionControl::new(cancel); + control.attach_nip_fi_shadow(nip_fi_shadow); let drain_reason = control.disconnect_reason(); let pre_expiry_task = audio_session_deadline.map(|deadline| { crate::nip_fi_session::spawn_nip_fi_expiry_task( @@ -560,6 +580,8 @@ pub(crate) async fn handle_active_audio_connection( // [nip_fi_test_hooks::audio_auth_verify_hook] #[cfg(test)] crate::nip_fi_test_hooks::before_auth_verify(tenant.community()).await; + // Every exit before the join commit leaves the shadow session unadmitted. + let shadow_attempt = crate::nip_fi_shadow_session::AuthAttempt(control.nip_fi_shadow()); let auth_ctx = tokio::select! { biased; @@ -578,6 +600,7 @@ pub(crate) async fn handle_active_audio_connection( Ok(ctx) => ctx, Err(e) => { warn!(channel_id = %channel_id, "audio auth failed: {e}"); + shadow_attempt.refused(); // Under NIP-FI the failure is classified exactly as on the // root route; Off-mode keeps the legacy bespoke frame. let fi = nip_fi_assertion.is_some(); @@ -593,6 +616,9 @@ pub(crate) async fn handle_active_audio_connection( .into(), ) }; + #[cfg(test)] + crate::nip_fi_test_hooks::before_audio_refusal_frames(tenant.community()) + .await; crate::connection::send_exit_frames_bounded(&mut ws_send, [frame]).await; if fi { cancel.cancel(); @@ -617,6 +643,9 @@ pub(crate) async fn handle_active_audio_connection( crate::nip_fi_test_hooks::record_pairing_reached_after_cancel(tenant.community()); } + if let Some(shadow) = control.nip_fi_shadow() { + shadow.observe_pairing(pubkey); + } // NIP-FI key pairing [FI-INV-05]: unconditional, using the shared production // seam. When an assertion was presented at upgrade, the proven NIP-42 key // MUST equal the assertion's `nostr_pubkey` claim. Claimless assertion is @@ -686,6 +715,9 @@ pub(crate) async fn handle_active_audio_connection( } } } + if let Some(shadow) = control.nip_fi_shadow() { + shadow.observe_deny_set(&state); + } #[cfg(test)] crate::nip_fi_test_hooks::after_deny_set_check_passed(tenant.community()).await; @@ -723,6 +755,9 @@ pub(crate) async fn handle_active_audio_connection( } }; if let Some(class) = relay_refusal { + shadow_attempt.refused(); + #[cfg(test)] + crate::nip_fi_test_hooks::before_audio_refusal_frames(tenant.community()).await; // Off mode keeps one legacy frame for both outcomes. exit_authorization_refusal( &mut ws_send, @@ -749,6 +784,7 @@ pub(crate) async fn handle_active_audio_connection( { Ok(admission) => admission, Err(refusal) => { + shadow_attempt.refused(); warn!(channel_id = %channel_id, pubkey = %pubkey_hex, "audio membership denied: {refusal}"); exit_authorization_refusal( &mut ws_send, @@ -799,6 +835,7 @@ pub(crate) async fn handle_active_audio_connection( .await }; if !linked { + shadow_attempt.refused(); warn!(channel_id = %channel_id, pubkey = %pubkey_hex, nip_oa_owner = %owner.to_hex(), "audio: NIP-OA owner could not be materialized, denying"); let deny_frame = authorization_exit_frame( @@ -842,6 +879,7 @@ pub(crate) async fn handle_active_audio_connection( } }; if let Some(denial) = denial { + shadow_attempt.refused(); let (class, message) = (denial.class, denial.reason); warn!(channel_id = %channel_id, pubkey = %pubkey_hex, reason = message, "audio: denied at final admission check"); exit_authorization_refusal( @@ -880,6 +918,7 @@ pub(crate) async fn handle_active_audio_connection( match state.mesh() { Some(mesh) => { if mesh.owners.is_draining() { + shadow_attempt.refused(); crate::connection::send_exit_frames_bounded( &mut ws_send, [WsMessage::Text( @@ -935,6 +974,7 @@ pub(crate) async fn handle_active_audio_connection( pending_remote = Some(resolved.outcome); } Err(e) => { + shadow_attempt.refused(); warn!( channel_id = %channel_id, pubkey = %pubkey_hex, @@ -962,6 +1002,7 @@ pub(crate) async fn handle_active_audio_connection( } None => { if !state.config.huddle_audio_available { + shadow_attempt.refused(); debug!( channel_id = %channel_id, pubkey = %pubkey_hex, @@ -1002,6 +1043,7 @@ pub(crate) async fn handle_active_audio_connection( // handles the same-room case. match state.db.get_channel(tenant.community(), channel_id).await { Ok(ch) if ch.archived_at.is_some() => { + shadow_attempt.refused(); debug!(channel_id = %channel_id, "channel archived before room join"); crate::connection::send_exit_frames_bounded( &mut ws_send, @@ -1024,6 +1066,7 @@ pub(crate) async fn handle_active_audio_connection( return; } Err(e) => { + shadow_attempt.refused(); warn!(channel_id = %channel_id, "pre-join channel check failed (fail-closed): {e}"); // I1 residual: release lease with an awaited call, not a detached task. if let Some((lease, directory)) = staged_lease { @@ -1046,6 +1089,7 @@ pub(crate) async fn handle_active_audio_connection( // pin a room to a version we can't speak. Versions 1..=CURRENT are OK. let requested_version = auth_msg.protocol_version; if requested_version == 0 || requested_version > CURRENT_PROTOCOL_VERSION { + shadow_attempt.refused(); warn!( channel_id = %channel_id, pubkey = %pubkey_hex, @@ -1123,6 +1167,7 @@ pub(crate) async fn handle_active_audio_connection( remote_fence = Some(Arc::clone(&mesh.audio_fence)); } Err(crate::audio::join::DialError::Rejected(reason)) => { + shadow_attempt.refused(); warn!(channel_id = %channel_id, pubkey = %pubkey_hex, "huddle owner rejected registration: {reason:?}"); crate::connection::send_exit_frames_bounded( &mut ws_send, @@ -1143,6 +1188,7 @@ pub(crate) async fn handle_active_audio_connection( return; } Err(crate::audio::join::DialError::Mesh(e)) => { + shadow_attempt.refused(); warn!(channel_id = %channel_id, pubkey = %pubkey_hex, "huddle owner registration failed: {e}"); crate::connection::send_exit_frames_bounded( &mut ws_send, @@ -1237,6 +1283,7 @@ pub(crate) async fn handle_active_audio_connection( match add_peer_result { Ok(v) => v, Err(crate::audio::room::AdmissionError::Full) => { + shadow_attempt.refused(); warn!(channel_id = %channel_id, "audio room participant capacity reached"); crate::connection::send_exit_frames_bounded(&mut ws_send, [WsMessage::Text(serde_json::json!({"type":"error","code":"room_full","message":"room participant capacity reached"}).to_string().into())]).await; // IMPORTANT 3: cancel + await expiry task before guard release. @@ -1248,6 +1295,7 @@ pub(crate) async fn handle_active_audio_connection( return; } Err(crate::audio::room::AdmissionError::Ended) => { + shadow_attempt.refused(); debug!(channel_id = %channel_id, "room ended before admission"); crate::connection::send_exit_frames_bounded(&mut ws_send, [WsMessage::Text(serde_json::json!({"type":"error","code":"room_ended","message":"huddle has ended"}).to_string().into())]).await; // IMPORTANT 3: cancel + await expiry task before guard release. @@ -1259,6 +1307,7 @@ pub(crate) async fn handle_active_audio_connection( return; } Err(crate::audio::room::AdmissionError::VersionMismatch { pinned, requested }) => { + shadow_attempt.refused(); info!(channel_id = %channel_id, pubkey = %pubkey_hex, pinned, requested, "audio: protocol version mismatch — upgrade required"); crate::connection::send_exit_frames_bounded(&mut ws_send, [WsMessage::Text(serde_json::json!({ "type": "error", "code": "upgrade_required", @@ -1425,7 +1474,7 @@ pub(crate) async fn handle_active_audio_connection( // the first `joined` the connecting client sees. [FI-TRACE-BOOTSTRAP-ORDER-BARRIER] let bootstrap_joined_msg: String; - match commit_participant_join( + let committed = commit_participant_join( &state, &tenant, channel_id, @@ -1446,9 +1495,13 @@ pub(crate) async fn handle_active_audio_connection( // `joined` payload's peers[] includes Alice and any other owner-pod participants. // Same-pod joins pass None — room.roster_snapshot() is authoritative there. guard.remote_session.as_ref().map(|s| s.roster()), + control.nip_fi_shadow().map(Arc::as_ref), ) - .await - { + .await; + if committed.is_err() { + shadow_attempt.refused(); + } + match committed { Ok(CommitJoinOutcome::JoinedSent(msg)) => { // Bootstrap prepared inside the permit; assigned here for ordered // write to `ctrl_tx` after it is created below, before task spawns. @@ -2954,7 +3007,15 @@ async fn commit_participant_join( // contains every live participant. `None` for same-pod joins (local room is // authoritative). [FI-TRACE-JOINED-PAYLOAD-COMMITTED] owner_roster: Option<&crate::audio::join::RosterSnapshot>, + // Admitted the instant the join commits, before publication awaits; + // retired at each refusal below, before its rollback awaits. + shadow: Option<&crate::nip_fi_shadow_session::ShadowSession>, ) -> Result { + let retire_shadow = || { + if let Some(shadow) = shadow { + shadow.retire_pending(); + } + }; // 1. Sign the 48101 event synchronously. // // Fix 1: include `generation` so desktop can fence the first liveness @@ -3033,6 +3094,9 @@ async fn commit_participant_join( crate::nip_fi_test_hooks::before_archive_recheck(tenant.community()).await; if channel_archived_early.is_some() { + retire_shadow(); + #[cfg(test)] + crate::nip_fi_test_hooks::before_join_refusal_rollback(tenant.community()).await; let _ = tx.rollback().await; return Err(JoinCommitError::Archived); } @@ -3077,6 +3141,7 @@ async fn commit_participant_join( .flatten(); if channel_archived.is_some() { + retire_shadow(); let _ = tx.rollback().await; return Err(JoinCommitError::Archived); } @@ -3093,6 +3158,7 @@ async fn commit_participant_join( .await?; if !parent_still_member { + retire_shadow(); let _ = tx.rollback().await; return Err(JoinCommitError::ParentMembershipLost); } @@ -3113,6 +3179,7 @@ async fn commit_participant_join( .await?; if !link_still_exists { + retire_shadow(); let _ = tx.rollback().await; return Err(JoinCommitError::HuddleLinkGone); } @@ -3171,6 +3238,9 @@ async fn commit_participant_join( if let Err(e) = tx.commit().await { return Err(JoinCommitError::Db(e.into())); } + if let Some(shadow) = shadow { + shadow.admit(); + } // Fix B (commit-before-publish): call `commit_peer` which atomically // marks the peer committed, increments the roster revision, and fires @@ -5838,6 +5908,7 @@ mod tests { channel_id, permit, Some(assertion_i), + None, conn_time, ) .await @@ -5931,6 +6002,180 @@ mod tests { Arc::new(state) } + /// Runs one audio connection observed by a shadow session asserting + /// `asserted` through a NIP-42 AUTH `key` signs, until the handler ends + /// it; returns the session and the text frames the client received. + async fn shadow_audio_auth( + state: Arc, + key: &nostr::Keys, + asserted: nostr::PublicKey, + ) -> ( + Arc, + Vec, + ) { + let session = audio_shadow_session(&state, asserted); + let community = buzz_core::tenant::CommunityId::from_uuid(uuid::Uuid::new_v4()); + let frames = shadow_audio_auth_in(state, key, Arc::clone(&session), community).await; + (session, frames) + } + + /// A shadow session for an audio connection asserting `asserted`. + fn audio_shadow_session( + state: &crate::state::AppState, + asserted: nostr::PublicKey, + ) -> Arc { + use chrono::{Duration, Utc}; + let assertion = + VerifiedAssertion::for_test(Some(asserted), vec![Utc::now() + Duration::hours(1)]); + crate::nip_fi_shadow_session::ShadowSession::start( + state, + "audio", + &axum::http::HeaderMap::new(), + assertion, + Utc::now(), + ) + } + + /// [`shadow_audio_auth`] observed by `session` in `community`. + async fn shadow_audio_auth_in( + state: Arc, + key: &nostr::Keys, + session: Arc, + community: buzz_core::tenant::CommunityId, + ) -> Vec { + use chrono::Utc; + let tenant = + buzz_core::tenant::TenantContext::resolved(community, "test.local".to_string()); + let control = crate::state::CommunityConnectionControl::new(CancellationToken::new()); + control.attach_nip_fi_shadow(Some(Arc::clone(&session))); + let slot = Arc::new(std::sync::Mutex::new(Some(control))); + let listener = TcpListener::bind("127.0.0.1:0").await.expect("bind"); + let addr = listener.local_addr().expect("addr"); + let app = Router::new().route( + "/", + get(move |ws: WebSocketUpgrade| async move { + let control = slot.lock().unwrap().take().expect("one connection"); + ws.on_upgrade(move |socket| { + handle_active_audio_connection( + socket, + state, + tenant, + uuid::Uuid::new_v4(), + control, + None, + Utc::now(), + None, + ) + }) + }), + ); + let server = tokio::spawn(async move { axum::serve(listener, app).await }); + let (mut client, _) = connect_async(format!("ws://{addr}/")) + .await + .expect("connect"); + let Some(Ok(tokio_tungstenite::tungstenite::Message::Text(challenge))) = + client.next().await + else { + panic!("expected a challenge"); + }; + let challenge: serde_json::Value = + serde_json::from_str(&challenge).expect("challenge JSON"); + let event = nostr::EventBuilder::new(nostr::Kind::Authentication, "") + .tag(nostr::Tag::parse(["relay", "ws://test.local"]).unwrap()) + .tag( + nostr::Tag::parse(["challenge", challenge["challenge"].as_str().unwrap()]).unwrap(), + ) + .sign_with_keys(key) + .unwrap(); + let auth = serde_json::json!({"type": "auth", "event": event}).to_string(); + client + .send(tokio_tungstenite::tungstenite::Message::Text(auth.into())) + .await + .expect("send auth"); + let mut frames = Vec::new(); + let drained = tokio::time::timeout(std::time::Duration::from_secs(5), async { + while let Some(Ok(msg)) = client.next().await { + if let tokio_tungstenite::tungstenite::Message::Text(text) = msg { + frames.push(text.to_string()); + } + } + }); + drained.await.expect("the handler ends the connection"); + server.abort(); + frames + } + + /// Runs [`shadow_audio_auth_in`] for a paired key, holding its refusal + /// frames while the deadline fires; returns the frames. + async fn held_refusal_deadline(state: crate::state::AppState) -> Vec { + let state = Arc::new(state); + let key = nostr::Keys::generate(); + let session = audio_shadow_session(&state, key.public_key()); + let community = buzz_core::tenant::CommunityId::from_uuid(uuid::Uuid::new_v4()); + let (arrived, release) = + crate::nip_fi_test_hooks::audio_refusal_frames_hook::arm(community); + let held = async { + arrived.await.expect("the refusal is decided"); + session.expire(); + release.notify_one(); + }; + let (frames, ()) = tokio::join!( + shadow_audio_auth_in(Arc::clone(&state), &key, Arc::clone(&session), community), + held + ); + frames + } + + // A NIP-42 failure retires the session where it is decided: the deadline + // firing while its refusal frame is held records nothing. Mutation: + // deleting the failure arm's `refused()` records `deadline/rejected`. + #[test] + fn shadow_audio_nip42_failure_retires_before_its_frame() { + use crate::nip_fi_shadow_session::tests::{shadow_records, shadow_state}; + let records = shadow_records(async { + let mut state = shadow_state(None).await; + let mut config = (*state.config).clone(); + // The AUTH names `ws://test.local`; a wss relay expects `wss://`. + config.relay_url = "wss://relay.test".to_owned(); + state.config = Arc::new(config); + let frames = held_refusal_deadline(state).await; + assert!( + frames.iter().any(|f| f.contains("auth failed")), + "{frames:?}" + ); + }); + assert!(records.is_empty(), "{records:?}"); + } + + // Audio records shadow pairing and the deny-set check where enforce runs + // them; a connection refused after a clean deny-set check is never + // admitted, so it records nothing, even at its deadline. Mutation: + // deleting audio's `observe_pairing` or `observe_deny_set` call drops + // its record; recording the admit before the join commit records one. + #[test] + fn shadow_audio_auth_records_one_verdict_where_enforce_decides() { + use crate::nip_fi_shadow_session::tests::{shadow_records, shadow_state}; + let cases: [(bool, bool, &[&str]); 3] = [ + (false, false, &["pairing/denied"]), + (true, true, &["deny_set/denied"]), + (true, false, &[]), + ]; + for (paired, deny_listed, expected) in cases { + let records = shadow_records(async move { + let key = nostr::Keys::generate(); + let asserted = if paired { + key.public_key() + } else { + nostr::Keys::generate().public_key() + }; + let state = Arc::new(shadow_state(deny_listed.then(|| key.public_key())).await); + let (session, _) = shadow_audio_auth(state, &key, asserted).await; + session.expire(); + }); + assert_eq!(records, expected); + } + } + /// Consumes Ping/Pong until the socket terminates (Close, EOF or error) /// within 5s; panics on any data frame or if termination never arrives. async fn assert_terminates_without_data(client: &mut S, tag: &str) @@ -6957,6 +7202,65 @@ mod tests { mod postgres_tests { use super::*; + // A paired, deny-clean audio AUTH refused by ordinary policy, relay + // membership on a closed relay or channel membership, is never + // admitted: no record, and none at its deadline. Mutation: recording + // the admit before the join commit records `admit/admit`. + #[test] + #[ignore = "requires Postgres — runs in postgres-ci nextest lane"] + fn shadow_audio_policy_refusal_records_nothing() { + use crate::nip_fi_shadow_session::tests::{shadow_records, shadow_state}; + for (closed_relay, refusal) in [(true, "not a relay member"), (false, "not a member")] { + let records = shadow_records(async move { + let key = nostr::Keys::generate(); + let mut state = (*audio_test_state_real_db() + .await + .expect("postgres-ci provides a database")) + .clone(); + let mut config = (*state.config).clone(); + config.require_relay_membership = closed_relay; + state.config = std::sync::Arc::new(config); + state.nip_fi_deny_map = shadow_state(None).await.nip_fi_deny_map; + state.nip_fi_shadow_sessions = std::sync::Arc::default(); + let (session, frames) = + shadow_audio_auth(std::sync::Arc::new(state), &key, key.public_key()).await; + assert!( + frames.iter().any(|f| f.contains(refusal)), + "{refusal}: {frames:?}" + ); + session.expire(); + }); + assert!(records.is_empty(), "{refusal}: {records:?}"); + } + } + + // A relay-membership refusal retires the session where it is decided: + // the deadline firing while its refusal frame is held records nothing. + // Mutation: deleting that site's `refused()` records + // `deadline/rejected`. + #[test] + #[ignore = "requires Postgres — runs in postgres-ci nextest lane"] + fn shadow_audio_policy_refusal_retires_before_its_frame() { + use crate::nip_fi_shadow_session::tests::{shadow_records, shadow_state}; + let records = shadow_records(async { + let mut state = (*audio_test_state_real_db() + .await + .expect("postgres-ci provides a database")) + .clone(); + let mut config = (*state.config).clone(); + config.require_relay_membership = true; + state.config = std::sync::Arc::new(config); + state.nip_fi_deny_map = shadow_state(None).await.nip_fi_deny_map; + state.nip_fi_shadow_sessions = std::sync::Arc::default(); + let frames = held_refusal_deadline(state).await; + assert!( + frames.iter().any(|f| f.contains("not a relay member")), + "{frames:?}" + ); + }); + assert!(records.is_empty(), "{records:?}"); + } + // ── Shared scripted mesh fixtures for cross-pod postgres witnesses ───── // // These types are used by `commit_confirm_timeout_at_seam`, @@ -7265,6 +7569,104 @@ mod tests { } } + /// One audio join commit observed by `shadow` after a clean deny-set + /// check and a real deny for its key (merged, or refused for capacity + /// when `capacity` is 1): the commit admits it, then ends it + /// `revoked`; a refused commit admits nothing, and retires at the + /// refusal: the deadline firing while its rollback is held records + /// nothing (deleting the helper's `retire_shadow()` records + /// `deadline/rejected`). A committed session + /// whose publication follows ends `expired`, never a pending + /// `deadline`. Mutation: admitting after the commit returns, or not + /// registering at the deny-set check, changes the records. + #[test] + #[ignore = "requires Postgres — runs in postgres-ci nextest lane"] + fn shadow_join_commit_is_the_audio_admission() { + use crate::nip_fi_shadow_session::tests::{deliver, pod, session, shadow_records}; + let cases: [(usize, bool, bool, &[&str]); 4] = [ + (8, true, false, &["admit/admit", "end/revoked"]), + (1, true, false, &["admit/admit", "end/revoked"]), + (8, false, true, &[]), + (8, false, false, &["admit/admit", "end/expired"]), + ]; + for (capacity, deny, archived, expected) in cases { + let records = shadow_records(async move { + let real = audio_test_state_real_db() + .await + .expect("postgres-ci provides a database"); + let mut state = pod(capacity).await; + state.db = real.db.clone(); + let pool = state.db.pool().clone(); + let (tenant, channel_id, member) = seed_audio_fixture(&pool).await; + if archived { + sqlx::query("UPDATE channels SET archived_at = NOW() WHERE id = $1") + .bind(channel_id) + .execute(&pool) + .await + .expect("archive channel"); + } + if capacity == 1 { + deliver(&state, &nostr::Keys::generate().public_key()); + } + let key = member.public_key(); + let shadow = session(&state, Some(key), chrono::Duration::hours(1)); + shadow.observe_pairing(key); + shadow.observe_deny_set(&state); + if deny { + deliver(&state, &key); + } + let gate = crate::nip_fi_gate::SessionAdmissionGate::new( + chrono::Utc::now() + chrono::Duration::hours(1), + CancellationToken::new(), + ); + let (arrived, release) = + crate::nip_fi_test_hooks::join_refusal_rollback_hook::arm( + tenant.community(), + ); + let held = async { + if archived { + arrived.await.expect("the join is refused"); + shadow.expire(); + release.notify_one(); + } + }; + let commit = async { + commit_participant_join( + &state, + &tenant, + channel_id, + channel_id, + &key.to_hex(), + &key.to_bytes(), + uuid::Uuid::new_v4(), + 0, + 0, + 1, + "1", + &MembershipAdmission::Existing { + parent_channel_id: channel_id, + }, + &gate, + &Arc::new(crate::audio::room::Room::new( + tenant.community(), + channel_id, + )), + None, + Some(&shadow), + ) + .await + }; + let (committed, ()) = tokio::join!(commit, held); + assert_eq!(committed.is_err(), archived, "{committed:?}"); + shadow.expire(); + }); + assert_eq!( + records, expected, + "capacity {capacity}, archived {archived}" + ); + } + } + /// F2a: committed join into an already-archived channel is rejected on /// the `Existing` path. /// @@ -7330,6 +7732,7 @@ mod tests { channel_id, )), None, // same-pod test — no owner roster + None, ) .await; @@ -7433,6 +7836,7 @@ mod tests { channel_id, )), None, // same-pod test — no owner roster + None, ) .await }); @@ -7639,6 +8043,7 @@ mod tests { child_channel_id, )), None, // same-pod test — no owner roster + None, ) .await }); @@ -7773,6 +8178,7 @@ mod tests { channel_id, )), None, // same-pod test — no owner roster + None, ) .await }); @@ -7910,6 +8316,7 @@ mod tests { channel_id, )), None, // same-pod test — no owner roster + None, ) .await }); @@ -7954,6 +8361,7 @@ mod tests { channel_id, )), None, // same-pod test — no owner roster + None, ) .await }); @@ -8061,6 +8469,7 @@ mod tests { channel_id, )), None, // same-pod test — no owner roster + None, ) .await }); @@ -8105,6 +8514,7 @@ mod tests { channel_id, )), None, // same-pod test — no owner roster + None, ) .await }); @@ -8339,6 +8749,7 @@ mod tests { child_channel_id, )), None, // same-pod test — no owner roster + None, ) .await }); @@ -8534,6 +8945,7 @@ mod tests { channel_id, )), None, // same-pod test — no owner roster + None, ) .await }); @@ -9099,6 +9511,7 @@ mod tests { channel_id, )), None, // same-pod test — no owner roster + None, ) .await }); @@ -9472,6 +9885,7 @@ mod tests { &gate, &room, None, // same-pod test — no owner roster + None, ) .await; @@ -9598,6 +10012,7 @@ mod tests { &gate2, &room2, None, // same-pod test — no owner roster + None, ) .await }); @@ -9792,6 +10207,7 @@ mod tests { &gate, &room, None, // same-pod test — no owner roster + None, ) .await; assert!( @@ -9964,6 +10380,7 @@ mod tests { &gate, &ingress_room, Some(&owner_snapshot), // cross-pod: use owner-pod roster for the broadcast + None, ) .await; assert!( @@ -10642,6 +11059,7 @@ mod tests { &gate, &room, None, + None, ) .await; assert!( diff --git a/crates/buzz-relay/src/config.rs b/crates/buzz-relay/src/config.rs index 9e4c3ef0fd..a9b300d9f2 100644 --- a/crates/buzz-relay/src/config.rs +++ b/crates/buzz-relay/src/config.rs @@ -379,10 +379,11 @@ pub struct Config { /// NIP-FI federated-identity enforcement configuration. /// - /// Present when `BUZZ_NIP_FI_MODE` is `enforce` or `deny_protected`; in - /// those modes the relay validates assertions at HTTP ingress and (via S3) - /// at WebSocket upgrade. `Off` mode (the default) leaves all identity - /// enforcement to NIP-42 alone. + /// Selected by `BUZZ_NIP_FI_MODE`: `enforce` validates assertions at HTTP + /// ingress and (via S3) at WebSocket upgrade; `deny_protected` denies + /// protected requests without evaluating; `shadow` evaluates and records + /// the enforce verdict but admits exactly as `off`. `Off` (the default) + /// leaves all identity enforcement to NIP-42 alone. pub nip_fi: crate::nip_fi_config::NipFiRelayConfig, } @@ -1417,7 +1418,9 @@ impl Config { .lock() .unwrap() .push(std::thread::current().id()); - let _fi_guard = crate::nip_fi_config::NIP_FI_ENV_LOCK.lock().unwrap(); + let _fi_guard = crate::nip_fi_config::NIP_FI_ENV_LOCK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); Self::from_env().expect("default config must load for test fixture") } } @@ -1454,7 +1457,9 @@ mod tests { std::sync::MutexGuard<'static, ()>, std::sync::MutexGuard<'static, ()>, ) { - let fi = crate::nip_fi_config::NIP_FI_ENV_LOCK.lock().unwrap(); + let fi = crate::nip_fi_config::NIP_FI_ENV_LOCK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); let cfg = ENV_MUTEX.lock().unwrap(); (fi, cfg) } diff --git a/crates/buzz-relay/src/connection.rs b/crates/buzz-relay/src/connection.rs index 61d85f63c9..f961ea4d67 100644 --- a/crates/buzz-relay/src/connection.rs +++ b/crates/buzz-relay/src/connection.rs @@ -353,12 +353,13 @@ pub(crate) fn compute_session_deadline( /// Acquires a connection semaphore permit, sends the NIP-42 AUTH challenge, /// then drives the send, heartbeat, and receive loops until the connection closes. -pub async fn handle_connection( +pub(crate) async fn handle_connection( socket: WebSocket, state: Arc, addr: SocketAddr, tenant: TenantContext, nip_fi_assertion: Option, + nip_fi_shadow: Option>, connection_time: chrono::DateTime, ) { let conn_id = Uuid::new_v4(); @@ -383,6 +384,7 @@ pub async fn handle_connection( }; let (pre_terminal_ctrl_tx, pre_terminal_ctrl_rx) = mpsc::channel::(1); let control = CommunityConnectionControl::new(cancel); + control.attach_nip_fi_shadow(nip_fi_shadow); let drain_reason = control.disconnect_reason(); let pre_expiry_task = pre_session_deadline.map(|deadline| { crate::nip_fi_session::spawn_nip_fi_expiry_task( @@ -3103,6 +3105,7 @@ pub(crate) mod tests { "127.0.0.1:9999".parse().unwrap(), tenant_i, Some(assertion_i), + None, conn_time, ) .await diff --git a/crates/buzz-relay/src/handlers/auth.rs b/crates/buzz-relay/src/handlers/auth.rs index 3e450dadad..c22c64968b 100644 --- a/crates/buzz-relay/src/handlers/auth.rs +++ b/crates/buzz-relay/src/handlers/auth.rs @@ -347,6 +347,8 @@ pub async fn handle_auth(event: nostr::Event, conn: Arc, state: crate::api::bridge::nip42_expected_relay_url(&state.config.relay_url, &conn.tenant); let auth_svc = Arc::clone(&state.auth); + let _shadow_attempt = + crate::nip_fi_shadow_session::AuthAttempt(conn.community_control.nip_fi_shadow()); // Pure NIP-42 verification — crypto only, no DB lookups. match auth_svc .verify_auth_event(event, &challenge, &relay_url) @@ -355,6 +357,10 @@ pub async fn handle_auth(event: nostr::Event, conn: Arc, state: Ok(mut auth_ctx) => { let pubkey = auth_ctx.pubkey; + let shadow = conn.community_control.nip_fi_shadow(); + if let Some(shadow) = shadow { + shadow.observe_pairing(pubkey); + } // NIP-FI key pairing [FI-INV-05]: immediately after successful // verify_auth_event, before community-ban/allowlist/membership gates. // Pre-DB positioning means a denied caller pays zero DB cost and the @@ -719,6 +725,10 @@ pub async fn handle_auth(event: nostr::Event, conn: Arc, state: } } } + if let Some(shadow) = shadow { + shadow.observe_deny_set(&state); + shadow.admit(); + } // A concurrent `disconnect_nip_fi` may have closed this session // after the deny-set check; its denial is the terminal reply. if conn.cancel.is_cancelled() { @@ -1424,9 +1434,161 @@ mod tests { // DB is available. The postgres-ci nextest lane discovers it via the `ignore` // attribute — do not remove the ignore even if a local DB is reachable. // [Fix 8: FI-TRACE-ISOLATED-DB] + /// A pending test connection observed by a shadow session asserting + /// `asserted` for `lifetime`, and an AUTH event `signer` signs for it. + pub(super) fn shadow_root_conn( + state: &crate::state::AppState, + asserted: nostr::PublicKey, + signer: &Keys, + lifetime: chrono::Duration, + ) -> (crate::connection::tests::TestConn, nostr::Event) { + use chrono::Utc; + let challenge = "shadow-root-challenge".to_owned(); + let pending = AuthState::Pending { + challenge: challenge.clone(), + started_at: Instant::now(), + }; + let t = crate::connection::tests::test_conn(pending, None); + let assertion = + buzz_auth::VerifiedAssertion::for_test(Some(asserted), vec![Utc::now() + lifetime]); + let headers = axum::http::HeaderMap::new(); + let session = crate::nip_fi_shadow_session::ShadowSession::start( + state, + "ws", + &headers, + assertion, + Utc::now(), + ); + t.conn.community_control.attach_nip_fi_shadow(Some(session)); + let event = EventBuilder::new(Kind::Authentication, "") + .tag(Tag::parse(["relay", "ws://test.local"]).unwrap()) + .tag(Tag::parse(["challenge", &challenge]).unwrap()) + .sign_with_keys(signer) + .unwrap(); + (t, event) + } + + // Shadow records pairing where enforce pairs, before any DB gate. + // Mutation: deleting root's `observe_pairing` call records nothing. + #[test] + fn shadow_root_auth_records_pairing_denial() { + use crate::nip_fi_shadow_session::tests::{shadow_records, shadow_state}; + let records = shadow_records(async { + let state = std::sync::Arc::new(shadow_state(None).await); + let (t, event) = shadow_root_conn( + &state, + Keys::generate().public_key(), + &Keys::generate(), + HOUR, + ); + handle_auth(event, std::sync::Arc::clone(&t.conn), state).await; + }); + assert_eq!(records, ["pairing/denied"]); + } + + const HOUR: chrono::Duration = chrono::Duration::hours(1); + + /// Fires the connection's shadow deadline once its AUTH has finished, + /// in place of waiting for the timer. + fn expire_shadow(t: &crate::connection::tests::TestConn) { + t.conn.community_control.nip_fi_shadow().unwrap().expire(); + } + + // A NIP-42 failure, a wrong challenge or a corrupted signature, ends the + // AUTH without a NIP-FI decision; the socket stays open until its auth + // timeout, but its observation is retired, so the deadline records + // nothing. Mutation: dropping the `AuthAttempt` guard records + // `deadline/rejected`. + #[test] + fn shadow_root_invalid_nip42_retires_without_record() { + use crate::nip_fi_shadow_session::tests::{shadow_records, shadow_state}; + let records = shadow_records(async { + let state = std::sync::Arc::new(shadow_state(None).await); + let key = Keys::generate(); + let wrong_challenge = EventBuilder::new(Kind::Authentication, "") + .tag(Tag::parse(["challenge", "other-challenge"]).unwrap()) + .sign_with_keys(&key) + .unwrap(); + let (_, signed) = shadow_root_conn(&state, key.public_key(), &key, HOUR); + let mut corrupted = serde_json::to_value(&signed).unwrap(); + let sig = corrupted["sig"].as_str().unwrap(); + let flipped = if sig.starts_with('0') { "1" } else { "0" }; + corrupted["sig"] = format!("{flipped}{}", &sig[1..]).into(); + let corrupted: nostr::Event = serde_json::from_value(corrupted).unwrap(); + for event in [wrong_challenge, corrupted] { + let (t, _) = shadow_root_conn(&state, key.public_key(), &key, HOUR); + handle_auth( + event, + std::sync::Arc::clone(&t.conn), + std::sync::Arc::clone(&state), + ) + .await; + assert!(!t.conn.cancel.is_cancelled(), "socket stays open"); + expire_shadow(&t); + } + }); + assert!(records.is_empty(), "{records:?}"); + } + mod postgres_tests { use super::*; + // Shadow records the deny-set check where enforce runs it, and admits + // with OK(true) either way, as Off does. Mutation: deleting root's + // `observe_admission` call records nothing. + #[test] + #[ignore = "requires Postgres — runs in postgres-ci nextest lane"] + fn shadow_root_auth_records_deny_set_or_admit_and_admits_as_off() { + use crate::nip_fi_shadow_session::tests::{shadow_records, shadow_state}; + for (deny_self, expected) in [(true, "deny_set/denied"), (false, "admit/admit")] { + let mut ok_true = false; + let records = shadow_records(async { + let key = Keys::generate(); + let shadow = shadow_state(deny_self.then(|| key.public_key())).await; + let mut state = (*auth_test_state_real_db_expect().await).clone(); + state.nip_fi_deny_map = shadow.nip_fi_deny_map; + let state = std::sync::Arc::new(state); + let (mut t, event) = + super::shadow_root_conn(&state, key.public_key(), &key, super::HOUR); + handle_auth(event, std::sync::Arc::clone(&t.conn), state).await; + while let Ok(WsMessage::Text(text)) = t.send_rx.try_recv() { + ok_true |= is_ok_true(text.as_str()); + } + }); + assert!(ok_true, "shadow admits as Off (deny_self = {deny_self})"); + assert_eq!(records, [expected]); + } + } + + // A paired proof refused by ordinary membership policy ends the AUTH + // without a NIP-FI decision, so the deadline records nothing. + // Mutation: dropping the `AuthAttempt` guard records + // `deadline/rejected`. + #[test] + #[ignore = "requires Postgres — runs in postgres-ci nextest lane"] + fn shadow_root_policy_refusal_retires_without_record() { + use crate::nip_fi_shadow_session::tests::{shadow_records, shadow_state}; + let records = shadow_records(async { + let key = Keys::generate(); + let mut state = (*auth_test_state_real_db_expect().await).clone(); + let mut config = (*state.config).clone(); + config.require_relay_membership = true; + state.config = std::sync::Arc::new(config); + state.nip_fi_deny_map = shadow_state(None).await.nip_fi_deny_map; + let state = std::sync::Arc::new(state); + let (mut t, event) = + super::shadow_root_conn(&state, key.public_key(), &key, super::HOUR); + handle_auth(event, std::sync::Arc::clone(&t.conn), state).await; + let refused = t.send_rx.try_recv(); + assert!( + format!("{refused:?}").contains("not a relay member"), + "{refused:?}" + ); + super::expire_shadow(&t); + }); + assert!(records.is_empty(), "{records:?}"); + } + /// A community whose row exists, so bans and users can reference it. async fn seeded_community( state: &crate::state::AppState, diff --git a/crates/buzz-relay/src/lib.rs b/crates/buzz-relay/src/lib.rs index 795d4423df..49385f6473 100644 --- a/crates/buzz-relay/src/lib.rs +++ b/crates/buzz-relay/src/lib.rs @@ -11,6 +11,8 @@ pub(crate) mod nip_fi_core; /// NIP-FI session admission gate — per-connection effect-permit and quiescence barrier. pub(crate) mod nip_fi_gate; pub(crate) mod nip_fi_session; +pub(crate) mod nip_fi_shadow; +pub(crate) mod nip_fi_shadow_session; /// NIP-FI test hooks — production barriers for deterministic B1/B2 witnesses. #[cfg(test)] pub(crate) mod nip_fi_test_hooks; diff --git a/crates/buzz-relay/src/main.rs b/crates/buzz-relay/src/main.rs index f7cc436d8b..9badfa6938 100644 --- a/crates/buzz-relay/src/main.rs +++ b/crates/buzz-relay/src/main.rs @@ -556,7 +556,13 @@ async fn run_relay_main(boot: BootTracker) -> anyhow::Result<()> { // channel capacity) instead of being dropped until the consumer below runs. let mut nip_fi_disconnect_rx = pubsub.subscribe_nip_fi_disconnect(); let pubsub_for_nip_fi = Arc::clone(&pubsub); - tokio::spawn(async move { pubsub_for_nip_fi.run_nip_fi_disconnect_subscriber().await }); + let nip_fi_channels = + buzz_relay::api::nip_fi::disconnect_subscribe_channels(config.nip_fi.mode); + tokio::spawn(async move { + pubsub_for_nip_fi + .run_nip_fi_disconnect_subscriber(nip_fi_channels) + .await + }); let auth = AuthService::new(config.auth.clone()); @@ -619,7 +625,7 @@ async fn run_relay_main(boot: BootTracker) -> anyhow::Result<()> { &nip_fi.command_configs, ) .map_err(|e| anyhow::anyhow!("NIP-FI startup failed: {e}"))?; - } else if nip_fi.is_enforce() { + } else if nip_fi.mode.evaluates() { return Err(anyhow::anyhow!( "NIP-FI: failed to construct JWKS key source \ (empty or duplicate issuer config)" @@ -1298,6 +1304,7 @@ async fn run_relay_main(boot: BootTracker) -> anyhow::Result<()> { // minimal and makes the exact production path testable end-to-end. { let state_for_nip_fi = Arc::clone(&state); + let nip_fi_mode = state.config.nip_fi.mode; tokio::spawn(async move { loop { match nip_fi_disconnect_rx.recv().await { @@ -1310,7 +1317,13 @@ async fn run_relay_main(boot: BootTracker) -> anyhow::Result<()> { ); } Err(tokio::sync::broadcast::error::RecvError::Lagged(n)) => { - metrics::counter!("buzz_nip_fi_disconnect_lag_total").increment(n); + buzz_relay::api::nip_fi::count_disconnect_event( + nip_fi_mode, + "buzz_nip_fi_disconnect_lag_total", + "cross_pod", + "lag", + n, + ); tracing::warn!("NIP-FI disconnect consumer lagged by {n} messages"); } Err(tokio::sync::broadcast::error::RecvError::Closed) => { diff --git a/crates/buzz-relay/src/nip11.rs b/crates/buzz-relay/src/nip11.rs index bd931cf320..d8f8d83c6e 100644 --- a/crates/buzz-relay/src/nip11.rs +++ b/crates/buzz-relay/src/nip11.rs @@ -111,8 +111,9 @@ pub struct RelayLimitation { pub payment_required: bool, /// Whether writes are restricted to authorized pubkeys. pub restricted_writes: bool, - /// Whether NIP-FI federated identity assertions are required at upgrade. - /// Advertised `true` when the relay is in `Enforce` mode. + /// Whether the relay supports NIP-FI federated identity assertions. + /// Advertised `true` in `Enforce` and `Shadow` mode, so clients attach + /// evidence; in `Shadow` the relay evaluates it without requiring it. #[serde(skip_serializing_if = "std::ops::Not::not")] pub federated_identity: bool, /// NIP-ER: how the relay delivers due reminders ("push" or "lazy"). @@ -345,7 +346,7 @@ pub(crate) async fn nip11_document(state: &crate::state::AppState, raw_host: &st let (relay_self, advertise_nip43) = nip11_facts(state); let icon = workspace_icon_for_host(state, raw_host).await; let admin_api = admin_api_advertisement(state.config.admin.as_ref()); - let advertise_fi = state.config.nip_fi.is_enforce(); + let advertise_fi = state.config.nip_fi.mode.evaluates(); let mut info = RelayInfo::build( relay_self.as_deref(), icon.as_deref(), @@ -474,6 +475,29 @@ const _RELAY_INFO_BUILD_STATIC_INPUT_FENCE: fn( #[cfg(test)] mod tests { + // Clients attach evidence only when the relay advertises NIP-FI, so a + // shadow relay must serve enforce's exact document. Mutation: gating on + // `enforces()` makes shadow serve Off's document → RED. + #[tokio::test] + async fn shadow_serves_the_same_nip11_document_as_enforce() { + use buzz_auth::NipFiMode; + let base = + crate::state::tests::test_state_with_database_url("postgres://127.0.0.1:1/none").await; + let mut docs = Vec::new(); + for mode in [NipFiMode::Off, NipFiMode::Enforce, NipFiMode::Shadow] { + let mut state = (*base).clone(); + let mut config = (*state.config).clone(); + config.nip_fi.mode = mode; + state.config = std::sync::Arc::new(config); + docs.push( + serde_json::to_string(&super::nip11_document(&state, "relay.example").await) + .unwrap(), + ); + } + assert_ne!(docs[0], docs[1], "enforce advertises NIP-FI"); + assert_eq!(docs[2], docs[1]); + } + use super::*; #[test] diff --git a/crates/buzz-relay/src/nip_fi_config.rs b/crates/buzz-relay/src/nip_fi_config.rs index 764ef648f6..50a6c56157 100644 --- a/crates/buzz-relay/src/nip_fi_config.rs +++ b/crates/buzz-relay/src/nip_fi_config.rs @@ -9,7 +9,7 @@ //! //! | Variable | Required | Description | //! |---|---|---| -//! | `BUZZ_NIP_FI_MODE` | No | `off` (default), `enforce`, or `deny_protected`. | +//! | `BUZZ_NIP_FI_MODE` | No | `off` (default), `shadow`, `enforce`, or `deny_protected`. `shadow` requires everything `enforce` does. | //! | `BUZZ_NIP_FI_ISSUERS` | If enforce | JSON array of issuer configs (see [`IssuerEnvConfig`]). | //! | `BUZZ_NIP_FI_COMMUNITIES` | If enforce | JSON array mapping each community's canonical URI to its authorized issuers (see [`CommunityEnvConfig`]). | //! | `BUZZ_NIP_FI_MAX_CONNECTION_LIFETIME_SECS` | If enforce | Per-partition limit on session lifetime. | @@ -97,9 +97,9 @@ pub(super) struct IssuerEnvConfig { /// Hard deadline for accepting a JWKS snapshot in seconds. pub jwks_hard_deadline_seconds: u64, - // ── S4 command-API fields (required in enforce mode) ────────────────── + // ── S4 command-API fields (required in enforce and shadow modes) ────── /// Maximum command JWT age in seconds; `0 < x ≤ 60`. Required on every - /// issuer in enforce mode. + /// issuer in enforce and shadow modes. #[serde(default)] pub maximum_command_age_seconds: Option, /// Non-empty list of authorized `sub` values. Required on every issuer @@ -271,7 +271,7 @@ impl NipFiRelayConfig { pub fn from_env() -> Result { let mode = parse_mode()?; - if let NipFiMode::Off | NipFiMode::DenyProtected = mode { + if !mode.evaluates() { return Ok(Self { mode, registry: IssuerRegistry::new(), @@ -282,12 +282,11 @@ impl NipFiRelayConfig { }); } - // Enforce mode: all fields required. + // Enforce and Shadow: all fields required. let issuers_json = std::env::var("BUZZ_NIP_FI_ISSUERS").map_err(|_| { - ConfigError::InvalidValue( - "BUZZ_NIP_FI_MODE=enforce but BUZZ_NIP_FI_ISSUERS is not set; \ - set it to a JSON array of issuer configs" - .to_string(), + mode_requires( + mode, + "BUZZ_NIP_FI_ISSUERS is not set; set it to a JSON array of issuer configs", ) })?; if issuers_json.trim().is_empty() { @@ -327,11 +326,10 @@ impl NipFiRelayConfig { MAX_CONNECTION_LIFETIME_SECS, )? .ok_or_else(|| { - ConfigError::InvalidValue( - "BUZZ_NIP_FI_MODE=enforce but \ - BUZZ_NIP_FI_MAX_CONNECTION_LIFETIME_SECS is not set; \ - every enforce deployment must configure a positive finite value" - .to_string(), + mode_requires( + mode, + "BUZZ_NIP_FI_MAX_CONNECTION_LIFETIME_SECS is not set; \ + every enforce deployment must configure a positive finite value", ) })?; @@ -429,7 +427,7 @@ impl NipFiRelayConfig { } let configured: BTreeSet<&str> = policies.iter().map(IssuerPolicy::issuer).collect(); - let communities = parse_communities(&configured)?; + let communities = parse_communities(mode, &configured)?; // Fold each issuer's community allowlist into its policy ID // (NIP-FI.md:145), and reject an issuer no community authorizes // (NIP-FI.md:183). @@ -446,7 +444,7 @@ impl NipFiRelayConfig { } // Delegate final validation to buzz-auth startup gate. - validate_nip_fi_config(NipFiMode::Enforce, ®istry, &jwks_configs).map_err( + validate_nip_fi_config(mode, ®istry, &jwks_configs).map_err( |e: NipFiStartupError| ConfigError::InvalidValue(format!("NIP-FI config invalid: {e}")), )?; @@ -462,13 +460,16 @@ impl NipFiRelayConfig { } /// Parse and validate `BUZZ_NIP_FI_COMMUNITIES` against `registry`. -fn parse_communities(configured: &BTreeSet<&str>) -> Result { +fn parse_communities( + mode: NipFiMode, + configured: &BTreeSet<&str>, +) -> Result { let raw = std::env::var("BUZZ_NIP_FI_COMMUNITIES").unwrap_or_default(); if raw.trim().is_empty() { - return Err(ConfigError::InvalidValue( - "BUZZ_NIP_FI_MODE=enforce but BUZZ_NIP_FI_COMMUNITIES is not set; \ - set it to a JSON array of {canonical_uri, authorized_issuers}" - .to_string(), + return Err(mode_requires( + mode, + "BUZZ_NIP_FI_COMMUNITIES is not set; \ + set it to a JSON array of {canonical_uri, authorized_issuers}", )); } let entries: Vec = serde_json::from_str(&raw).map_err(|e| { @@ -494,12 +495,25 @@ fn parse_mode() -> Result { None | Some("") | Some("off") => Ok(NipFiMode::Off), Some("enforce") => Ok(NipFiMode::Enforce), Some("deny_protected") => Ok(NipFiMode::DenyProtected), + Some("shadow") => Ok(NipFiMode::Shadow), Some(other) => Err(ConfigError::InvalidValue(format!( - "BUZZ_NIP_FI_MODE must be \"enforce\", \"deny_protected\", or \"off\"; got {other:?}" + "BUZZ_NIP_FI_MODE must be \"enforce\", \"shadow\", \"deny_protected\", or \"off\"; \ + got {other:?}" ))), } } +/// A missing-setting startup error naming the configured mode. +fn mode_requires(mode: NipFiMode, missing: &str) -> ConfigError { + let value = match mode { + NipFiMode::Enforce => "enforce", + NipFiMode::Shadow => "shadow", + NipFiMode::Off => "off", + NipFiMode::DenyProtected => "deny_protected", + }; + ConfigError::InvalidValue(format!("BUZZ_NIP_FI_MODE={value} but {missing}")) +} + /// Parse an optional positive `u64` env var bounded to `[min_val, max_val]`. /// Returns `None` when the variable is absent or empty. fn parse_u64_bounded(name: &str, min_val: u64, max_val: u64) -> Result, ConfigError> { @@ -619,7 +633,7 @@ impl NipFiRelayConfig { /// Returns `true` when the relay is in `Enforce` mode. pub fn is_enforce(&self) -> bool { - matches!(self.mode, NipFiMode::Enforce) + self.mode.enforces() } } @@ -723,7 +737,9 @@ mod tests { /// [FI-TRACE-ENV-RACE] #[test] fn fixture_config_read_waits_for_fi_env_lock() { - let guard = super::NIP_FI_ENV_LOCK.lock().unwrap(); + let guard = super::NIP_FI_ENV_LOCK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); let env = EnvGuard::new(NIP_FI_VARS); std::env::set_var("BUZZ_NIP_FI_MODE", "enforce"); std::env::remove_var("BUZZ_NIP_FI_ISSUERS"); @@ -760,7 +776,9 @@ mod tests { #[test] fn off_mode_requires_no_other_config() { - let _guard = super::NIP_FI_ENV_LOCK.lock().unwrap(); + let _guard = super::NIP_FI_ENV_LOCK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); let _env = EnvGuard::new(NIP_FI_VARS); // NipFiMode::Off is the default: no issuers, no age limit. @@ -772,7 +790,9 @@ mod tests { #[test] fn deny_protected_requires_no_other_config() { - let _guard = super::NIP_FI_ENV_LOCK.lock().unwrap(); + let _guard = super::NIP_FI_ENV_LOCK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); let _env = EnvGuard::new(NIP_FI_VARS); std::env::set_var("BUZZ_NIP_FI_MODE", "deny_protected"); @@ -782,7 +802,9 @@ mod tests { #[test] fn enforce_without_issuers_fails_closed() { - let _guard = super::NIP_FI_ENV_LOCK.lock().unwrap(); + let _guard = super::NIP_FI_ENV_LOCK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); let _env = EnvGuard::new(NIP_FI_VARS); std::env::set_var("BUZZ_NIP_FI_MODE", "enforce"); @@ -817,7 +839,9 @@ mod tests { #[test] fn enforce_without_assertion_age_fails_closed() { - let _guard = super::NIP_FI_ENV_LOCK.lock().unwrap(); + let _guard = super::NIP_FI_ENV_LOCK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); let _env = EnvGuard::new(NIP_FI_VARS); std::env::set_var("BUZZ_NIP_FI_MODE", "enforce"); std::env::set_var("BUZZ_NIP_FI_MAX_CONNECTION_LIFETIME_SECS", "3600"); @@ -850,7 +874,9 @@ mod tests { #[test] fn unknown_mode_is_rejected() { - let _guard = super::NIP_FI_ENV_LOCK.lock().unwrap(); + let _guard = super::NIP_FI_ENV_LOCK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); let _env = EnvGuard::new(NIP_FI_VARS); std::env::set_var("BUZZ_NIP_FI_MODE", "permissive"); @@ -886,7 +912,9 @@ mod tests { /// passes even if the code leaks values from valid-but-wrong-typed fields. #[test] fn malformed_issuer_json_error_does_not_leak_raw_value() { - let _guard = super::NIP_FI_ENV_LOCK.lock().unwrap(); + let _guard = super::NIP_FI_ENV_LOCK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); let _env = EnvGuard::new(NIP_FI_VARS); // A sentinel that serde would echo in a type-mismatch error if not suppressed. @@ -929,7 +957,9 @@ mod tests { fn invalid_algorithm_error_does_not_leak_raw_value() { // parse_algorithm is private; we test it indirectly by passing a full // issuer config with a sentinel algorithm name. - let _guard = super::NIP_FI_ENV_LOCK.lock().unwrap(); + let _guard = super::NIP_FI_ENV_LOCK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); let _env = EnvGuard::new(NIP_FI_VARS); const SENTINEL_ALG: &str = "SENTINEL_ALGORITHM_HS256_SECRET"; @@ -964,7 +994,9 @@ mod tests { /// Policy-build rejection error must not leak the issuer URL. #[test] fn policy_build_rejection_error_does_not_leak_issuer_url() { - let _guard = super::NIP_FI_ENV_LOCK.lock().unwrap(); + let _guard = super::NIP_FI_ENV_LOCK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); let _env = EnvGuard::new(NIP_FI_VARS); const SENTINEL_ISSUER: &str = "https://sentinel-issuer-secret.example"; @@ -1128,7 +1160,9 @@ mod tests { // ── NIP-FI S4 deny witnesses ── #[test] fn config_error_does_not_expose_sensitive_principal_value() { - let _guard = super::NIP_FI_ENV_LOCK.lock().unwrap(); + let _guard = super::NIP_FI_ENV_LOCK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); let _env = EnvGuard::new(NIP_FI_VARS); // A syntactically broken JSON object that contains a sensitive sentinel @@ -1180,7 +1214,9 @@ mod tests { #[test] fn enforce_command_age_without_principals_fails_closed() { - let _guard = super::NIP_FI_ENV_LOCK.lock().unwrap(); + let _guard = super::NIP_FI_ENV_LOCK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); let _env = EnvGuard::new(NIP_FI_VARS); std::env::set_var("BUZZ_NIP_FI_MODE", "enforce"); @@ -1215,7 +1251,9 @@ mod tests { // JWKS/assertion issuer but carries no command config. Without this // rejection from_env() would succeed with an empty command_configs, // the endpoint would permanently return 503, and startup would log nothing. - let _guard = super::NIP_FI_ENV_LOCK.lock().unwrap(); + let _guard = super::NIP_FI_ENV_LOCK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); let _env = EnvGuard::new(NIP_FI_VARS); std::env::set_var("BUZZ_NIP_FI_MODE", "enforce"); @@ -1244,7 +1282,9 @@ mod tests { #[test] fn orphan_authorized_principals_without_command_age_fails_closed() { - let _guard = super::NIP_FI_ENV_LOCK.lock().unwrap(); + let _guard = super::NIP_FI_ENV_LOCK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); let _env = EnvGuard::new(NIP_FI_VARS); std::env::set_var("BUZZ_NIP_FI_MODE", "enforce"); @@ -1278,7 +1318,9 @@ mod tests { #[test] fn orphan_deny_set_capacity_without_command_age_fails_closed() { - let _guard = super::NIP_FI_ENV_LOCK.lock().unwrap(); + let _guard = super::NIP_FI_ENV_LOCK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); let _env = EnvGuard::new(NIP_FI_VARS); std::env::set_var("BUZZ_NIP_FI_MODE", "enforce"); @@ -1334,7 +1376,9 @@ mod tests { #[test] fn enforce_communities_map_hosts_to_bindings() { - let _guard = super::NIP_FI_ENV_LOCK.lock().unwrap(); + let _guard = super::NIP_FI_ENV_LOCK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); let _env = EnvGuard::new(NIP_FI_VARS); let cfg = enforce_with_communities(serde_json::json!([ community_entry("https://a.relay.test", &["https://issuer.test"]), @@ -1358,7 +1402,9 @@ mod tests { #[test] fn enforce_without_communities_fails_closed() { - let _guard = super::NIP_FI_ENV_LOCK.lock().unwrap(); + let _guard = super::NIP_FI_ENV_LOCK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); let _env = EnvGuard::new(NIP_FI_VARS); enforce_with_communities(serde_json::json!([])).expect_err("empty array"); std::env::remove_var("BUZZ_NIP_FI_COMMUNITIES"); @@ -1372,7 +1418,9 @@ mod tests { #[test] fn enforce_community_uri_must_be_https_authority_only() { - let _guard = super::NIP_FI_ENV_LOCK.lock().unwrap(); + let _guard = super::NIP_FI_ENV_LOCK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); let _env = EnvGuard::new(NIP_FI_VARS); for uri in [ "http://a.relay.test", @@ -1413,7 +1461,9 @@ mod tests { #[test] fn enforce_communities_must_have_unique_hosts() { - let _guard = super::NIP_FI_ENV_LOCK.lock().unwrap(); + let _guard = super::NIP_FI_ENV_LOCK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); let _env = EnvGuard::new(NIP_FI_VARS); let err = enforce_with_communities(serde_json::json!([ community_entry("https://a.relay.test", &["https://issuer.test"]), @@ -1428,7 +1478,9 @@ mod tests { #[test] fn enforce_community_issuers_must_be_configured() { - let _guard = super::NIP_FI_ENV_LOCK.lock().unwrap(); + let _guard = super::NIP_FI_ENV_LOCK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); let _env = EnvGuard::new(NIP_FI_VARS); let err = enforce_with_communities(serde_json::json!([community_entry( "https://a.relay.test", @@ -1460,7 +1512,9 @@ mod tests { #[test] fn enforce_issuer_in_no_community_fails_closed() { - let _guard = super::NIP_FI_ENV_LOCK.lock().unwrap(); + let _guard = super::NIP_FI_ENV_LOCK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); let _env = EnvGuard::new(NIP_FI_VARS); let err = enforce_with_communities(serde_json::json!([community_entry( "https://a.relay.test", @@ -1475,7 +1529,9 @@ mod tests { #[test] fn enforce_issuer_rejects_removed_audiences_field() { - let _guard = super::NIP_FI_ENV_LOCK.lock().unwrap(); + let _guard = super::NIP_FI_ENV_LOCK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); let _env = EnvGuard::new(NIP_FI_VARS); std::env::set_var("BUZZ_NIP_FI_MODE", "enforce"); std::env::set_var("BUZZ_NIP_FI_MAX_CONNECTION_LIFETIME_SECS", "3600"); @@ -1496,7 +1552,9 @@ mod tests { #[test] fn enforce_community_uri_accepts_canonical_authorities() { - let _guard = super::NIP_FI_ENV_LOCK.lock().unwrap(); + let _guard = super::NIP_FI_ENV_LOCK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); let _env = EnvGuard::new(NIP_FI_VARS); for uri in [ "https://a.relay.test", @@ -1513,7 +1571,9 @@ mod tests { #[test] fn enforce_empty_or_whitespace_host_resolves_to_no_community() { - let _guard = super::NIP_FI_ENV_LOCK.lock().unwrap(); + let _guard = super::NIP_FI_ENV_LOCK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); let _env = EnvGuard::new(NIP_FI_VARS); // `https://.` would normalize to an empty Host key if startup let it // through, so whichever configs boot must never match a blank Host. @@ -1542,11 +1602,100 @@ mod tests { #[test] fn deny_protected_ignores_invalid_communities() { - let _guard = super::NIP_FI_ENV_LOCK.lock().unwrap(); + let _guard = super::NIP_FI_ENV_LOCK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); let _env = EnvGuard::new(NIP_FI_VARS); std::env::set_var("BUZZ_NIP_FI_MODE", "deny_protected"); std::env::set_var("BUZZ_NIP_FI_COMMUNITIES", "not json"); let cfg = NipFiRelayConfig::from_env().expect("repair mode must still boot"); assert!(cfg.communities.resolve("a.relay.test").is_none()); } + + // Pins the startup-error formatter: the enforce text is unchanged and + // shadow names its own mode. Mutation: a hard-coded `enforce` fails the + // shadow test; any rewording fails the enforce test. + fn missing_issuers_error(mode: &str) -> String { + let _guard = super::NIP_FI_ENV_LOCK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); + let _env = EnvGuard::new(NIP_FI_VARS); + std::env::set_var("BUZZ_NIP_FI_MODE", mode); + std::env::remove_var("BUZZ_NIP_FI_ISSUERS"); + match NipFiRelayConfig::from_env() { + Err(ConfigError::InvalidValue(msg)) => msg, + other => panic!("expected InvalidValue, got {other:?}"), + } + } + + #[test] + fn enforce_missing_issuers_error_is_exact() { + assert_eq!( + missing_issuers_error("enforce"), + "BUZZ_NIP_FI_MODE=enforce but BUZZ_NIP_FI_ISSUERS is not set; \ + set it to a JSON array of issuer configs" + ); + } + + #[test] + fn shadow_missing_issuers_error_names_shadow() { + assert_eq!( + missing_issuers_error("shadow"), + "BUZZ_NIP_FI_MODE=shadow but BUZZ_NIP_FI_ISSUERS is not set; \ + set it to a JSON array of issuer configs" + ); + } + + // Pins D1: shadow loads the full enforce configuration, communities + // included, and refuses to start without them. + #[test] + fn shadow_startup_requires_full_enforce_config() { + let _guard = super::NIP_FI_ENV_LOCK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); + let _env = EnvGuard::new(NIP_FI_VARS); + let communities = serde_json::json!([community_entry( + "https://a.relay.test", + &["https://issuer.test", "https://issuer-b.test"] + )]); + enforce_with_communities(communities.clone()).expect("enforce fixture is valid"); + std::env::set_var("BUZZ_NIP_FI_MODE", "shadow"); + let cfg = NipFiRelayConfig::from_env().expect("shadow accepts the enforce config"); + assert_eq!(cfg.mode, NipFiMode::Shadow); + assert!(cfg.communities.resolve("a.relay.test").is_some()); + std::env::remove_var("BUZZ_NIP_FI_COMMUNITIES"); + let err = NipFiRelayConfig::from_env().expect_err("shadow without communities"); + assert!( + err.to_string() + .contains("BUZZ_NIP_FI_MODE=shadow but BUZZ_NIP_FI_COMMUNITIES is not set"), + "{err}" + ); + } + + // Pins the lifetime refusal operators see in each evaluating mode. + #[test] + fn missing_lifetime_error_is_exact_in_enforce_and_shadow() { + let _guard = super::NIP_FI_ENV_LOCK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); + let _env = EnvGuard::new(NIP_FI_VARS); + let communities = serde_json::json!([community_entry( + "https://a.relay.test", + &["https://issuer.test", "https://issuer-b.test"] + )]); + enforce_with_communities(communities).expect("enforce fixture is valid"); + std::env::remove_var("BUZZ_NIP_FI_MAX_CONNECTION_LIFETIME_SECS"); + for mode in ["enforce", "shadow"] { + std::env::set_var("BUZZ_NIP_FI_MODE", mode); + let err = NipFiRelayConfig::from_env().expect_err("lifetime is required"); + assert_eq!( + err.to_string(), + format!( + "invalid config: BUZZ_NIP_FI_MODE={mode} but \ + BUZZ_NIP_FI_MAX_CONNECTION_LIFETIME_SECS is not set; \ + every enforce deployment must configure a positive finite value" + ) + ); + } + } } diff --git a/crates/buzz-relay/src/nip_fi_http.rs b/crates/buzz-relay/src/nip_fi_http.rs index e28995d40b..eafb68862b 100644 --- a/crates/buzz-relay/src/nip_fi_http.rs +++ b/crates/buzz-relay/src/nip_fi_http.rs @@ -66,6 +66,7 @@ use crate::nip_fi_core::{ asserted_key_matches, evaluate_attached_assertion, http_denial, resolve_community, AssertionRejection, }; +use crate::nip_fi_shadow::Stage; // ── Deny-map seam ───────────────────────────────────────────────────────────── @@ -139,6 +140,8 @@ pub(crate) struct Nip98Proof { pubkey: PublicKey, /// Side-data threaded through from the extraction closure. pub(crate) extra: X, + /// False for a dev-mode `X-Pubkey` identity, which is never an enforce proof. + signed: bool, } impl Nip98Proof { @@ -146,7 +149,19 @@ impl Nip98Proof { /// and the media/git surfaces (which already hold a proven pubkey from a /// prior extractor) can build the token without leaking the key. pub(crate) fn new(pubkey: PublicKey, extra: X) -> Self { - Self { pubkey, extra } + Self { + pubkey, + extra, + signed: true, + } + } + + /// A dev-mode `X-Pubkey` identity: admitted where legacy allows it. + pub(crate) fn unsigned(pubkey: PublicKey, extra: X) -> Self { + Self { + signed: false, + ..Self::new(pubkey, extra) + } } } @@ -233,9 +248,10 @@ impl NipFiAdmission { /// 2. Enforce mode: reject more than one `Authorization` field (403). /// 3. Run `extract_nip98` — the caller's NIP-98 extraction closure. Returns /// `(proven_pubkey, X)` on success, or a `Response` to emit on failure. -/// Off mode returns `Ok(NipFiAdmission { proven_pubkey, assertion: None, -/// extra: X })` here; Off-mode behavior is identical to pre-NIP-FI (no -/// assertion requirement). [FI-INV-15] +/// Off and Shadow return `Ok(NipFiAdmission { proven_pubkey, assertion: +/// None, extra: X })` here, identical to pre-NIP-FI (no assertion +/// requirement); Shadow first records the verdict steps 4-7 would reach. +/// [FI-INV-15] /// 4. Enforce mode: extract `Nostr-Federated-Identity: Bearer `. /// 5. Verify assertion (signature, issuer, expiry, claims). /// 6. Assert `assertion.asserted_key == proven_pubkey`. [FI-INV-05] @@ -250,7 +266,7 @@ impl NipFiAdmission { /// - Absent `Authorization` header → `MissingEvidence` (401 "authentication required") /// - Present but malformed/invalid `Authorization` → `EvidenceRejected` (403) /// -/// In Off mode the legacy response is returned unchanged ([FI-INV-15]). +/// In Off and Shadow the legacy response is returned unchanged ([FI-INV-15]). /// [FI-TRACE-DENIAL-ORACLE] /// /// ## What the private constructor guarantees @@ -259,15 +275,16 @@ impl NipFiAdmission { /// `NipFiAdmission` value is this function. It does not force a handler to /// call this function. In Enforce, a handler that skips it and runs its own /// NIP-98 is still subject to the router's assertion guard, but a request with -/// a valid assertion passes without key pairing or a deny-map check. Off skips -/// the guard entirely; DenyProtected denies without verifying. +/// a valid assertion passes without key pairing or a deny-map check. Off and +/// Shadow pass the guard unchanged; DenyProtected denies without verifying. /// -/// ## Off-mode semantics +/// ## Off and Shadow semantics /// -/// In Off mode the NIP-98 closure is always called (step 3). In Off mode the closure +/// In Off and Shadow the NIP-98 closure is always called (step 3) and its /// result still gates entry — if NIP-98 auth is required for non-NIP-FI -/// reasons (e.g. `require_auth_token`), the closure encodes that. NIP-FI -/// layers (assertion/pairing/deny) are skipped entirely. +/// reasons (e.g. `require_auth_token`), the closure encodes that. The NIP-FI +/// layers (assertion/pairing/deny) never deny: Off skips them, Shadow +/// evaluates them only to record the would-be verdict. /// /// [FI-TRACE-AUTHORITY-UNIFORM] // Response is intentionally large (axum's design); boxing it here @@ -289,33 +306,101 @@ where // Step 1 — DenyProtected mode: unconditional 503. Checked first so no // request shape (duplicate, missing, or invalid `Authorization`) can turn // it into a 401/403, and so neither NIP-98 nor the verifier runs. - if matches!(mode, NipFiMode::DenyProtected) { + if mode.denies_unconditionally() { return Err(http_denial(DenialClass::AuthorizationUnavailable)); } - // Off mode: NIP-FI not required. Run the NIP-98 closure and propagate - // its result unchanged (legacy behavior) — no community resolution and no - // cardinality gate. [FI-INV-15 exemption] - if matches!(mode, NipFiMode::Off) { - let Nip98Proof { - pubkey: proven_pubkey, - extra, - } = extract_nip98()?; + // Off and Shadow: the NIP-98 result passes through unchanged (legacy + // behavior); shadow only records the enforce verdict. [FI-INV-15 exemption] + if !mode.restricts() { + let proof = extract_nip98(); + if mode.observes_only() { + let signed = proof.as_ref().ok().filter(|p| p.signed).map(|p| p.pubkey); + observe_shadow_verdict(headers, communities, verifier, deny_map, signed); + } + let proof = proof?; return Ok(NipFiAdmission { - proven_pubkey, + proven_pubkey: proof.pubkey, assertion: None, - extra, + extra: proof.extra, }); } - // Enforce step 1 — the Host must map to a configured community, before - // any evidence (`Authorization`, NIP-98, assertion) is examined. Explicit + let (proof, assertion) = + evaluate_enforce_steps(headers, communities, verifier, deny_map, extract_nip98).map_err( + |(stage, class)| { + let reason = match stage { + Stage::Pairing => Some("nip_fi_http_key_mismatch"), + Stage::DenySet => Some("nip_fi_http_denied_pubkey"), + _ => None, + }; + if let Some(reason) = reason { + metrics::counter!("buzz_auth_failures_total", "reason" => reason).increment(1); + } + http_denial(class) + }, + )?; + + // Step 8: admit. + Ok(NipFiAdmission { + proven_pubkey: proof.pubkey, + assertion: Some(assertion), + extra: proof.extra, + }) +} + +/// Record the verdict enforce would reach for a request whose NIP-98 step +/// proved `signed` (`None`: missing, rejected or unsigned). +fn observe_shadow_verdict( + headers: &HeaderMap, + communities: &NipFiCommunities, + verifier: Option<&dyn VerifyAssertion>, + deny_map: &D, + signed: Option, +) { + let verdict = evaluate_enforce_steps(headers, communities, verifier, deny_map, || { + signed.map(|key| Nip98Proof::new(key, ())).ok_or(()) + }); + crate::nip_fi_shadow::record("http", headers, communities, verdict.map(drop)); +} + +/// Shadow only: record the enforce verdict for a request whose NIP-98 proof +/// already failed, without any DB work. Off and enforce are untouched. +pub(crate) fn observe_failed_proof(state: &crate::state::AppState, headers: &HeaderMap) { + let nip_fi = &state.config.nip_fi; + if nip_fi.mode.observes_only() { + let verifier = state.nip_fi_verifier.as_deref(); + observe_shadow_verdict( + headers, + &nip_fi.communities, + verifier, + &NoDenyMapConfigured, + None, + ); + } +} + +/// Enforce admission steps 1–7 in normative order. A denial names the step +/// (the shadow `stage` label) and its class. +fn evaluate_enforce_steps( + headers: &HeaderMap, + communities: &NipFiCommunities, + verifier: Option<&dyn VerifyAssertion>, + deny_map: &D, + extract_nip98: F, +) -> Result<(Nip98Proof, VerifiedAssertion), crate::nip_fi_shadow::WouldDeny> +where + D: HttpDenyMap, + F: FnOnce() -> Result, E>, +{ + // Step 1 — the Host must map to a configured community, before any + // evidence (`Authorization`, NIP-98, assertion) is examined. Explicit // here because some admitted routes are exempt from the router guard. // [NIP-FI.md:266-268] - let community = resolve_community(headers, communities).map_err(http_denial)?; + let community = resolve_community(headers, communities).map_err(|c| (Stage::Community, c))?; - // Step 2 — cardinality gate: Enforce mode requires exactly one Authorization - // field per NIP-FI.md:695-700. + // Step 2 — cardinality gate: exactly one Authorization field per + // NIP-FI.md:695-700. // // Axum / hyper de-duplicates most header fields during HTTP/1.1 parsing, but // RFC 7230 permits comma-separated combining or multiple header lines; @@ -323,77 +408,44 @@ where // closes the attack where a relay-aware adversary slips a second credential // past the NIP-98 verifier. [FI-INV-15] if headers.get_all("authorization").iter().count() > 1 { - return Err(http_denial(DenialClass::EvidenceRejected)); + return Err((Stage::Cardinality, DenialClass::EvidenceRejected)); } - // Step 3: run NIP-98 extraction. - let nip98_result = extract_nip98(); - - // Enforce mode: NIP-98 closure failure MUST - // produce a NIP-FI DenialClass response, not a legacy JSON error. + // Step 3: NIP-98. Failure is a NIP-FI DenialClass, not a legacy error. // [NIP-FI.md §Admission procedure step 3; FI-TRACE-DENIAL-ORACLE] - let Nip98Proof { - pubkey: proven_pubkey, - extra, - } = nip98_result.map_err(|_legacy| { - // Determine the appropriate denial class from Authorization header presence. - // Absent header → MissingEvidence (401); present-but-invalid → EvidenceRejected (403). - // [FI-TRACE-DENIAL-ORACLE] + let proof = extract_nip98().map_err(|_legacy| { let class = if headers.contains_key("authorization") { DenialClass::EvidenceRejected } else { DenialClass::MissingEvidence }; - http_denial(class) + (Stage::Nip98, class) })?; - // Steps 4–8 — Enforce mode. - // Steps 4–5: extract and verify the assertion. let assertion = evaluate_attached_assertion(headers, community, verifier).map_err(|rejection| { if let AssertionRejection::Verifier(e) = rejection { tracing::debug!(code = e.code(), "nip-fi assertion denied at http ingress"); } - http_denial(rejection.denial_class()) + (Stage::Assertion, rejection.denial_class()) })?; // Step 6: key pairing — assertion.asserted_key MUST equal proven NIP-98 key. - // A claimless assertion (no nostr_pubkey) is also a denial. [FI-INV-05] - if !asserted_key_matches(&assertion, proven_pubkey) { - metrics::counter!( - "buzz_auth_failures_total", - "reason" => "nip_fi_http_key_mismatch" - ) - .increment(1); - tracing::debug!( - proven = %proven_pubkey.to_hex(), - "NIP-FI HTTP key pairing mismatch" - ); - // Key mismatch is a private-state denial: authorization_denied (403). - // [FI-TRACE-DENIAL-ORACLE] - return Err(http_denial(DenialClass::AuthorizationDenied)); + // A claimless assertion (no nostr_pubkey) is also a denial; key mismatch + // is a private-state denial (403). [FI-INV-05] [FI-TRACE-DENIAL-ORACLE] + if !asserted_key_matches(&assertion, proof.pubkey) { + tracing::debug!(proven = %proof.pubkey.to_hex(), "NIP-FI HTTP key pairing mismatch"); + return Err((Stage::Pairing, DenialClass::AuthorizationDenied)); } - // Step 7: deny-map check — (iss, pubkey) must not be in an active deny window. - // [FI-INV-14] [NIP-FI.md:624-627] - let issuer = assertion.identity().issuer(); - if deny_map.is_denied(issuer, &proven_pubkey, Utc::now()) { - metrics::counter!( - "buzz_auth_failures_total", - "reason" => "nip_fi_http_denied_pubkey" - ) - .increment(1); - // Denied-pubkey is a private-state denial. [FI-TRACE-DENIAL-ORACLE] - return Err(http_denial(DenialClass::AuthorizationDenied)); + // Step 7: deny-map check — (iss, pubkey) must not be in an active deny + // window. Private-state denial. [FI-INV-14] [NIP-FI.md:624-627] + if deny_map.is_denied(assertion.identity().issuer(), &proof.pubkey, Utc::now()) { + return Err((Stage::DenySet, DenialClass::AuthorizationDenied)); } - // Step 8: admit. - Ok(NipFiAdmission { - proven_pubkey, - assertion: Some(assertion), - extra, - }) + Ok((proof, assertion)) } // ── State-convenience wrapper ───────────────────────────────────────────────── @@ -1425,4 +1477,111 @@ mod tests { assert_eq!(verifier.calls(), 0); } } + + // Pins D8: in shadow, an unmapped or absent Host is admitted exactly as + // Off — the NIP-98 result passes through and no assertion is attached. + // Mutation: routing shadow through the enforce path makes this a 503. + #[test] + fn shadow_unmapped_host_admits_like_off() { + let mut unmapped = bearer_headers("Bearer a.b.c"); + unmapped.insert( + axum::http::header::HOST, + HeaderValue::from_static("other.example"), + ); + let mut absent = bearer_headers("Bearer a.b.c"); + absent.remove(axum::http::header::HOST); + let key = any_pubkey(); + for headers in [unmapped, absent] { + let admission = admit_nip_fi_http::<_, (), _>( + &headers, + || Ok(Nip98Proof::new(key, ())), + &communities(), + None, + NipFiMode::Shadow, + &AlwaysAdmitStubDenyMap, + ) + .expect("shadow admits an unmapped Host"); + assert_eq!(admission.proven_pubkey, key); + assert!(admission.assertion.is_none()); + } + } + + // Pins: shadow runs the verifier on a mapped Host yet keeps Off's + // admission — a verified assertion is never attached, and a NIP-98 + // failure is the legacy error, not a NIP-FI denial. Mutation: attaching + // the verdict's assertion fails `is_none`; skipping evaluation fails the + // call count. + // Pins: with the dev fallback allowed, shadow admits an `X-Pubkey` + // identity as Off does but records it as a NIP-98 would-deny, never an + // admit. Mutation: feeding the unsigned proof to the enforce steps + // records `admit` and calls the verifier. + #[test] + fn shadow_never_counts_x_pubkey_as_an_enforce_proof() { + let key = any_pubkey(); + let verifier = ScriptedVerifier::new(Ok(Some(key))); + let mut headers = bearer_headers("Bearer a.b.c"); + headers.insert("x-pubkey", HeaderValue::from_str(&key.to_hex()).unwrap()); + let recorder = metrics_util::debugging::DebuggingRecorder::new(); + let admission = metrics::with_local_recorder(&recorder, || { + admit_nip_fi_http( + &headers, + || { + crate::api::bridge::verify_bridge_auth_with_options( + &headers, "GET", "u", None, false, false, + ) + .map(|auth| auth.proof(())) + .map_err(axum::response::IntoResponse::into_response) + }, + &communities(), + Some(&verifier as &dyn VerifyAssertion), + NipFiMode::Shadow, + &AlwaysAdmitStubDenyMap, + ) + }) + .expect("shadow admits the dev fallback like Off"); + assert_eq!(admission.proven_pubkey, key); + let stages: Vec = recorder + .snapshotter() + .snapshot() + .into_vec() + .into_iter() + .filter_map(|(k, ..)| { + k.key() + .labels() + .find(|l| l.key() == "stage") + .map(|l| l.value().to_owned()) + }) + .collect(); + assert_eq!((stages, verifier.calls()), (vec!["nip98".to_owned()], 0)); + } + + #[test] + fn shadow_evaluates_assertion_without_changing_admission() { + let key = any_pubkey(); + let verifier = ScriptedVerifier::new(Ok(Some(key))); + let admission = admit_nip_fi_http::<_, (), _>( + &bearer_headers("Bearer a.b.c"), + || Ok(Nip98Proof::new(key, ())), + &communities(), + Some(&verifier as &dyn VerifyAssertion), + NipFiMode::Shadow, + &AlwaysAdmitStubDenyMap, + ) + .expect("shadow admits"); + assert!(admission.assertion.is_none()); + assert_eq!(verifier.calls(), 1); + + let legacy = admit_nip_fi_http::<_, (), _>( + &bearer_headers("Bearer a.b.c"), + || Err(Response::new(Body::from("legacy"))), + &communities(), + Some(&verifier as &dyn VerifyAssertion), + NipFiMode::Shadow, + &AlwaysAdmitStubDenyMap, + ); + let Err(resp) = legacy else { + panic!("NIP-98 failure must propagate") + }; + assert_eq!(body_bytes(resp), b"legacy"); + } } diff --git a/crates/buzz-relay/src/nip_fi_session.rs b/crates/buzz-relay/src/nip_fi_session.rs index 8fff0f9d46..986efc0028 100644 --- a/crates/buzz-relay/src/nip_fi_session.rs +++ b/crates/buzz-relay/src/nip_fi_session.rs @@ -31,6 +31,16 @@ pub(crate) enum NipFiWsRoute { Audio, } +impl NipFiWsRoute { + /// The shadow metrics `route` label. + pub(crate) const fn shadow_label(self) -> &'static str { + match self { + Self::Root => "ws", + Self::Audio => "audio", + } + } +} + // ── Pairing seam ────────────────────────────────────────────────────────────── /// Outcome of [`enforce_nip_fi_key_pairing`]. diff --git a/crates/buzz-relay/src/nip_fi_shadow.rs b/crates/buzz-relay/src/nip_fi_shadow.rs new file mode 100644 index 0000000000..bf42f09ff1 --- /dev/null +++ b/crates/buzz-relay/src/nip_fi_shadow.rs @@ -0,0 +1,407 @@ +//! Shadow-mode recording of what enforce would have decided. The verdict +//! never reaches admission, a connection, or a response. Labels are bounded: +//! the community is its configured URI or `unmapped`, never the raw `Host`, +//! and no token, claim, or issuer is logged. The one key logged is the NIP-98 +//! proof key, in the debug line of the key-pairing step shadow shares with +//! enforce. [FI-TRACE-PRIVACY-NONPUBLIC] +//! +//! # Metrics +//! +//! | Series | Labels | One increment per | +//! |---|---|---| +//! | `buzz_nip_fi_shadow_total` | `route` (`http`, `ws`, `audio`), `stage`, `outcome`, `community` | NIP-FI decision enforce would make: one per HTTP request that reaches admission, one per WebSocket upgrade enforce would refuse, and one per observed session: at the AUTH where enforce would admit or deny it, or `deadline` when its deadline passes first | +//! | `buzz_nip_fi_shadow_strict_proof_total` | `route` (`bridge`, `blossom`), `outcome` (`pass`, `rejected`), `community` | strict NIP-98 side check on a route whose proof is stricter in enforce (payload tag, Blossom proof); not every request runs one | +//! | `buzz_nip_fi_shadow_session_end_total` | `route` (`ws`, `audio`), `reason` (`expired`, `revoked`), `community` | admitted shadow session enforce would have ended; a session never admitted records none | +//! | `buzz_nip_fi_shadow_disconnect_total` | `route` (`admin`, `cross_pod`), `outcome` | disconnect-path event that moves a real disconnect counter in enforce: an accepted admin command, a capacity rejection, a subscriber lag, or a cross-pod capacity or poison failsafe. An admin command refused for its header, body, proof, replay, or a dependency error records nothing | +//! +//! Denominators differ. A would-deny rate is the non-`admit` share of +//! `buzz_nip_fi_shadow_total` for a route. A strict-proof rejection rate is +//! `rejected` over `pass + rejected` of its own series, never over +//! `buzz_nip_fi_shadow_total`: the side check runs once per proof, not once +//! per admission verdict. A session-end rate divides by the `admit` count of +//! `buzz_nip_fi_shadow_total` for the same `ws` or `audio` route. + +use axum::http::HeaderMap; +use buzz_auth::DenialClass; + +use crate::nip_fi_config::NipFiCommunities; + +/// Which enforce step would have denied, and with which class. +pub(crate) type WouldDeny = (Stage, DenialClass); + +/// The enforce step a would-deny stops at: the shadow `stage` label. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub(crate) enum Stage { + Community, + Cardinality, + Nip98, + Assertion, + Pairing, + DenySet, + Deadline, +} + +impl Stage { + const fn label(self) -> &'static str { + match self { + Self::Community => "community", + Self::Cardinality => "cardinality", + Self::Nip98 => "nip98", + Self::Assertion => "assertion", + Self::Pairing => "pairing", + Self::DenySet => "deny_set", + Self::Deadline => "deadline", + } + } +} + +/// Record one would-be enforce verdict for a request at `route`. +pub(crate) fn record( + route: &'static str, + headers: &HeaderMap, + communities: &NipFiCommunities, + verdict: Result<(), WouldDeny>, +) { + record_for(route, community_label(headers, communities), verdict); +} + +/// [`record`] for a caller that resolved its community label earlier. +pub(crate) fn record_for(route: &'static str, community: String, verdict: Result<(), WouldDeny>) { + let (stage, outcome) = match verdict { + Ok(()) => ("admit", "admit"), + Err((stage, class)) => (stage.label(), class_label(class)), + }; + emit(route, community, stage, outcome); +} + +/// The bounded community label for `headers`: its configured URI or `unmapped`. +pub(crate) fn community_label(headers: &HeaderMap, communities: &NipFiCommunities) -> String { + crate::nip_fi_core::resolve_community(headers, communities) + .map_or("unmapped", |binding| binding.expected_aud()) + .to_owned() +} + +tokio::task_local! { + /// Set by the router guard in shadow: whether enforce's guard would deny. + static GUARD_DENIED: bool; +} + +/// Record the router guard's would-deny, then run the request inside its +/// verdict. Enforce never lets a guard-denied request reach a handler, so +/// none of that request's handler-side records are emitted. +pub(crate) async fn observe_guard( + state: &crate::state::AppState, + headers: &HeaderMap, + verdict: Result<(), WouldDeny>, + run: F, +) -> F::Output { + let denied = verdict.is_err(); + if denied { + record("http", headers, &state.config.nip_fi.communities, verdict); + } + GUARD_DENIED.scope(denied, run).await +} + +/// The enclosing router guard's verdict; `None` outside a guarded request. +fn guard_denied() -> Option { + GUARD_DENIED.try_with(|denied| *denied).ok() +} + +fn emit(route: &'static str, community: String, stage: &'static str, outcome: &'static str) { + if guard_denied() == Some(true) { + return; + } + metrics::counter!( + "buzz_nip_fi_shadow_total", + "route" => route, + "stage" => stage, + "outcome" => outcome, + "community" => community.clone() + ) + .increment(1); + // The counter is the record; only a would-deny earns a (debug) line. + if stage != "admit" { + tracing::debug!(route, stage, outcome, community, "nip-fi shadow would-deny"); + } +} + +const fn class_label(class: DenialClass) -> &'static str { + match class { + DenialClass::MissingEvidence => "missing", + DenialClass::EvidenceRejected => "rejected", + DenialClass::AuthorizationDenied => "denied", + DenialClass::AuthorizationUnavailable => "unavailable", + } +} + +/// In shadow mode only, record whether `strict` (a pure check) would reject, +/// on its own counter: a pass is one proof, not an admission verdict. +pub(crate) fn observe_strict_proof( + state: &crate::state::AppState, + headers: &HeaderMap, + route: &'static str, + strict: impl FnOnce() -> Result<(), E>, +) { + let nip_fi = &state.config.nip_fi; + if nip_fi.mode.observes_only() && guard_denied() != Some(true) { + let outcome = strict().map_or(class_label(DenialClass::EvidenceRejected), |()| "pass"); + metrics::counter!( + "buzz_nip_fi_shadow_strict_proof_total", + "route" => route, + "outcome" => outcome, + "community" => community_label(headers, &nip_fi.communities) + ) + .increment(1); + } +} + +/// In shadow mode, record the verdict enforce reaches for a request whose +/// Host bound no tenant: `community` when the Host maps to no configured +/// community; an admit when enforce's guard passed it, since the handler's +/// own rejection is no NIP-FI denial; nothing outside a guarded route. +pub(crate) fn observe_unbound(state: &crate::state::AppState, headers: &HeaderMap) { + let nip_fi = &state.config.nip_fi; + if nip_fi.mode.observes_only() { + let verdict = match crate::nip_fi_core::resolve_community(headers, &nip_fi.communities) { + Err(class) => Err((Stage::Community, class)), + Ok(_) if guard_denied() == Some(false) => Ok(()), + Ok(_) => return, + }; + record("http", headers, &nip_fi.communities, verdict); + } +} + +#[cfg(test)] +mod tests { + use std::sync::atomic::{AtomicUsize, Ordering}; + use std::sync::Arc; + + use axum::http::{header::HOST, HeaderValue}; + use buzz_auth::NipFiMode; + use metrics_util::debugging::{DebugValue, DebuggingRecorder}; + + use crate::nip_fi_core::test_support::{communities, TEST_COMMUNITY_URI, TEST_HOST}; + + /// `(community, stage, outcome)` labels of every shadow counter increment. + fn shadow_counts(recorder: &DebuggingRecorder) -> Vec<(String, String, String)> { + counts_of(recorder, "buzz_nip_fi_shadow_total") + } + + /// `(community, stage, outcome)` of every increment of `metric`; a + /// missing label reads `-`. + fn counts_of(recorder: &DebuggingRecorder, metric: &str) -> Vec<(String, String, String)> { + let mut out = Vec::new(); + for (key, _, _, value) in recorder.snapshotter().snapshot().into_vec() { + let key = key.key(); + if key.name() != metric { + continue; + } + let label = |name| { + key.labels() + .find(|l| l.key() == name) + .map_or("-", |l| l.value()) + .to_owned() + }; + let DebugValue::Counter(n) = value else { + unreachable!() + }; + for _ in 0..n { + out.push((label("community"), label("stage"), label("outcome"))); + } + } + out.sort(); + out + } + + fn with_host(host: Option<&'static str>) -> axum::http::HeaderMap { + let mut headers = axum::http::HeaderMap::new(); + if let Some(host) = host { + headers.insert(HOST, HeaderValue::from_static(host)); + } + headers + } + + // Mutation: labelling by the raw Host leaks `attacker.example`. + #[test] + fn community_label_is_configured_uri_or_unmapped() { + let recorder = DebuggingRecorder::new(); + metrics::with_local_recorder(&recorder, || { + for host in [Some(TEST_HOST), Some("attacker.example"), None] { + super::record("http", &with_host(host), &communities(), Ok(())); + } + }); + let labels: Vec = shadow_counts(&recorder) + .into_iter() + .map(|(c, _, _)| c) + .collect(); + assert_eq!(labels, [TEST_COMMUNITY_URI, "unmapped", "unmapped"]); + } + + struct CountingReplayGuard(AtomicUsize); + + impl buzz_auth::Nip98ReplayGuard for CountingReplayGuard { + fn try_mark_in_scope<'a>( + &'a self, + _scope: &'a str, + _event_id: &'a nostr::EventId, + _ttl_secs: u64, + ) -> std::pin::Pin< + Box> + Send + 'a>, + > { + self.0.fetch_add(1, Ordering::SeqCst); + Box::pin(async { Ok(true) }) + } + } + + // A NIP-98 event without a `payload` tag passes lax checks but fails the + // strict one. Mutations: running the side check outside shadow, + // claiming the event in the replay guard, or recording either outcome on + // the admission counter → RED. + #[tokio::test(flavor = "current_thread")] + async fn strict_proof_side_check_records_only_in_shadow_and_spares_replay() { + const STRICT: &str = "buzz_nip_fi_shadow_strict_proof_total"; + use base64::Engine as _; + let url = "https://relay.example/api/bridge"; + let event = nostr::EventBuilder::new(nostr::Kind::HttpAuth, "") + .tags([ + nostr::Tag::parse(["u", url]).unwrap(), + nostr::Tag::parse(["method", "POST"]).unwrap(), + ]) + .sign_with_keys(&nostr::Keys::generate()) + .unwrap(); + let mut headers = with_host(Some(TEST_HOST)); + let auth = base64::engine::general_purpose::STANDARD + .encode(serde_json::to_string(&event).unwrap()); + headers.insert( + axum::http::header::AUTHORIZATION, + format!("Nostr {auth}").parse().unwrap(), + ); + + let base = crate::state::tests::test_state().await; + for (mode, expected) in [ + (NipFiMode::Off, 0), + (NipFiMode::Enforce, 0), + (NipFiMode::Shadow, 1), + ] { + let guard = Arc::new(CountingReplayGuard(AtomicUsize::new(0))); + let mut state = (*base).clone(); + let mut config = (*state.config).clone(); + config.nip_fi.mode = mode; + config.nip_fi.communities = communities(); + state.config = Arc::new(config); + state.nip98_replay = guard.clone(); + let recorder = DebuggingRecorder::new(); + metrics::with_local_recorder(&recorder, || { + super::observe_strict_proof(&state, &headers, "bridge", || { + crate::api::bridge::verify_bridge_auth_with_options( + &headers, + "POST", + url, + Some(b"{}"), + true, + true, + ) + .map(drop) + }); + }); + let counts = counts_of(&recorder, STRICT); + assert_eq!(counts.len(), expected, "{mode:?}"); + assert!( + counts.iter().all(|(.., outcome)| outcome == "rejected"), + "{mode:?}" + ); + assert!(shadow_counts(&recorder).is_empty(), "{mode:?}"); + assert_eq!(guard.0.load(Ordering::SeqCst), 0, "{mode:?}"); + if mode.observes_only() { + // Both outcomes stay on the proof counter, never an admit + // that could be summed with whole-request verdicts. + let recorder = DebuggingRecorder::new(); + metrics::with_local_recorder(&recorder, || { + super::observe_strict_proof(&state, &headers, "bridge", || Ok::<_, ()>(())); + super::observe_strict_proof(&state, &headers, "bridge", || Err(())); + }); + let label = |outcome: &str| { + ( + "https://relay.example".to_owned(), + "-".to_owned(), + outcome.to_owned(), + ) + }; + let counts = counts_of(&recorder, STRICT); + assert_eq!(counts, [label("pass"), label("rejected")]); + assert!(shadow_counts(&recorder).is_empty()); + } + } + } + + /// Mode semantics live in `NipFiMode`'s predicates. Naming a variant in + /// production code anywhere else (`==`, `matches!`, `if let`, or-patterns, + /// match arms) is how shadow silently inherits Off or Enforce behavior. + #[test] + fn nip_fi_mode_is_inspected_only_through_predicates() { + const ALLOWED: [&str; 2] = [ + "buzz-auth/src/nip_fi/startup/mod.rs", + "buzz-relay/src/nip_fi_config.rs", + ]; + let crates = std::path::Path::new(concat!(env!("CARGO_MANIFEST_DIR"), "/..")); + let mut offenders = Vec::new(); + let mut dirs: Vec<_> = std::fs::read_dir(crates) + .expect("read crates dir") + .map(|e| e.expect("crates entry").path().join("src")) + .filter(|src| src.is_dir()) + .collect(); + while let Some(dir) = dirs.pop() { + for entry in std::fs::read_dir(&dir).expect("read source dir") { + let path = entry.expect("source entry").path(); + if path.is_dir() { + dirs.push(path); + continue; + } + let name = path + .file_name() + .and_then(|n| n.to_str()) + .unwrap_or_default(); + if !name.ends_with(".rs") || name == "tests.rs" || name.ends_with("_tests.rs") { + continue; + } + let shown = path + .strip_prefix(crates) + .unwrap() + .to_string_lossy() + .into_owned(); + if ALLOWED.contains(&shown.as_str()) { + continue; + } + let text = std::fs::read_to_string(&path).expect("read source file"); + for (i, line) in production_lines(&text).enumerate() { + let code = line.split("//").next().unwrap_or_default(); + if ["Off", "Shadow", "Enforce", "DenyProtected"] + .iter() + .any(|v| code.contains(&format!("NipFiMode::{v}"))) + { + offenders.push(format!("{shown}:{}", i + 1)); + } + } + } + } + assert!( + offenders.is_empty(), + "raw NipFiMode variants: {offenders:?}" + ); + } + + /// Lines before the first inline `#[cfg(test)] mod x {`; an out-of-line + /// `#[cfg(test)] mod x;` declaration does not end production code. + fn production_lines(text: &str) -> impl Iterator { + let mut lines = text.lines().peekable(); + std::iter::from_fn(move || { + let line = lines.next()?; + let inline_test_mod = line.trim() == "#[cfg(test)]" + && lines.peek().is_some_and(|n| { + let n = n.trim_start().trim_start_matches("pub(crate) "); + n.starts_with("mod ") && n.trim_end().ends_with('{') + }); + (!inline_test_mod).then_some(line) // ends the iterator + }) + } +} diff --git a/crates/buzz-relay/src/nip_fi_shadow_session.rs b/crates/buzz-relay/src/nip_fi_shadow_session.rs new file mode 100644 index 0000000000..e841737f82 --- /dev/null +++ b/crates/buzz-relay/src/nip_fi_shadow_session.rs @@ -0,0 +1,637 @@ +//! Shadow-mode observation of a NIP-FI WebSocket session whose upgrade passed. +//! The assertion lives only here: never on the connection, the issuer-scoped +//! identity registries, the admission gate, a terminal frame or cancellation. +//! A session records at most one admission verdict: a pairing or deny-set +//! would-deny where enforce checks them, the admit once the connection is +//! actually admitted, or `deadline` when enforce's deadline passes first. An +//! admitted session then records at most one end: `expired` at the deadline, +//! or `revoked` when a deny for its key arrives once it registered. Every +//! transition runs under one session lock, and a session whose socket was +//! cancelled, or whose AUTH ended unadmitted for a non-NIP-FI reason, retires +//! without a record. + +use std::collections::HashMap; +use std::sync::atomic::{AtomicU64, Ordering}; +use std::sync::{Arc, Mutex, OnceLock, PoisonError, Weak}; + +use axum::http::HeaderMap; +use buzz_auth::{DenialClass, VerifiedAssertion}; +use nostr::PublicKey; +use tokio_util::sync::CancellationToken; + +use crate::nip_fi_shadow::{Stage, WouldDeny}; +use crate::state::AppState; + +#[derive(Clone, Copy)] +enum Phase { + /// `revoked`: a deny for the key arrived after registration, which + /// enforce's registered-session scan acts on whatever the phase. + Pending { + revoked: bool, + }, + Admitted, + Ended, +} + +/// Sessions that reached the deny-set check, matched by a shadow disconnect. +/// Separate from every registry enforce's close scan reads. +#[derive(Default)] +pub(crate) struct ShadowSessions { + next_id: AtomicU64, + registered: Mutex>>, +} + +impl ShadowSessions { + fn registered(&self) -> std::sync::MutexGuard<'_, HashMap>> { + self.registered + .lock() + .unwrap_or_else(PoisonError::into_inner) + } + + /// Records one `revoked` end for each registered session of `pubkey` + /// under `issuer`, at once if admitted or at its admit if still pending; + /// a session already ended records nothing, so re-sent denies are + /// idempotent. + pub(crate) fn would_close(&self, issuer: &str, pubkey: &[u8]) { + let live: Vec<_> = self + .registered() + .values() + .filter_map(Weak::upgrade) + .collect(); + for session in live { + let key = session.assertion.asserted_key().map(|k| k.to_bytes()); + if session.assertion.identity().issuer() == issuer + && key.as_ref().map(|k| &k[..]) == Some(pubkey) + { + session.revoke(); + } + } + } +} + +pub(crate) struct ShadowSession { + id: u64, + route: &'static str, + community: String, + assertion: VerifiedAssertion, + phase: Mutex, + /// The socket's cancellation, read only to stop recording. + cancel: OnceLock, + sessions: Arc, + expiry: OnceLock, +} + +impl ShadowSession { + /// Starts observing at the upgrade instant, arming a record-only timer at + /// the deadline enforce would apply; dropping the session stops it. + pub(crate) fn start( + state: &AppState, + route: &'static str, + headers: &HeaderMap, + assertion: VerifiedAssertion, + connection_time: chrono::DateTime, + ) -> Arc { + let nip_fi = &state.config.nip_fi; + let deadline = crate::connection::compute_session_deadline( + &assertion, + connection_time, + nip_fi.max_connection_lifetime(), + ); + let sessions = Arc::clone(&state.nip_fi_shadow_sessions); + let session = Arc::new(Self { + id: sessions.next_id.fetch_add(1, Ordering::Relaxed), + route, + community: crate::nip_fi_shadow::community_label(headers, &nip_fi.communities), + assertion, + phase: Mutex::new(Phase::Pending { revoked: false }), + cancel: OnceLock::new(), + sessions, + expiry: OnceLock::new(), + }); + let weak = Arc::downgrade(&session); + let timer = tokio::spawn(async move { + let remaining = (deadline - chrono::Utc::now()).to_std().unwrap_or_default(); + tokio::time::sleep(remaining).await; + if let Some(session) = weak.upgrade() { + session.expire(); + } + }); + let _ = session.expiry.set(timer.abort_handle()); + session + } + + /// Stops recording once `cancel` fires, checked inside every transition. + pub(crate) fn fence(&self, cancel: CancellationToken) { + let _ = self.cancel.set(cancel); + } + + /// At enforce's key-pairing point: records the would-deny when the + /// NIP-42 key is not the asserted one, or the assertion has no key. + pub(crate) fn observe_pairing(&self, pubkey: PublicKey) { + if self.assertion.asserted_key() != Some(pubkey) { + self.decide(Err((Stage::Pairing, DenialClass::AuthorizationDenied))); + } + } + + /// At enforce's post-registration deny-set check: registers the session, + /// so a later deny marks it, then records the deny-set would-deny. + pub(crate) fn observe_deny_set(self: &Arc, state: &AppState) { + self.sessions + .registered() + .insert(self.id, Arc::downgrade(self)); + if deny_listed(state, &self.assertion) { + self.decide(Err((Stage::DenySet, DenialClass::AuthorizationDenied))); + } + } + + /// Where the connection is actually admitted: records the admit, then + /// `revoked` if a deny marked the session since its deny-set check. + pub(crate) fn admit(&self) { + self.decide(Ok(())); + } + + /// Enforce's deadline: the admission verdict while pending, else the end. + /// A pending session a deny already marked records nothing, as enforce's + /// scan closed it first. + pub(crate) fn expire(&self) { + self.transition(|phase| match *phase { + Phase::Pending { revoked: false } => { + *phase = Phase::Ended; + let verdict = Err((Stage::Deadline, DenialClass::EvidenceRejected)); + crate::nip_fi_shadow::record_for(self.route, self.community.clone(), verdict); + } + Phase::Admitted => self.end(phase, "expired"), + _ => {} + }); + } + + /// Retires a pending session without a record: its connection was + /// refused for a non-NIP-FI reason, so enforce would record nothing. + pub(crate) fn retire_pending(&self) { + self.transition(|phase| { + if let Phase::Pending { .. } = phase { + *phase = Phase::Ended; + } + }); + } + + fn revoke(&self) { + self.transition(|phase| match phase { + Phase::Pending { revoked } => *revoked = true, + Phase::Admitted => self.end(phase, "revoked"), + Phase::Ended => {} + }); + } + + fn decide(&self, verdict: Result<(), WouldDeny>) { + self.transition(|phase| { + let Phase::Pending { revoked } = *phase else { + return; + }; + let admitted = verdict.is_ok(); + crate::nip_fi_shadow::record_for(self.route, self.community.clone(), verdict); + *phase = if admitted { + Phase::Admitted + } else { + Phase::Ended + }; + if admitted && revoked { + self.end(phase, "revoked"); + } + }); + } + + /// Ends an admitted session. + fn end(&self, phase: &mut Phase, reason: &'static str) { + *phase = Phase::Ended; + metrics::counter!( + "buzz_nip_fi_shadow_session_end_total", + "route" => self.route, + "reason" => reason, + "community" => self.community.clone() + ) + .increment(1); + } + + /// Runs one transition under the session lock, after retiring a session + /// whose socket is already cancelled. + fn transition(&self, step: impl FnOnce(&mut Phase)) { + let mut phase = self.phase.lock().unwrap_or_else(PoisonError::into_inner); + if self + .cancel + .get() + .is_some_and(CancellationToken::is_cancelled) + { + *phase = Phase::Ended; + } + step(&mut phase); + } +} + +/// One AUTH attempt, audio's through its join commit: refused, or dropped, +/// while the session is still pending, it ended unadmitted for a non-NIP-FI +/// reason (NIP-42 failure, ordinary policy refusal), so the session retires +/// without a record. +pub(crate) struct AuthAttempt<'a>(pub(crate) Option<&'a Arc>); + +impl AuthAttempt<'_> { + /// Retires a pending session where the refusal is decided, before the + /// refusal's frames and cleanup are awaited. + pub(crate) fn refused(&self) { + if let Some(session) = self.0 { + session.retire_pending(); + } + } +} + +impl Drop for AuthAttempt<'_> { + fn drop(&mut self) { + self.refused(); + } +} + +impl Drop for ShadowSession { + fn drop(&mut self) { + if let Some(timer) = self.expiry.get() { + timer.abort(); + } + self.sessions.registered().remove(&self.id); + } +} + +/// Root's upgrade-time deny-map bounce, observed: records the would-deny and +/// returns `true` when enforce would have refused the upgrade here. +pub(crate) fn upgrade_denied( + state: &AppState, + headers: &HeaderMap, + assertion: &VerifiedAssertion, +) -> bool { + let denied = deny_listed(state, assertion); + if denied { + let verdict = Err((Stage::DenySet, DenialClass::AuthorizationDenied)); + crate::nip_fi_shadow::record("ws", headers, &state.config.nip_fi.communities, verdict); + } + denied +} + +fn deny_listed(state: &AppState, assertion: &VerifiedAssertion) -> bool { + let issuer = assertion.identity().issuer(); + assertion + .asserted_key() + .zip(state.nip_fi_deny_map.as_deref()) + .is_some_and(|(key, map)| map.is_denied(issuer, &key, chrono::Utc::now())) +} + +#[cfg(test)] +pub(crate) mod tests { + use std::future::Future; + + use buzz_auth::{IssuerCapacity, NipFiDenyMap}; + use chrono::{Duration, Utc}; + use metrics_util::debugging::{DebugValue, DebuggingRecorder}; + use nostr::Keys; + + use super::*; + + /// Runs `body` on a current-thread runtime, returning every shadow + /// admission record as `stage/outcome` and every session end as + /// `end/reason`, sorted. + pub(crate) fn shadow_records(body: impl Future) -> Vec { + collect(body, false, false) + } + + /// [`shadow_records`], each prefixed with its `route` label. + pub(crate) fn shadow_records_by_route(body: impl Future) -> Vec { + collect(body, true, false) + } + + /// [`shadow_records`] on paused Tokio time, which `tokio::time::advance` + /// moves; a timer reads its delay from the wall clock when first polled. + fn shadow_records_paused(body: impl Future) -> Vec { + collect(body, false, true) + } + + fn collect(body: impl Future, by_route: bool, paused: bool) -> Vec { + let recorder = DebuggingRecorder::new(); + let snapshotter = recorder.snapshotter(); + let rt = tokio::runtime::Builder::new_current_thread() + .enable_all() + .start_paused(paused) + .build() + .unwrap(); + metrics::with_local_recorder(&recorder, || rt.block_on(body)); + let mut out = Vec::new(); + for (key, _, _, value) in snapshotter.snapshot().into_vec() { + let key = key.key(); + let label = |name| { + key.labels() + .find(|l| l.key() == name) + .map(|l| l.value().to_owned()) + }; + let record = match key.name() { + "buzz_nip_fi_shadow_total" => { + format!("{}/{}", label("stage").unwrap(), label("outcome").unwrap()) + } + "buzz_nip_fi_shadow_session_end_total" => { + format!("end/{}", label("reason").unwrap()) + } + _ => continue, + }; + let record = match (by_route, label("route")) { + (true, Some(route)) => format!("{route} {record}"), + _ => record, + }; + let DebugValue::Counter(n) = value else { + unreachable!() + }; + out.extend(std::iter::repeat_n(record, n as usize)); + } + out.sort(); + out + } + + /// A shadow test state whose deny map lists `denied` under the + /// `for_test` issuer. Its database is unreachable with a 100 ms acquire + /// timeout, so a path that reaches Postgres fails fast whether or not + /// one runs locally. + pub(crate) async fn shadow_state(denied: Option) -> AppState { + let mut state = + (*crate::state::tests::test_state_with_database_url("postgres://127.0.0.1:1/none") + .await) + .clone(); + let capacity = vec![IssuerCapacity { + issuer: "test-issuer".to_owned(), + capacity: 8, + }]; + let map = NipFiDenyMap::new(8, capacity); + if let Some(key) = denied { + map.merge_cross_pod_deny( + "test-issuer", + &key, + Utc::now() + Duration::hours(1), + Utc::now(), + ); + } + state.nip_fi_deny_map = Some(Arc::new(map)); + state.nip_fi_shadow_sessions = Arc::default(); + state + } + + pub(crate) fn session( + state: &AppState, + key: Option, + lifetime: Duration, + ) -> Arc { + let assertion = VerifiedAssertion::for_test(key, vec![Utc::now() + lifetime]); + ShadowSession::start(state, "ws", &HeaderMap::new(), assertion, Utc::now()) + } + + fn auth(session: &Arc, state: &AppState, pubkey: PublicKey) { + session.observe_pairing(pubkey); + session.observe_deny_set(state); + session.admit(); + } + + /// Lets every spawned timer read its deadline, then moves paused time + /// past `ms` milliseconds. + async fn advance_ms(ms: u64) { + tokio::task::yield_now().await; + tokio::time::advance(std::time::Duration::from_millis(ms)).await; + tokio::task::yield_now().await; + } + + fn revoke(state: &AppState, key: &PublicKey) { + state + .nip_fi_shadow_sessions + .would_close("test-issuer", &key.to_bytes()); + } + + // Mutation: recording on a repeated AUTH, or letting the re-sent deny + // end the session again, adds a record. + #[test] + fn admitted_session_records_one_admit_and_one_revoked_end() { + let records = shadow_records(async { + let state = shadow_state(None).await; + let key = Keys::generate().public_key(); + let s = session(&state, Some(key), Duration::hours(1)); + auth(&s, &state, key); + auth(&s, &state, key); + revoke(&state, &Keys::generate().public_key()); + state + .nip_fi_shadow_sessions + .would_close("other-issuer", &key.to_bytes()); + revoke(&state, &key); + revoke(&state, &key); + }); + assert_eq!(records, ["admit/admit", "end/revoked"]); + } + + // Mismatched and claimless keys are both pairing denials; nothing after a + // denial records, because enforce would have closed the connection. + #[test] + fn pairing_denial_is_the_only_record() { + let records = shadow_records(async { + let state = shadow_state(None).await; + let key = Keys::generate().public_key(); + for asserted in [Some(Keys::generate().public_key()), None] { + let s = session(&state, asserted, Duration::hours(1)); + auth(&s, &state, key); + auth(&s, &state, asserted.unwrap_or(key)); + revoke(&state, &key); + } + }); + assert_eq!(records, ["pairing/denied", "pairing/denied"]); + } + + // Mutation: checking the deny map before registering, or not at all, + // records an admit. + #[test] + fn deny_listed_key_records_deny_set_and_no_revoked_end() { + let key = Keys::generate().public_key(); + let records = shadow_records(async move { + let state = shadow_state(Some(key)).await; + let s = session(&state, Some(key), Duration::hours(1)); + auth(&s, &state, key); + revoke(&state, &key); + }); + assert_eq!(records, ["deny_set/denied"]); + } + + /// Delivers a real cross-pod disconnect for `key` to a shadow pod. + pub(crate) fn deliver(state: &AppState, key: &PublicKey) -> buzz_auth::CrossPodMergeResult { + let msg = buzz_pubsub::NipFiDisconnect { + issuer: "test-issuer".to_owned(), + pubkey_bytes: key.to_bytes().to_vec(), + until_unix: (Utc::now() + Duration::minutes(5)).timestamp(), + until_unix_nanos: 0, + }; + match crate::api::nip_fi::apply_nip_fi_disconnect(state, &msg, Utc::now()) { + crate::api::nip_fi::NipFiDisconnectApplyResult::Applied(merged) => merged, + other => panic!("disconnect not applied: {other:?}"), + } + } + + /// A shadow pod whose deny map holds `capacity` entries for the + /// `for_test` issuer. + pub(crate) async fn pod(capacity: usize) -> AppState { + let mut state = shadow_state(None).await; + let mut config = (*state.config).clone(); + config.nip_fi.mode = buzz_auth::NipFiMode::Shadow; + use buzz_auth::{FreshnessClass, IssuerPolicy, JwksSourceContract, TokenClass}; + let jwks = JwksSourceContract::new("https://idp.test/jwks.json".into(), 300, 86400); + let policy = IssuerPolicy::new( + "test-issuer".to_owned(), + vec!["https://relay.test".to_owned()], + TokenClass::DedicatedNipFi, + FreshnessClass::OfflineJwt, + vec![jsonwebtoken::Algorithm::ES256], + 30, + 3600, + None, + jwks.unwrap(), + ); + config.nip_fi.registry.insert(policy.unwrap()); + state.config = Arc::new(config); + let capacity = vec![IssuerCapacity { + issuer: "test-issuer".to_owned(), + capacity, + }]; + state.nip_fi_deny_map = Some(Arc::new(NipFiDenyMap::new(8, capacity))); + state + } + + // A deny landing between the clean deny-set check and the admit, + // whether merged or refused for capacity, ends the admitted session + // `revoked`, as enforce's registered-session scan closes it in any phase; + // expiry contending on either side of the admit cannot take that end. + // Mutation: dropping the revocation mark, or letting a marked pending + // session expire, loses the `revoked` end. + #[test] + fn deny_between_map_read_and_admit_records_admit_then_revoked() { + let records = shadow_records(async { + for capacity in [8, 1] { + let state = pod(capacity).await; + let key = Keys::generate().public_key(); + if capacity == 1 { + deliver(&state, &Keys::generate().public_key()); + } + let s = session(&state, Some(key), Duration::hours(1)); + s.observe_pairing(key); + s.observe_deny_set(&state); + let merged = deliver(&state, &key); + assert_eq!( + merged == buzz_auth::CrossPodMergeResult::Merged, + capacity == 8 + ); + s.expire(); + s.admit(); + s.expire(); + } + }); + let both = ["admit/admit", "admit/admit", "end/revoked", "end/revoked"]; + assert_eq!(records, both); + } + + // Expiry before AUTH is the session's admission verdict, so AUTH records + // nothing; expiry after admission ends it once, so a later deny records + // nothing. + #[test] + fn deadline_records_one_expired_end_before_or_after_admission() { + let records = shadow_records_paused(async { + let state = shadow_state(None).await; + let key = Keys::generate().public_key(); + let early = session(&state, Some(key), Duration::seconds(10)); + let late = session(&state, Some(key), Duration::seconds(30)); + advance_ms(20_000).await; + auth(&early, &state, key); + auth(&late, &state, key); + advance_ms(20_000).await; + revoke(&state, &key); + }); + assert_eq!(records, ["admit/admit", "deadline/rejected", "end/expired"]); + } + + // The deadline passing while the socket is still bootstrapping, before + // AUTH, records one `deadline` verdict and no end, and the observer + // cancels nothing. + #[test] + fn deadline_during_bootstrap_records_one_verdict_and_cancels_nothing() { + let records = shadow_records_paused(async { + let state = shadow_state(None).await; + let key = Keys::generate().public_key(); + let s = session(&state, Some(key), Duration::seconds(10)); + let cancel = tokio_util::sync::CancellationToken::new(); + let control = crate::state::CommunityConnectionControl::new(cancel.clone()); + control.attach_nip_fi_shadow(Some(Arc::clone(&s))); + advance_ms(20_000).await; + assert!(!cancel.is_cancelled(), "shadow never cancels the socket"); + auth(&s, &state, key); + revoke(&state, &key); + }); + assert_eq!(records, ["deadline/rejected"]); + } + + // A connection closed before its deadline records no end. Mutation: not + // unregistering on drop leaves a stale registry entry. + #[test] + fn dropped_session_records_no_end() { + let records = shadow_records_paused(async { + let state = shadow_state(None).await; + let key = Keys::generate().public_key(); + let s = session(&state, Some(key), Duration::seconds(10)); + auth(&s, &state, key); + let weak = Arc::downgrade(&s); + drop(s); + assert!(weak.upgrade().is_none()); + assert!(state.nip_fi_shadow_sessions.registered().is_empty()); + advance_ms(20_000).await; + }); + assert_eq!(records, ["admit/admit"]); + } + + // Once the socket is cancelled, no transition records, even while the + // connection still holds references and before anything else runs: an + // admitted session takes no `revoked` end from an immediate real deny, + // and a pending one records no verdict or deadline. Mutation: skipping + // the cancellation check in `transition` records both. + #[test] + fn cancelled_socket_records_nothing_without_yielding() { + let records = shadow_records(async { + let state = pod(8).await; + let key = Keys::generate().public_key(); + let watch = |s: &Arc| { + let cancel = tokio_util::sync::CancellationToken::new(); + let control = crate::state::CommunityConnectionControl::new(cancel.clone()); + control.attach_nip_fi_shadow(Some(Arc::clone(s))); + (cancel, control) + }; + let admitted = session(&state, Some(key), Duration::hours(1)); + let (cancel, _control) = watch(&admitted); + auth(&admitted, &state, key); + cancel.cancel(); + deliver(&state, &key); + + let pending = session(&state, Some(key), Duration::hours(1)); + let (cancel, _control) = watch(&pending); + cancel.cancel(); + pending.observe_pairing(Keys::generate().public_key()); + pending.expire(); + }); + assert_eq!(records, ["admit/admit"]); + } + + #[test] + fn upgrade_deny_map_hit_records_and_reports_denied() { + let key = Keys::generate().public_key(); + let records = shadow_records(async move { + let state = shadow_state(Some(key)).await; + let listed = + VerifiedAssertion::for_test(Some(key), vec![Utc::now() + Duration::hours(1)]); + let clean = + VerifiedAssertion::for_test(Some(Keys::generate().public_key()), vec![Utc::now()]); + assert!(upgrade_denied(&state, &HeaderMap::new(), &listed)); + assert!(!upgrade_denied(&state, &HeaderMap::new(), &clean)); + }); + assert_eq!(records, ["deny_set/denied"]); + } +} diff --git a/crates/buzz-relay/src/nip_fi_test_hooks.rs b/crates/buzz-relay/src/nip_fi_test_hooks.rs index cb8029c6bb..55d8e5b0e8 100644 --- a/crates/buzz-relay/src/nip_fi_test_hooks.rs +++ b/crates/buzz-relay/src/nip_fi_test_hooks.rs @@ -195,8 +195,19 @@ make_hook!(audio_auth_verify_hook, before_auth_verify); // `check_cancel!` fence. Arms cancel here → proves the cleanup branch // (`room.remove_peer` + `cleanup_if_empty`) runs before the handler returns. make_hook!(audio_membership_check_hook, before_membership_check); +// `before_git_membership`: fires in the Git transport after NIP-FI admission, +// immediately before the relay membership lookup. Closing the pool here makes +// that lookup fail with the tenant already bound. +make_hook!(git_membership_hook, before_git_membership); make_hook!(audio_membership_lock_hook, before_membership_lock); make_hook!(audio_participant_commit_hook, before_participant_commit); +// `before_audio_refusal_frames`: fires in `audio/handler.rs` once a NIP-42 +// failure or relay-membership refusal is decided, before its frames are +// written. `before_join_refusal_rollback`: fires in `commit_participant_join` +// once the early archived check refuses, before `tx.rollback()`. A test holds +// either window to prove the shadow session already retired. +make_hook!(audio_refusal_frames_hook, before_audio_refusal_frames); +make_hook!(join_refusal_rollback_hook, before_join_refusal_rollback); make_hook!(audio_participant_fanout_hook, after_participant_fanout); make_hook!(audio_add_peer_hook, after_add_peer); // `before_lease_permit`: fires in `handle_active_audio_connection` immediately diff --git a/crates/buzz-relay/src/nip_fi_upgrade.rs b/crates/buzz-relay/src/nip_fi_upgrade.rs index 534bd787a6..7edabfa99c 100644 --- a/crates/buzz-relay/src/nip_fi_upgrade.rs +++ b/crates/buzz-relay/src/nip_fi_upgrade.rs @@ -24,13 +24,30 @@ use crate::nip_fi_core::{ pub(crate) enum NipFiUpgradeOutcome { /// Assertion validated successfully. Carry the result into the connection. Admitted(VerifiedAssertion), - /// Enforcement is off — no assertion required. + /// Off, or Shadow without a passing assertion — none is required. NotRequired, + /// Shadow: the assertion passed and is only observed from here on. + Observed(VerifiedAssertion), /// Enforcement active but assertion absent/rejected — return the HTTP /// denial response. Denied(Response), } +/// The enforced and the shadow-observed assertion slots of an admitted upgrade. +pub(crate) type UpgradeAssertions = (Option, Option); + +impl NipFiUpgradeOutcome { + /// The enforced and the shadow-observed assertion, or the denial response. + pub(crate) fn into_assertions(self) -> Result>> { + match self { + Self::NotRequired => Ok((None, None)), + Self::Admitted(assertion) => Ok((Some(assertion), None)), + Self::Observed(assertion) => Ok((None, Some(assertion))), + Self::Denied(response) => Err(Box::new(response)), + } + } +} + /// Validate the NIP-FI assertion on a WebSocket upgrade request. /// /// Returns: @@ -45,16 +62,38 @@ pub(crate) enum NipFiUpgradeOutcome { /// denied" would be false. "authorization unavailable, retry after repair" is /// the accurate and correct signal. [FI-TRACE-DENIAL-ORACLE] pub(crate) fn check_nip_fi_at_upgrade( + route: crate::nip_fi_session::NipFiWsRoute, headers: &HeaderMap, communities: &NipFiCommunities, verifier: Option<&dyn VerifyAssertion>, mode: NipFiMode, ) -> NipFiUpgradeOutcome { - if matches!(mode, NipFiMode::Off) { + if !mode.restricts() { + // Shadow records an upgrade would-deny; a passing assertion is + // observed until AUTH, where its admission verdict is recorded. + if mode.observes_only() { + let verdict = resolve_community(headers, communities) + .map_err(|class| (crate::nip_fi_shadow::Stage::Community, class)) + .and_then(|community| { + evaluate_attached_assertion(headers, community, verifier).map_err(|rejection| { + ( + crate::nip_fi_shadow::Stage::Assertion, + rejection.denial_class(), + ) + }) + }); + match verdict { + Ok(assertion) => return NipFiUpgradeOutcome::Observed(assertion), + Err(deny) => { + let route = route.shadow_label(); + crate::nip_fi_shadow::record(route, headers, communities, Err(deny)); + } + } + } return NipFiUpgradeOutcome::NotRequired; } - if matches!(mode, NipFiMode::DenyProtected) { + if mode.denies_unconditionally() { return NipFiUpgradeOutcome::Denied(http_denial(DenialClass::AuthorizationUnavailable)); } @@ -77,6 +116,8 @@ pub(crate) fn check_nip_fi_at_upgrade( #[cfg(test)] mod tests { use super::*; + + const ROOT: crate::nip_fi_session::NipFiWsRoute = crate::nip_fi_session::NipFiWsRoute::Root; use crate::nip_fi_core::extract_bearer_token; use crate::nip_fi_core::test_support::{communities, host_headers}; use crate::nip_fi_core::tests::ScriptedVerifier; @@ -346,8 +387,13 @@ mod tests { CLIENT_ATTACHED_HEADER, axum::http::HeaderValue::from_static("Bearer eyJhbGciOiJFUzI1NiJ9.e30.sig"), ); - let outcome = - check_nip_fi_at_upgrade(&h, &communities(), None, buzz_auth::NipFiMode::Enforce); + let outcome = check_nip_fi_at_upgrade( + ROOT, + &h, + &communities(), + None, + buzz_auth::NipFiMode::Enforce, + ); match outcome { NipFiUpgradeOutcome::Denied(resp) => { assert_eq!(resp.status(), axum::http::StatusCode::SERVICE_UNAVAILABLE); @@ -362,6 +408,7 @@ mod tests { fn enforce_missing_header_returns_401_exact_bytes() { let headers = host_headers(); let outcome = check_nip_fi_at_upgrade( + ROOT, &headers, &communities(), None, @@ -386,12 +433,17 @@ mod tests { } } - // Off mode → NotRequired (no assertion needed — OSS default, no regression) + // Off → NotRequired (no assertion needed — OSS default, no regression) #[test] fn off_mode_returns_not_required() { let headers = host_headers(); // no assertion header - let outcome = - check_nip_fi_at_upgrade(&headers, &communities(), None, buzz_auth::NipFiMode::Off); + let outcome = check_nip_fi_at_upgrade( + ROOT, + &headers, + &communities(), + None, + buzz_auth::NipFiMode::Off, + ); assert!( matches!(outcome, NipFiUpgradeOutcome::NotRequired), "Off mode must not require assertion — OSS default must not regress" @@ -414,6 +466,7 @@ mod tests { fn deny_protected_returns_503_authorization_unavailable() { let headers = host_headers(); let outcome = check_nip_fi_at_upgrade( + ROOT, &headers, &communities(), None, @@ -467,6 +520,7 @@ mod tests { for (err, status, body) in rows { let verifier = ScriptedVerifier::new(Err(err)); let (got_status, got_body) = denied_parts(check_nip_fi_at_upgrade( + ROOT, &headers_with("Bearer a.b.c"), &communities(), Some(&verifier), @@ -482,6 +536,7 @@ mod tests { #[test] fn characterize_upgrade_transport_precedes_verifier_presence() { let (status, body) = denied_parts(check_nip_fi_at_upgrade( + ROOT, &headers_with("junk"), &communities(), None, @@ -498,6 +553,7 @@ mod tests { fn characterize_upgrade_admits_claimless_assertion_for_later_pairing() { let verifier = ScriptedVerifier::new(Ok(None)); match check_nip_fi_at_upgrade( + ROOT, &headers_with("Bearer a.b.c"), &communities(), Some(&verifier), @@ -519,10 +575,17 @@ mod tests { let verifier = ScriptedVerifier::new(Ok(Some(nostr::Keys::generate().public_key()))); let headers = headers_with("Bearer a.b.c"); assert!(matches!( - check_nip_fi_at_upgrade(&headers, &communities(), Some(&verifier), NipFiMode::Off), + check_nip_fi_at_upgrade( + ROOT, + &headers, + &communities(), + Some(&verifier), + NipFiMode::Off + ), NipFiUpgradeOutcome::NotRequired )); let (status, _) = denied_parts(check_nip_fi_at_upgrade( + ROOT, &headers, &communities(), Some(&verifier), @@ -531,4 +594,70 @@ mod tests { assert_eq!(status, StatusCode::SERVICE_UNAVAILABLE); assert_eq!(verifier.calls(), 0); } + + // Pins: shadow evaluates the assertion but only observes it: the enforced + // slot stays empty, so no connection is handed a verified assertion, and a + // rejected one admits as Off. Mutation: returning `Admitted` in shadow, or + // `Observed` for a rejection, fails the split. + #[test] + fn shadow_upgrade_observes_but_never_admits_an_assertion() { + let verifier = ScriptedVerifier::new(Ok(Some(nostr::Keys::generate().public_key()))); + let headers = headers_with("Bearer a.b.c"); + let check = || { + check_nip_fi_at_upgrade( + ROOT, + &headers, + &communities(), + Some(&verifier), + NipFiMode::Shadow, + ) + }; + let Ok((None, Some(_))) = check().into_assertions() else { + panic!("a passing shadow assertion is observed, never enforced"); + }; + assert_eq!(verifier.calls(), 1, "shadow must evaluate the assertion"); + let Ok((None, None)) = check_nip_fi_at_upgrade( + ROOT, + &host_headers(), + &communities(), + Some(&verifier), + NipFiMode::Shadow, + ) + .into_assertions() else { + panic!("a missing shadow assertion admits as Off"); + }; + } + + // Pins: a shadow upgrade would-deny records under its own ingress, so + // audio refusals never count toward root's rate. Mutation: hard-coding + // the `ws` route moves the audio records. + #[test] + fn shadow_upgrade_would_deny_records_its_ingress_route() { + use crate::nip_fi_session::NipFiWsRoute::Audio; + use crate::nip_fi_shadow_session::tests::shadow_records_by_route; + let rejecting = + ScriptedVerifier::new(Err(buzz_auth::VerifierError::InvalidSignatureOrClaims)); + let records = shadow_records_by_route(async { + for route in [ROOT, Audio] { + for headers in [host_headers(), headers_with("Bearer a.b.c")] { + let verifier = Some(&rejecting as &dyn VerifyAssertion); + let outcome = check_nip_fi_at_upgrade( + route, + &headers, + &communities(), + verifier, + NipFiMode::Shadow, + ); + assert!(matches!(outcome, NipFiUpgradeOutcome::NotRequired)); + } + } + }); + let expected = [ + "audio assertion/missing", + "audio assertion/rejected", + "ws assertion/missing", + "ws assertion/rejected", + ]; + assert_eq!(records, expected); + } } diff --git a/crates/buzz-relay/src/router.rs b/crates/buzz-relay/src/router.rs index 6c0fbb4230..17501fbaa9 100644 --- a/crates/buzz-relay/src/router.rs +++ b/crates/buzz-relay/src/router.rs @@ -49,7 +49,7 @@ use crate::state::AppState; // // What is guaranteed: in Enforce, this guard rejects a missing or invalid // assertion on every non-exempt route; in DenyProtected it returns 503 -// without verifying; in Off it is transparent. Key pairing +// without verifying; in Off and Shadow it is transparent. Key pairing // (`asserted_key == proven_pubkey`) and the deny map run only in handlers // that call `admit_nip_fi_http_on_state`. In Enforce, a handler that omits // the call and does its own NIP-98 is still subject to the assertion guard, @@ -65,10 +65,10 @@ use crate::state::AppState; // Enforce mode, which is intentional: the default is DENY; public status is // explicit. // -// ## Relationship to Off mode +// ## Relationship to Off and Shadow // -// When `NipFiMode::Off` the guard is fully transparent — no request is -// touched. [FI-INV-15] +// When `NipFiMode::Off` or `NipFiMode::Shadow` the guard is fully transparent — +// no request is touched; Shadow records its verdict elsewhere. [FI-INV-15] // // ## What this guard checks (and does NOT check) // @@ -171,19 +171,68 @@ const NIP_FI_EXEMPT_PREFIXES: &[&str] = &[ /// handler then performs the key pairing and deny-map checks via /// `admit_nip_fi_http_on_state`. /// -/// In Off mode the middleware is fully transparent. +/// In Off mode the middleware is fully transparent; in Shadow it only records. async fn nip_fi_assertion_guard( State(state): State>, request: Request, next: middleware::Next, ) -> axum::response::Response { - use buzz_auth::NipFiMode; - - // Off mode: fully transparent. [FI-INV-15] - if matches!(state.config.nip_fi.mode, NipFiMode::Off) { + let mode = state.config.nip_fi.mode; + // Off: fully transparent. [FI-INV-15] + if mode.is_off() || nip_fi_guard_exempts(&state, &request) { return next.run(request).await; } + // DenyProtected: unconditional 503 regardless of assertion presence. + // (`admit_nip_fi_http_on_state` also does this; the guard is the backstop.) + if mode.denies_unconditionally() { + return http_denial(buzz_auth::DenialClass::AuthorizationUnavailable); + } + + let verdict = nip_fi_guard_steps(&state, request.headers()); + // Shadow: transparent, but the guard's verdict is recorded in enforce's + // order, ahead of any handler step. + if mode.observes_only() { + let headers = request.headers().clone(); + return crate::nip_fi_shadow::observe_guard(&state, &headers, verdict, next.run(request)) + .await; + } + match verdict { + Ok(()) => next.run(request).await, + Err((_, class)) => http_denial(class), + } +} + +/// The guard's enforce steps: the Host must map to a configured community, +/// then the attached assertion must verify offline (transport, then +/// signature, issuer, community, expiry and claims). A forgotten-gate +/// handler that omits `admit_nip_fi_http_on_state` can only be reached with a +/// cryptographically valid assertion. Key pairing and deny-map are performed +/// by `admit_nip_fi_http_on_state` in the handler, not here. +/// [FI-TRACE-TRANSPORT-CLOSED] [FI-TRACE-AUTHORITY-UNIFORM] +fn nip_fi_guard_steps( + state: &AppState, + headers: &axum::http::HeaderMap, +) -> Result<(), crate::nip_fi_shadow::WouldDeny> { + let community = + crate::nip_fi_core::resolve_community(headers, &state.config.nip_fi.communities) + .map_err(|class| (crate::nip_fi_shadow::Stage::Community, class))?; + crate::nip_fi_core::evaluate_attached_assertion( + headers, + community, + state.nip_fi_verifier.as_deref(), + ) + .map(drop) + .map_err(|rejection| { + ( + crate::nip_fi_shadow::Stage::Assertion, + rejection.denial_class(), + ) + }) +} + +/// Paths the assertion guard never checks. +fn nip_fi_guard_exempts(state: &AppState, request: &Request) -> bool { let path = request.uri().path(); // Exempt paths bypass the assertion-token check. @@ -209,7 +258,7 @@ async fn nip_fi_assertion_guard( }); if exempt { - return next.run(request).await; + return true; } // Admin SPA document routes are exempt when the request is on the admin @@ -219,43 +268,9 @@ async fn nip_fi_assertion_guard( // go to `/api/admin/v1/...` (already exempt via the `/api/admin/` prefix). // // The exemption is host-qualified: `/reports` on a tenant host is NOT - // exempt and stays protected. Off mode is already handled above. + // exempt and stays protected. // [FI-TRACE-AUTHORITY-UNIFORM] - if is_admin_spa_path(path) && api::admin::is_admin_host(&state, request.headers()) { - return next.run(request).await; - } - - // Non-exempt path in Enforce or DenyProtected mode. - // - // DenyProtected: unconditional 503 regardless of assertion presence. - // (`admit_nip_fi_http_on_state` also does this; the guard is the backstop.) - if matches!(state.config.nip_fi.mode, NipFiMode::DenyProtected) { - return http_denial(buzz_auth::DenialClass::AuthorizationUnavailable); - } - - // Enforce mode, step 1: the Host must map to a configured community. - let community = match crate::nip_fi_core::resolve_community( - request.headers(), - &state.config.nip_fi.communities, - ) { - Ok(community) => community, - Err(class) => return http_denial(class), - }; - - // Enforce mode: full offline assertion verification (transport, then - // signature, issuer, community, expiry and claims). A forgotten-gate handler that - // omits `admit_nip_fi_http_on_state` can only be reached with a - // cryptographically valid assertion. Key pairing and deny-map are - // performed by `admit_nip_fi_http_on_state` in the handler, not here. - // [FI-TRACE-TRANSPORT-CLOSED] [FI-TRACE-AUTHORITY-UNIFORM] - match crate::nip_fi_core::evaluate_attached_assertion( - request.headers(), - community, - state.nip_fi_verifier.as_deref(), - ) { - Ok(_) => next.run(request).await, - Err(rejection) => http_denial(rejection.denial_class()), - } + is_admin_spa_path(path) && api::admin::is_admin_host(state, request.headers()) } /// Build the axum [`Router`] with all relay routes, middleware, and CORS configuration. @@ -611,7 +626,7 @@ async fn nip11_or_ws_handler( // // Keying on the header pair (not on `Accept`) means an HTML Accept header // on a real WS upgrade is still gated correctly. - let nip_fi_assertion = { + let (upgrade_checked, nip_fi_assertion, shadow_assertion) = { let is_h1_ws_upgrade = headers .get(axum::http::header::UPGRADE) .and_then(|v| v.to_str().ok()) @@ -634,20 +649,22 @@ async fn nip11_or_ws_handler( let is_h2_ws_connect = req.version() == axum::http::Version::HTTP_2 && req.method() == axum::http::Method::CONNECT; if is_h1_ws_upgrade || is_h2_ws_connect { - use crate::nip_fi_upgrade::{check_nip_fi_at_upgrade, NipFiUpgradeOutcome}; + use crate::nip_fi_session::NipFiWsRoute::Root; + use crate::nip_fi_upgrade::check_nip_fi_at_upgrade; let mode = state.config.nip_fi.mode; let verifier = state.nip_fi_verifier.as_deref(); let communities = &state.config.nip_fi.communities; - match check_nip_fi_at_upgrade(&headers, communities, verifier, mode) { - NipFiUpgradeOutcome::NotRequired => None, - NipFiUpgradeOutcome::Admitted(assertion) => Some(assertion), - NipFiUpgradeOutcome::Denied(resp) => return resp.into_response(), + match check_nip_fi_at_upgrade(Root, &headers, communities, verifier, mode) + .into_assertions() + { + Ok((assertion, shadow)) => (true, assertion, shadow), + Err(resp) => return (*resp).into_response(), } } else { // Not a WS upgrade shape — a NIP-11 request or a plain browser GET. // The `Ok(ws)` arm below backstops any extractor-accepted shape this // predicate misses. [F3-H2-GATE] - None + (false, None, None) } }; @@ -677,6 +694,8 @@ async fn nip11_or_ws_handler( } } } + let shadow_assertion = shadow_assertion + .filter(|a| !crate::nip_fi_shadow_session::upgrade_denied(&state, &headers, a)); // Row zero: bind the connection to its community from the request host // BEFORE the WebSocket upgrade, so no frame is ever read on an unbound @@ -716,19 +735,21 @@ async fn nip11_or_ws_handler( // For HTTP/1.1 requests, `nip_fi_assertion` was already set above // and this block is unreachable (the h1 denial is returned before // we get here). - let nip_fi_assertion = if nip_fi_assertion.is_none() { + let (nip_fi_assertion, shadow_assertion) = if !upgrade_checked { // Only re-check if the pre-extractor gate did not fire (h2 path). - use crate::nip_fi_upgrade::{check_nip_fi_at_upgrade, NipFiUpgradeOutcome}; + use crate::nip_fi_session::NipFiWsRoute::Root; + use crate::nip_fi_upgrade::check_nip_fi_at_upgrade; let mode = state.config.nip_fi.mode; let verifier = state.nip_fi_verifier.as_deref(); let communities = &state.config.nip_fi.communities; - match check_nip_fi_at_upgrade(&headers, communities, verifier, mode) { - NipFiUpgradeOutcome::NotRequired => None, - NipFiUpgradeOutcome::Admitted(assertion) => Some(assertion), - NipFiUpgradeOutcome::Denied(resp) => return resp.into_response(), + match check_nip_fi_at_upgrade(Root, &headers, communities, verifier, mode) + .into_assertions() + { + Ok(assertions) => assertions, + Err(resp) => return (*resp).into_response(), } } else { - nip_fi_assertion + (nip_fi_assertion, shadow_assertion) }; // Shutting down: refuse new sockets instead of accepting a @@ -745,6 +766,15 @@ async fn nip11_or_ws_handler( // handshake, not the post-community-active-check instant. // [FI-TRACE-LEASE-BOUND] let connection_time = chrono::Utc::now(); + let shadow = shadow_assertion.map(|a| { + crate::nip_fi_shadow_session::ShadowSession::start( + &state, + "ws", + &headers, + a, + connection_time, + ) + }); limit_relay_websocket(ws, max_frame_bytes) .on_upgrade(move |socket| { handle_connection( @@ -753,6 +783,7 @@ async fn nip11_or_ws_handler( addr, tenant, nip_fi_assertion, + shadow, connection_time, ) }) @@ -2497,6 +2528,389 @@ mod tests { use super::*; use std::sync::Arc; + // Pins: a real shadow root handshake runs the upgrade check once, and + // a passing assertion closed before AUTH records no verdict. + // Mutation: re-running the fallback whenever no assertion is held + // doubles the verifier call; recording at upgrade adds a record. + #[tokio::test(flavor = "current_thread")] + #[ignore = "requires Postgres — runs in postgres-ci nextest lane"] + async fn shadow_root_handshake_is_checked_once() { + use tokio_tungstenite::tungstenite::client::IntoClientRequest; + let recorder = metrics_util::debugging::DebuggingRecorder::new(); + let snapshotter = recorder.snapshotter(); + let _guard = metrics::set_default_local_recorder(&recorder); + + let base = real_db_state().await.expect("PostgreSQL must be available"); + let community_id = uuid::Uuid::new_v4(); + let host = format!("shadow-ws-{}.example", community_id.simple()); + sqlx::query("INSERT INTO communities (id, host) VALUES ($1, $2)") + .bind(community_id) + .bind(&host) + .execute(base.db.pool()) + .await + .expect("seed community"); + let verifier = Arc::new(ScriptedVerifier::new(Ok(Some( + nostr::Keys::generate().public_key(), + )))); + let mut state = (*base).clone(); + let config = Arc::make_mut(&mut state.config); + config.nip_fi.mode = buzz_auth::NipFiMode::Shadow; + config.nip_fi.communities = crate::nip_fi_config::NipFiCommunities::for_test( + &format!("https://{host}"), + &["https://issuer.test"], + ); + state.nip_fi_verifier = Some(verifier.clone()); + + let listener = TcpListener::bind("127.0.0.1:0").await.expect("bind"); + let addr = listener.local_addr().expect("addr"); + let app = build_router(Arc::new(state)) + .into_make_service_with_connect_info::(); + let server = tokio::spawn(async move { axum::serve(listener, app).await }); + let mut req = format!("ws://{addr}/") + .into_client_request() + .expect("request"); + req.headers_mut().insert("host", host.parse().unwrap()); + req.headers_mut() + .insert("nostr-federated-identity", "Bearer a.b.c".parse().unwrap()); + let (client, resp) = connect_async(req) + .await + .expect("shadow admits the handshake"); + assert_eq!(resp.status(), axum::http::StatusCode::SWITCHING_PROTOCOLS); + drop(client); + server.abort(); + + let records: u64 = snapshotter + .snapshot() + .into_vec() + .into_iter() + .filter(|(key, ..)| key.key().name() == "buzz_nip_fi_shadow_total") + .map(|(.., value)| match value { + metrics_util::debugging::DebugValue::Counter(n) => n, + _ => 0, + }) + .sum(); + assert_eq!((verifier.calls(), records), (1, 0)); + } + + // Pins the guard order for a Host mapped in config but absent from the + // communities table: Off's rejection is unchanged and shadow records + // the verdict enforce's guard reaches first — `assertion/missing` + // without one (enforce: 401), an admit with a valid one (enforce: + // the guard passes and the handler's 404 is no NIP-FI denial). + // Mutation: recording `community` in `observe_unbound` whenever the + // Host is unbound, or replaying the handler order, fails both rows. + #[tokio::test(flavor = "current_thread")] + #[ignore = "requires Postgres — runs in postgres-ci nextest lane"] + async fn shadow_db_unmapped_host_records_the_guard_verdict() { + let base = real_db_state().await.expect("PostgreSQL must be available"); + let host = format!("unmapped-{}.example", uuid::Uuid::new_v4().simple()); + let with_mode = |mode| { + let mut state = (*base).clone(); + let config = Arc::make_mut(&mut state.config); + config.nip_fi.mode = mode; + config.nip_fi.communities = crate::nip_fi_config::NipFiCommunities::for_test( + &format!("https://{host}"), + &["https://issuer.test"], + ); + state.nip_fi_verifier = Some(Arc::new(ScriptedVerifier::new(Ok(Some( + nostr::Keys::generate().public_key(), + ))))); + Arc::new(state) + }; + for (assertion, stage, outcome) in [ + (None, "assertion", "missing"), + (Some("Bearer a.b.c"), "admit", "admit"), + ] { + let labels = shadow_matches_off_with_one_record( + with_mode(buzz_auth::NipFiMode::Off), + with_mode(buzz_auth::NipFiMode::Shadow), + || { + let mut req = + axum::http::Request::post("/api/invites").header("host", &host); + if let Some(token) = assertion { + req = req.header(buzz_auth::CLIENT_ATTACHED_HEADER, token); + } + req.body(axum::body::Body::empty()).unwrap() + }, + ) + .await + .0; + for label in [ + format!("\"stage\", \"{stage}\""), + format!("\"outcome\", \"{outcome}\""), + format!("\"community\", \"https://{host}\""), + ] { + assert!(labels.contains(&label), "{assertion:?}: {labels}"); + } + } + } + + // Pins: the workflow handler's shared NIP-98 closure keeps the dev + // `X-Pubkey` proof unsigned, so shadow admits it as Off does but + // records a NIP-98 would-deny; only the guard verifies, as in enforce. + // Mutation: dropping the unsigned marker records `admit`. + #[tokio::test(flavor = "current_thread")] + #[ignore = "requires Postgres — runs in postgres-ci nextest lane"] + async fn shadow_workflow_never_counts_x_pubkey_as_an_enforce_proof() { + use tower::ServiceExt; + let recorder = metrics_util::debugging::DebuggingRecorder::new(); + let snapshotter = recorder.snapshotter(); + let _guard = metrics::set_default_local_recorder(&recorder); + + let base = real_db_state().await.expect("PostgreSQL must be available"); + let community_id = uuid::Uuid::new_v4(); + let host = format!("shadow-wf-{}.example", community_id.simple()); + sqlx::query("INSERT INTO communities (id, host) VALUES ($1, $2)") + .bind(community_id) + .bind(&host) + .execute(base.db.pool()) + .await + .expect("seed community"); + let key = nostr::Keys::generate().public_key(); + let verifier = Arc::new(ScriptedVerifier::new(Ok(Some(key)))); + let mut state = (*base).clone(); + let config = Arc::make_mut(&mut state.config); + config.require_auth_token = false; + config.nip_fi.mode = buzz_auth::NipFiMode::Shadow; + config.nip_fi.communities = crate::nip_fi_config::NipFiCommunities::for_test( + &format!("https://{host}"), + &["https://issuer.test"], + ); + state.nip_fi_verifier = Some(verifier.clone()); + + let req = axum::http::Request::get(format!("/workflows/{}/runs", uuid::Uuid::nil())) + .header("host", &host) + .header("x-pubkey", key.to_hex()) + .header(buzz_auth::CLIENT_ATTACHED_HEADER, "Bearer a.b.c") + .body(axum::body::Body::empty()) + .unwrap(); + let resp = build_router(Arc::new(state)).oneshot(req).await.unwrap(); + assert_ne!( + resp.status(), + axum::http::StatusCode::UNAUTHORIZED, + "admitted" + ); + + let stages: Vec = snapshotter + .snapshot() + .into_vec() + .into_iter() + .filter(|(k, ..)| k.key().name() == "buzz_nip_fi_shadow_total") + .filter_map(|(k, ..)| { + let stage = k.key().labels().find(|l| l.key() == "stage")?; + Some(stage.value().to_owned()) + }) + .collect(); + assert_eq!((stages, verifier.calls()), (vec!["nip98".to_owned()], 1)); + } + + fn nip98(keys: &nostr::Keys, url: &str, method: &str) -> String { + use base64::Engine as _; + let event = nostr::EventBuilder::new(nostr::Kind::HttpAuth, "") + .tags([ + nostr::Tag::parse(["u", url]).unwrap(), + nostr::Tag::parse(["method", method]).unwrap(), + // A distinct event per call: Off and Shadow share Redis. + nostr::Tag::parse(["nonce", &uuid::Uuid::new_v4().to_string()]).unwrap(), + ]) + .sign_with_keys(keys) + .unwrap(); + let json = serde_json::to_string(&event).unwrap(); + format!( + "Nostr {}", + base64::engine::general_purpose::STANDARD.encode(json) + ) + } + + /// Off and Shadow states on a seeded, mapped Host whose assertion + /// names `key`; each gets its own pool so a row may close it. + async fn seeded_pair( + key: nostr::PublicKey, + ) -> (String, buzz_core::CommunityId, [Arc; 2]) { + let probe = real_db_state().await.expect("PostgreSQL must be available"); + let community_id = uuid::Uuid::new_v4(); + let host = format!("shadow-row-{}.example", community_id.simple()); + sqlx::query("INSERT INTO communities (id, host) VALUES ($1, $2)") + .bind(community_id) + .bind(&host) + .execute(probe.db.pool()) + .await + .expect("seed community"); + let mut states = Vec::new(); + for mode in [buzz_auth::NipFiMode::Off, buzz_auth::NipFiMode::Shadow] { + let mut state = (*real_db_state().await.unwrap()).clone(); + let config = Arc::make_mut(&mut state.config); + config.relay_url = "wss://relay.example".to_owned(); + config.require_relay_membership = true; + config.nip_fi.mode = mode; + config.nip_fi.communities = crate::nip_fi_config::NipFiCommunities::for_test( + &format!("https://{host}"), + &["https://issuer.test"], + ); + // Real Redis so admission reaches the handler, not the + // fail-closed rate limiter. + let redis = deadpool_redis::Config::from_url( + std::env::var("REDIS_URL").unwrap_or("redis://127.0.0.1:6379".into()), + ) + .create_pool(Some(deadpool_redis::Runtime::Tokio1)) + .unwrap(); + state.admission_rate_limiter = Arc::new( + buzz_pubsub::rate_limiter::RedisRateLimiter::new(redis.clone()), + ); + state.nip98_replay = + Arc::new(buzz_pubsub::RedisNip98ReplayGuard::new(redis.clone())); + state.redis_pool = redis; + state.nip_fi_verifier = Some(Arc::new(ScriptedVerifier::new(Ok(Some(key))))); + states.push(Arc::new(state)); + } + let community = buzz_core::CommunityId::from_uuid(community_id); + (host, community, states.try_into().ok().unwrap()) + } + + // Pins Off parity on a seeded Host for the bridge's payload strictness + // and junk proofs, Blossom reads, and the Git membership lookup + // failure. Each row matches Off byte for byte, leaves exactly one + // shadow verdict, and the expected strict-proof side checks. + // Mutation: recording the bridge strict proof twice, requiring the + // payload tag in shadow, or answering the Git lookup failure with the + // NIP-FI 503 in shadow each fails a row. + #[tokio::test(flavor = "current_thread")] + #[ignore = "requires Postgres — runs in postgres-ci nextest lane"] + async fn shadow_seeded_host_rows_match_off() { + let keys = nostr::Keys::generate(); + let sha = "a".repeat(64); + let body = serde_json::to_vec( + &nostr::EventBuilder::new(nostr::Kind::TextNote, "hi") + .sign_with_keys(&keys) + .unwrap(), + ) + .unwrap(); + let git = "/git/o/r.git/info/refs?service=git-upload-pack"; + // (method, path, proof, body, strict-proof outcomes, close the pool) + type Row<'a> = ( + &'a str, + String, + Option<&'a str>, + Vec, + Vec<&'a str>, + bool, + ); + let rows: Vec = vec![ + ( + "POST", + "/events".into(), + None, + body.clone(), + vec!["rejected"], + false, + ), + ( + "POST", + "/events".into(), + Some("Nostr !!junk!!"), + body, + vec!["rejected"], + false, + ), + ( + "GET", + format!("/media/{sha}"), + None, + Vec::new(), + vec!["rejected"], + false, + ), + ("GET", git.into(), None, Vec::new(), vec![], true), + ]; + for (method, path, proof, body, strict, close) in rows { + let (host, community, [off, shadow]) = seeded_pair(keys.public_key()).await; + let signed_path = path.split("/info/refs").next().unwrap(); + let url = format!("https://{host}{signed_path}"); + if close { + // Fail the membership lookup after the tenant bound: Off + // runs first, then Shadow, each against its own pool. + use crate::nip_fi_test_hooks::git_membership_hook::arm; + let (off_pool, shadow_pool) = (off.db.pool().clone(), shadow.db.pool().clone()); + let mut gate = arm(community); + tokio::spawn(async move { + for pool in [off_pool, shadow_pool] { + let (arrived, release) = gate; + arrived.await.unwrap(); + pool.close().await; + // Re-arm before releasing, so Shadow's request + // cannot reach the hook unarmed. + gate = arm(community); + release.notify_one(); + } + }); + } + let (_, outcomes) = shadow_matches_off_with_one_record(off, shadow, || { + axum::http::Request::builder() + .method(method) + .uri(&path) + .header("host", &host) + .header( + "authorization", + proof.map_or_else(|| nip98(&keys, &url, method), str::to_owned), + ) + .header(buzz_auth::CLIENT_ATTACHED_HEADER, "Bearer a.b.c") + .body(axum::body::Body::from(body.clone())) + .unwrap() + }) + .await; + assert_eq!(outcomes, strict, "{method} {path} {proof:?}"); + } + } + + // Pins Off parity on a seeded Host for the remaining NIP-FI routes: + // a signed NIP-98 proof and an attached assertion leave Off's exact + // response and exactly one shadow verdict. + #[tokio::test(flavor = "current_thread")] + #[ignore = "requires Postgres — runs in postgres-ci nextest lane"] + async fn shadow_seeded_host_remaining_routes_match_off() { + let keys = nostr::Keys::generate(); + let sha = "a".repeat(64); + let branch = format!("/git/{sha}/r/default-branch"); + let filter = br#"{"kinds":[1],"limit":1}"#.to_vec(); + let rows = [ + ("POST", "/query".to_owned(), filter.clone()), + ("POST", "/count".to_owned(), filter), + ("GET", "/moderation/reports".to_owned(), Vec::new()), + ("HEAD", format!("/media/{sha}"), Vec::new()), + ("GET", format!("/media/{sha}"), Vec::new()), + ("GET", branch.clone(), Vec::new()), + ("POST", branch, br#"{"branch":"main"}"#.to_vec()), + ( + "POST", + "/gifs/search".to_owned(), + br#"{"q":"cat"}"#.to_vec(), + ), + ("POST", "/gifs/share".to_owned(), br#"{"id":"x"}"#.to_vec()), + ("POST", "/api/invites".to_owned(), b"{}".to_vec()), + ( + "GET", + format!("/workflows/{}/runs", uuid::Uuid::nil()), + Vec::new(), + ), + ]; + for (method, path, body) in rows { + let (host, _, [off, shadow]) = seeded_pair(keys.public_key()).await; + let url = format!("https://{host}{path}"); + shadow_matches_off_with_one_record(off, shadow, || { + axum::http::Request::builder() + .method(method) + .uri(&path) + .header("host", &host) + .header("content-type", "application/json") + .header("authorization", nip98(&keys, &url, method)) + .header(buzz_auth::CLIENT_ATTACHED_HEADER, "Bearer a.b.c") + .body(axum::body::Body::from(body.clone())) + .unwrap() + }) + .await; + } + } + async fn real_db_state() -> Option> { let db_url = crate::test_support::database_url(); let pool = sqlx::PgPool::connect(&db_url).await.ok()?; @@ -3708,6 +4122,120 @@ mod tests { ); } + /// Sends one request through the built router in Off and in Shadow. + /// Asserts identical status, challenge and body, no Off record, and one + /// Shadow record; returns that record's labels and the shadow + /// strict-proof outcomes. + async fn shadow_matches_off_with_one_record( + off: Arc, + shadow: Arc, + request: impl Fn() -> axum::http::Request, + ) -> (String, Vec) { + use tower::ServiceExt; + let mut outcomes = Vec::new(); + for state in [off, shadow] { + let recorder = metrics_util::debugging::DebuggingRecorder::new(); + let snapshotter = recorder.snapshotter(); + let _guard = metrics::set_default_local_recorder(&recorder); + let resp = build_router(state).oneshot(request()).await.unwrap(); + let challenge = resp.headers().get("WWW-Authenticate").cloned(); + let snapshot = snapshotter.snapshot().into_vec(); + let records: Vec<_> = snapshot + .iter() + .filter(|(key, ..)| key.key().name() == "buzz_nip_fi_shadow_total") + .map(|(key, .., value)| { + let labels = format!("{:?}", key.key().labels().collect::>()); + (labels, counter(value)) + }) + .collect(); + let strict: Vec = snapshot + .iter() + .filter(|(key, ..)| key.key().name() == "buzz_nip_fi_shadow_strict_proof_total") + .flat_map(|(key, .., value)| { + let outcome = key.key().labels().find(|l| l.key() == "outcome"); + let outcome = outcome.map_or("-", |l| l.value()).to_owned(); + std::iter::repeat_n(outcome, counter(value) as usize) + }) + .collect(); + outcomes.push(((challenge, status_and_body(resp).await), records, strict)); + } + let (shadow, off) = (outcomes.pop().unwrap(), outcomes.pop().unwrap()); + assert_eq!(shadow.0, off.0, "rejection unchanged"); + assert!(off.1.is_empty() && off.2.is_empty(), "off records nothing"); + let [(labels, value)] = <[_; 1]>::try_from(shadow.1).expect("exactly one shadow record"); + assert_eq!(value, 1); + (labels, shadow.2) + } + + fn counter(value: &metrics_util::debugging::DebugValue) -> u64 { + match value { + metrics_util::debugging::DebugValue::Counter(n) => *n, + _ => 0, + } + } + + // Pins D8 on every handler that returns before NIP-FI admission: an empty + // Host keeps Off's exact rejection and leaves one `community` record. + // Mutation: dropping the record at any site leaves no shadow record. + #[tokio::test(flavor = "current_thread")] + async fn shadow_empty_host_keeps_off_rejection_and_records_community() { + use axum::body::Body; + use base64::Engine as _; + let event = nostr::EventBuilder::new(nostr::Kind::HttpAuth, "") + .sign_with_keys(&nostr::Keys::generate()) + .unwrap(); + let signed = format!( + "Nostr {}", + base64::engine::general_purpose::STANDARD + .encode(serde_json::to_string(&event).unwrap()) + ); + let sha = "a".repeat(64); + let git = "/git/o/r.git/info/refs?service=git-upload-pack".to_owned(); + let branch = format!("/git/{sha}/r/default-branch"); + let rows = [ + ("POST", "/events".to_owned(), Some(signed.as_str())), + ("PUT", "/upload".to_owned(), Some(&signed)), + ("GET", format!("/media/{sha}"), Some(&signed)), + ("GET", "/moderation/reports".to_owned(), Some(&signed)), + ( + "GET", + format!("/workflows/{}/runs", uuid::Uuid::nil()), + Some(&signed), + ), + ("GET", git.clone(), Some(&signed)), + ("GET", git.clone(), None), + ("GET", git, Some("Nostr !!not-base64!!")), + ("POST", "/gifs/search".to_owned(), Some(&signed)), + ("POST", "/gifs/share".to_owned(), Some(&signed)), + ("GET", branch.clone(), Some(&signed)), + ("POST", branch, Some(&signed)), + ("POST", "/api/invites".to_owned(), Some(&signed)), + ]; + for (method, path, auth) in rows { + let request = || { + let mut req = axum::http::Request::builder().method(method).uri(&path); + if let Some(auth) = auth { + req = req.header("authorization", auth); + } + req.body(Body::empty()).unwrap() + }; + let labels = shadow_matches_off_with_one_record( + nip_fi_state(buzz_auth::NipFiMode::Off).await, + nip_fi_state(buzz_auth::NipFiMode::Shadow).await, + request, + ) + .await + .0; + for label in [ + "\"stage\", \"community\"", + "\"outcome\", \"unavailable\"", + "\"community\", \"unmapped\"", + ] { + assert!(labels.contains(label), "{method} {path} {auth:?}: {labels}"); + } + } + } + // ── Characterization: HTTP guard evaluation contract ───────────────────── const GUARD_PROTECTED_PATH: &str = "/workflows/wf/runs"; @@ -3861,6 +4389,27 @@ mod tests { ); } + // Pins the guard's remaining enforce rows before shadow replays them: a + // mapped Host with no assertion is 401 before verification, and an exempt + // path passes the guard untouched. + // Mutation: dropping the guard's assertion step lets the handler answer; + // guarding `/health` turns its 200 into 401. + #[tokio::test] + async fn characterize_guard_missing_assertion_is_401_and_exempt_passes() { + let (state, verifier) = guard_state_with(Ok(None)).await; + let resp = nip_fi_gate_response(state.clone(), GUARD_PROTECTED_PATH, None, None).await; + assert_eq!( + status_and_body(resp).await, + ( + axum::http::StatusCode::UNAUTHORIZED, + b"authentication required\n".to_vec() + ) + ); + let resp = nip_fi_gate_response(state, "/health", None, None).await; + assert_eq!(resp.status(), axum::http::StatusCode::OK); + assert_eq!(verifier.calls(), 0); + } + // Pins: the guard verifies but does not pair keys. A claimless assertion // passes the guard (one verify) and reaches the handler, whose path // extractor rejects the non-UUID workflow id with 400 before any NIP-98 diff --git a/crates/buzz-relay/src/state.rs b/crates/buzz-relay/src/state.rs index 6ec3870f53..954b220685 100644 --- a/crates/buzz-relay/src/state.rs +++ b/crates/buzz-relay/src/state.rs @@ -24,7 +24,7 @@ use buzz_media::MediaStorage; use buzz_pubsub::cache_invalidation::CacheInvalidation; use buzz_pubsub::conn_control::ConnControl; use buzz_pubsub::rate_limiter::RedisRateLimiter; -use buzz_pubsub::{PubSubManager, RedisCommandReplayGuard, RedisNip98ReplayGuard}; +use buzz_pubsub::{PubSubManager, RedisNip98ReplayGuard}; use buzz_search::SearchService; use buzz_workflow::WorkflowEngine; use deadpool_redis; @@ -98,6 +98,8 @@ pub(crate) struct CommunityConnectionControl { pubkey: Arc>, /// Owner of an admitted agent; revoking the owner closes this socket. owner: Arc>, + /// Shadow-mode observation of this socket; no enforce path reads it. + nip_fi_shadow: Arc>>, } impl CommunityConnectionControl { @@ -110,9 +112,28 @@ impl CommunityConnectionControl { terminal_frame_tx: Arc::new(std::sync::Mutex::new(None)), pubkey: Arc::default(), owner: Arc::default(), + nip_fi_shadow: Arc::default(), } } + /// Carries the socket's shadow session to its AUTH handler, fenced by + /// the socket's cancellation so it records nothing once cancelled. + pub(crate) fn attach_nip_fi_shadow( + &self, + session: Option>, + ) { + if let Some(session) = session { + session.fence(self.cancel.clone()); + let _ = self.nip_fi_shadow.set(session); + } + } + + pub(crate) fn nip_fi_shadow( + &self, + ) -> Option<&Arc> { + self.nip_fi_shadow.get() + } + /// Records the authenticated pubkey so pubkey-scoped disconnects reach this socket. pub(crate) fn bind_pubkey(&self, pubkey: [u8; 32]) { let _ = self.pubkey.set(pubkey); @@ -1337,6 +1358,8 @@ pub struct AppState { /// lets tests wait for every publish to finish; nothing waits on it in /// production. pub nip_fi_publish_tasks: tokio_util::task::TaskTracker, + /// Shadow-mode sessions a shadow disconnect records a would-close for. + pub(crate) nip_fi_shadow_sessions: Arc, } impl AppState { @@ -1422,8 +1445,8 @@ impl AppState { ); let nip98_replay: Arc = Arc::new(RedisNip98ReplayGuard::new(redis_pool.clone())); - let nip_fi_command_replay: Arc = - Arc::new(RedisCommandReplayGuard::new(redis_pool.clone())); + let nip_fi_command_replay = + crate::api::nip_fi::command_replay_guard(redis_pool.clone(), config.nip_fi.mode); let gif_http_client = crate::api::gifs::build_gif_http_client(); let admission_rate_limiter = Arc::new(RedisRateLimiter::new(redis_pool.clone())); let audit_enabled = audit_arc.is_some(); @@ -1531,6 +1554,7 @@ impl AppState { nip_fi_command_verifier: None, nip_fi_command_replay, nip_fi_publish_tasks: tokio_util::task::TaskTracker::new(), + nip_fi_shadow_sessions: Arc::default(), }; ( state, @@ -2088,12 +2112,9 @@ type NipFiComponents = ( ); fn build_nip_fi_components(config: &crate::config::Config) -> NipFiComponents { - use buzz_auth::{FederatedAssertionVerifier, HttpJwksFetcher, NipFiMode, ProductionJwksSource}; + use buzz_auth::{FederatedAssertionVerifier, HttpJwksFetcher, ProductionJwksSource}; - if matches!( - config.nip_fi.mode, - NipFiMode::Off | NipFiMode::DenyProtected - ) { + if !config.nip_fi.mode.evaluates() { // Off: no enforcement. DenyProtected: verifier never consulted (always 503). return (None, None); } diff --git a/scripts/run-tests.sh b/scripts/run-tests.sh index 96a16da9f2..84061935ca 100755 --- a/scripts/run-tests.sh +++ b/scripts/run-tests.sh @@ -195,6 +195,12 @@ run_unit_tests() { run_test_step "buzz-relay NIP-FI session tests" \ cargo test -p buzz-relay --lib nip_fi_session::tests:: -- --nocapture + run_test_step "buzz-relay NIP-FI shadow recorder tests" \ + cargo test -p buzz-relay --lib nip_fi_shadow::tests:: -- --nocapture + + run_test_step "buzz-relay NIP-FI shadow session tests" \ + cargo test -p buzz-relay --lib nip_fi_shadow_session::tests:: -- --nocapture + # Mirror the NIP-FI (S3) stanza from `just test-unit`: module filters, then # each exact name. Keep this list in step with that stanza's `test(=...)`s. run_test_step "buzz-relay NIP-FI config tests" \ @@ -267,6 +273,8 @@ run_unit_tests() { handlers::auth::tests::fi_root_authorization_denied_rows_emit_identical_frames handlers::auth::tests::fi_invalid_nip42_proof_emits_terminal_evidence_rejected handlers::auth::tests::handle_auth_pairing_mismatch_runs_full_root_denial_path + handlers::auth::tests::shadow_root_auth_records_pairing_denial + handlers::auth::tests::shadow_root_invalid_nip42_retires_without_record handlers::auth::tests::nip42_denial_class_separates_internal_failure_from_bad_evidence handlers::event::tests::fanout_access::owner_only_kinds_keep_only_the_owner handlers::event::tests::pubsub_fanout::pubsub_owner_only_kinds_reach_only_the_owner