feat(relay): let channel-state filters opt in to system channels (#8182)

🤖
## Summary
After part 2, a client can read a system channel only if it already
knows the ID. This PR lets a client find system channels on purpose,
with an ordinary Nostr filter and no new filter key.

A filter on the channel-state kinds (39000–39003) now also returns the
system channels the reader can read when it has either:

- `#t:["system"]`, the channel type tag that every channel-state event
already carries; or
- any `#P`, the creator tag from #8174. Asking for the channels of a
named creator returns a small result, so it does not fill generic
channel lists.

The filter then matches as usual. Existing clients send neither on these
kinds, so they still never see a system channel.

| Reader wants | Filter |
|---|---|
| My system channels | `{kinds:[39002], #p:[me], #t:["system"]}`, then
their kind:39000 by `#d` |
| Open system channels | `{kinds:[39000], #t:["system"]}` |
| One creator's labeled channel, system or not | `{kinds:[39000],
#P:[creator], #t:[label]}` |

Private system channels stay hidden from non-members, as for any private
channel. Values in one tag filter combine with OR, so `#t:["system",
label]` means `system` **or** the label; use `#P` plus the label
instead.

Part 3 of 3, stacked on part 2.

### Related issue
None found.

### Details
- The opt-in is decided per filter in `handlers/system_channels.rs` and
applies on REQ, COUNT and the HTTP bridge. It applies only when every
kind in the filter is 39000–39003.
- A label alone is not an opt-in.
- `NOSTR.md` documents the system channel query rules.
- **Spec.** NIP-CL says how a filter asks for system channels.
- The relay information document (NIP-11) now lists `"nip-cl"` in
`supported_extensions`. Clients use this to tell if a relay supports
labels and system channels, and refuse those calls on a relay that does
not. NIP-CL documents this.

### Testing
- End-to-end relay test: `#t:["system"]` finds both system channels for
a member and only the open one for a non-member; 39002 `#p:[me]` with
`#t:["system"]` lists the member's system channels; `#P` includes a
creator's system channels; creator plus label returns exactly one
channel with `limit: 1` and excludes another creator's channel with the
same label; a label alone does not return the system channel; COUNT and
HTTP `/query` and `/count` agree.
- The test fails when the opt-in is disabled.
- Unit tests for which filters opt in.

### Stack
These are two stacks. Merge them in this order. Client support
(creating, labeling and finding these channels) will be in
block/buzz-app, not the buzz SDK and CLI.

**Stack 1: channel labels and creator lookup (#8174–#8176)**

1. #8174 channel type and creator tags, and the channel creation time
2. #8175 channel labels
3. #8176 database-side label and creator match, and the rule for
choosing one channel

**Stack 2: system channels (#8180–#8182)**, stacked on stack 1 because
it uses labels and the creator tag

4. #8180 the system channel type
5. #8181 leave system channels out of reads that do not name them
6. #8182 channel-state filters opt in to system channels ← this PR

---------

Signed-off-by: Logan Johnson <loganj@squareup.com>
Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz>

Justfile

@@ -531,7 +531,7 @@ test-unit:
         # are infra-free; config parses env into a struct and never connects.
         # `^config::` is anchored so it does not also select nip_fi_config.
         cargo nextest run -p buzz-relay --lib --bin buzz-relay \
-            -E 'test(/^api::admin::/) + test(/^push_runtime::/) + test(/^operator_listener::tests::/) + test(/^handlers::channel_authz::/) + test(/^handlers::moderation_authz::/) + test(/^handlers::side_effects::tests::/) + test(/^handlers::system_channels::tests::/) + test(/^storage_sweep::tests::/) + test(/^nip_fi_core::tests::/) + test(/^nip_fi_http::tests::/) + test(/^nip_fi_config::tests::/) + test(/^readiness::tests::/) + test(/^router::tests::/) + test(/^api::parse_query_tests::/) + test(/^api::git::transport::off_mode_precedence_tests::/) + test(/^audio::join::tests::/) + test(/^audio::handler::tests::/) + test(/^nip_fi_gate::tests::/) + test(/^nip_fi_session::tests::/) + test(/^nip_fi_shadow::tests::/) + test(/^nip_fi_shadow_session::tests::/) + test(/^startup_steps::tests::/) + test(/^telemetry::tests::/) + test(/^config::tests::/) + test(=state::tests::neither_a_confirmed_inactive_community_nor_a_failed_lookup_admits_the_socket) + test(=state::tests::periodic_revalidation_disconnects_inside_the_fenced_callback) + test(=state::tests::bare_community_disconnect_fails_closed_when_the_fence_is_unavailable) + test(=connection::tests::send_loop_sends_policy_close_when_community_is_archived) + test(=state::tests::community_disconnect_then_nip_fi_keeps_community_deleted_reason) + test(=state::tests::disconnect_community_wins_reason_losing_nip_fi_does_not_enqueue_frame) + test(=handlers::req::tests::timed_out_historical_read_deregisters_before_closed) + test(=handlers::req::tests::superseded_timeout_leaves_replacement_intact) + test(=handlers::req::tests::search_claim_retires_live_and_yields_to_replacement) + test(=handlers::req::tests::concurrent_claims_and_stale_teardowns_keep_the_last_owner) + test(=handlers::req::tests::timeout_closed_is_emitted_before_a_replacement_can_claim) + test(=handlers::req::tests::revoke_then_replacement_keeps_replacement_whole) + test(=handlers::req::tests::claims_after_connection_cleanup_are_refused) + test(=handlers::req::tests::dropped_terminal_frame_cancels_connection) + test(=handlers::req::tests::revoke_dropped_terminal_frame_cancels_connection) + test(=handlers::req::tests::count_fallback_covers_filters_that_can_match_private_deletions) + test(=handlers::req::tests::group_state_t_and_creator_tags_reach_sql_only_for_group_state_kinds) + (kind(bin) & (test(/^tests::/) + test(/^composition_tests::/) + test(/^env_filter_tests::/)) - test(/^tests::postgres_tests::/))'
+            -E 'test(/^api::admin::/) + test(/^push_runtime::/) + test(/^operator_listener::tests::/) + test(/^handlers::channel_authz::/) + test(/^handlers::moderation_authz::/) + test(/^handlers::side_effects::tests::/) + test(/^handlers::system_channels::tests::/) + test(/^nip11::tests::/) + test(/^storage_sweep::tests::/) + test(/^nip_fi_core::tests::/) + test(/^nip_fi_http::tests::/) + test(/^nip_fi_config::tests::/) + test(/^readiness::tests::/) + test(/^router::tests::/) + test(/^api::parse_query_tests::/) + test(/^api::git::transport::off_mode_precedence_tests::/) + test(/^audio::join::tests::/) + test(/^audio::handler::tests::/) + test(/^nip_fi_gate::tests::/) + test(/^nip_fi_session::tests::/) + test(/^nip_fi_shadow::tests::/) + test(/^nip_fi_shadow_session::tests::/) + test(/^startup_steps::tests::/) + test(/^telemetry::tests::/) + test(/^config::tests::/) + test(=state::tests::neither_a_confirmed_inactive_community_nor_a_failed_lookup_admits_the_socket) + test(=state::tests::periodic_revalidation_disconnects_inside_the_fenced_callback) + test(=state::tests::bare_community_disconnect_fails_closed_when_the_fence_is_unavailable) + test(=connection::tests::send_loop_sends_policy_close_when_community_is_archived) + test(=state::tests::community_disconnect_then_nip_fi_keeps_community_deleted_reason) + test(=state::tests::disconnect_community_wins_reason_losing_nip_fi_does_not_enqueue_frame) + test(=handlers::req::tests::timed_out_historical_read_deregisters_before_closed) + test(=handlers::req::tests::superseded_timeout_leaves_replacement_intact) + test(=handlers::req::tests::search_claim_retires_live_and_yields_to_replacement) + test(=handlers::req::tests::concurrent_claims_and_stale_teardowns_keep_the_last_owner) + test(=handlers::req::tests::timeout_closed_is_emitted_before_a_replacement_can_claim) + test(=handlers::req::tests::revoke_then_replacement_keeps_replacement_whole) + test(=handlers::req::tests::claims_after_connection_cleanup_are_refused) + test(=handlers::req::tests::dropped_terminal_frame_cancels_connection) + test(=handlers::req::tests::revoke_dropped_terminal_frame_cancels_connection) + test(=handlers::req::tests::count_fallback_covers_filters_that_can_match_private_deletions) + test(=handlers::req::tests::group_state_t_and_creator_tags_reach_sql_only_for_group_state_kinds) + (kind(bin) & (test(/^tests::/) + test(/^composition_tests::/) + test(/^env_filter_tests::/)) - test(/^tests::postgres_tests::/))'
         # Note on audio::join::tests scope: the full suite is infra-free (no DB,
         # no Redis). The infra-free audio/FI regression witnesses — bootstrap
         # ordering barrier, CommitConfirmed arm, pending-close invisibility,

NOSTR.md

@@ -136,6 +136,24 @@ that is not 1-64 characters from `a-z`, `0-9`, `.`, `:` and `-`, or a
 label equal to a channel type name (`stream`, `forum`, `dm`, `workflow`,
 `system`).
 
+**System channels.** A channel with `channel_type` `system` exists only for
+access control, for example to hold config that only an agent and its owner
+can read. It is an ordinary channel in every other way, but queries leave it
+out unless they ask for it:
+
+- A filter that does not name it, for example a REQ with no `#h`, a COUNT,
+  a NIP-50 search, or kind:39002 `#p:[me]`, does not return it.
+- A filter that names it with `#h`, or with `#d` on kinds 39000–39003, reads
+  it under the ordinary rules: the reader is a member, or it is open.
+- A filter on kinds 39000–39003 with `#t:["system"]` or any `#P` also
+  returns the system channels the reader can read, then matches as usual.
+  For example, `{kinds:[39000], #P:[<creator>], #t:[<label>]}` finds one
+  creator's labeled channel, system or not. Values in one tag filter
+  combine with OR, so `#t:["system", <label>]` does not mean "a system
+  channel with this label".
+
+Push and kind:44100/44101 membership notifications skip system channels.
+
 Events are stored **channel-scoped** so access control applies — private channel member lists are
 only visible to members. Discover groups via historical REQ:
 

crates/buzz-relay/src/handlers/system_channel_postgres_tests.rs

@@ -443,3 +443,123 @@ async fn system_channels_send_no_membership_notifications() {
     }
     assert_eq!(sorted(notified), vec![ordinary.to_string()]);
 }
+
+/// A group-state filter with `#t:["system"]` or any `#P` finds the system
+/// channels the reader can read, and still matches as usual: a creator plus
+/// a label finds that creator's one labeled channel.
+/// Mutation: make `filter_opts_in` return false → RED (discovery finds
+/// nothing). Drop the readability check from the opt-in → RED (an outsider
+/// finds the private channel).
+#[tokio::test]
+#[ignore = "requires Postgres and Redis"]
+async fn system_channels_are_found_by_type_or_creator() {
+    let f = fixture().await;
+    let owner = Keys::generate();
+    let outsider = Keys::generate();
+    let owner_hex = owner.public_key().to_hex();
+    let label = "agent-attention";
+
+    let private = create_channel(
+        &f.state,
+        &f.tenant,
+        &owner,
+        &[&["channel_type", "system"], &["t", label]],
+    )
+    .await;
+    let open = create_channel(&f.state, &f.tenant, &owner, &[&["channel_type", "system"]]).await;
+    edit_channel(
+        &f.state,
+        &f.tenant,
+        &owner,
+        open,
+        &[&["visibility", "open"]],
+    )
+    .await
+    .expect("open the system channel");
+    let ordinary = create_channel(&f.state, &f.tenant, &owner, &[]).await;
+    // Another creator's channel with the same label: #P must exclude it.
+    let lookalike = create_channel(&f.state, &f.tenant, &outsider, &[&["t", label]]).await;
+
+    let meta = |channel| group_state_id(&f, channel, 39000);
+    let (private_meta, open_meta, ordinary_meta, lookalike_meta) = (
+        meta(private).await,
+        meta(open).await,
+        meta(ordinary).await,
+        meta(lookalike).await,
+    );
+
+    // #t:["system"]: members find both; outsiders find only the open one.
+    assert_eq!(
+        sorted(req(&f, &owner, json!({"kinds": [39000], "#t": ["system"]})).await),
+        sorted(vec![private_meta.clone(), open_meta.clone()]),
+        "member #t:system"
+    );
+    assert_eq!(
+        req(&f, &outsider, json!({"kinds": [39000], "#t": ["system"]})).await,
+        vec![open_meta.clone()],
+        "outsider #t:system"
+    );
+    // A member lists its own system channels from kind:39002.
+    assert_eq!(
+        sorted(
+            req(
+                &f,
+                &owner,
+                json!({"kinds": [39002], "#p": [owner_hex], "#t": ["system"]})
+            )
+            .await
+        ),
+        sorted(vec![
+            group_state_id(&f, private, 39002).await,
+            group_state_id(&f, open, 39002).await,
+        ]),
+        "member 39002 #p:[me] #t:system"
+    );
+    // #P opts in, so a creator's channels include its system channels.
+    assert_eq!(
+        sorted(req(&f, &owner, json!({"kinds": [39000], "#P": [owner_hex]})).await),
+        sorted(vec![private_meta.clone(), open_meta.clone(), ordinary_meta]),
+        "member #P"
+    );
+    // Creator plus label finds exactly the one channel, in SQL.
+    let lookup = json!({"kinds": [39000], "#P": [owner_hex], "#t": [label], "limit": 1});
+    assert_eq!(
+        req(&f, &owner, lookup.clone()).await,
+        vec![private_meta.clone()],
+        "creator + label"
+    );
+    assert!(
+        req(&f, &outsider, lookup.clone()).await.is_empty(),
+        "an outsider cannot find a private system channel"
+    );
+    // A label alone is not an opt-in.
+    assert_eq!(
+        req(&f, &owner, json!({"kinds": [39000], "#t": [label]})).await,
+        vec![lookalike_meta],
+        "label alone"
+    );
+    // The same opt-in on COUNT and the HTTP bridge.
+    assert_eq!(
+        count(&f, &owner, json!({"kinds": [39000], "#t": ["system"]})).await,
+        2,
+        "COUNT #t:system"
+    );
+    assert_eq!(
+        bridge_ids(bridge(&f, &owner, "/query", json!([lookup])).await),
+        vec![private_meta],
+        "bridge creator + label"
+    );
+    assert_eq!(
+        bridge_count(
+            bridge(
+                &f,
+                &outsider,
+                "/count",
+                json!([{"kinds": [39000], "#t": ["system"]}])
+            )
+            .await
+        ),
+        1,
+        "bridge /count outsider #t:system"
+    );
+}

crates/buzz-relay/src/handlers/system_channels.rs

@@ -9,6 +9,12 @@
 //! - `#h` lists the channel, on any kinds;
 //! - `#d` lists the channel, on a filter whose kinds are all 39000–39003.
 //!
+//! A filter whose kinds are all 39000–39003 also opts in to every readable
+//! system channel when it has `#t:["system"]` (the channel type tag) or any
+//! `#P` (the creator tag). The filter then matches as usual, so
+//! `{kinds:[39000], #P:[creator], #t:[label]}` finds one creator's labeled
+//! channel, system or not. Existing clients send neither on these kinds.
+//!
 //! The addition is per filter. Another filter in the same request keeps the
 //! implicit set, so naming one system channel cannot widen a sibling filter.
 
@@ -52,13 +58,34 @@ fn named_channels(filter: &Filter) -> impl Iterator<Item = Uuid> + '_ {
         .filter_map(|value| value.parse::<Uuid>().ok())
 }
 
+/// Whether `filter` opts in to every readable system channel: its kinds are
+/// all group-state kinds, and it has `#t:["system"]` or any `#P`.
+fn filter_opts_in(filter: &Filter) -> bool {
+    if !filter_is_group_state_only(filter) {
+        return false;
+    }
+    let system_type = filter
+        .generic_tags
+        .get(&SingleLetterTag::lowercase(Alphabet::T))
+        .is_some_and(|values| {
+            values
+                .iter()
+                .any(|value| value == buzz_core::channel::ChannelType::System.as_str())
+        });
+    let creator = filter
+        .generic_tags
+        .get(&SingleLetterTag::uppercase(Alphabet::P))
+        .is_some_and(|values| !values.is_empty());
+    system_type || creator
+}
+
 /// Whether `filter` could need a system channel added to its scope.
 fn filter_needs_system_channels(filter: &Filter, implicit: &[Uuid]) -> bool {
-    named_channels(filter).any(|id| !implicit.contains(&id))
+    filter_opts_in(filter) || named_channels(filter).any(|id| !implicit.contains(&id))
 }
 
 /// The channels `filter` may read: the implicit set, plus the readable
-/// system channels the filter names.
+/// system channels the filter names, or all of them when it opts in.
 pub(crate) fn filter_channel_scope<'a>(
     filter: &Filter,
     implicit: &'a [Uuid],
@@ -67,8 +94,14 @@ pub(crate) fn filter_channel_scope<'a>(
     if readable_system.is_empty() {
         return Cow::Borrowed(implicit);
     }
+    let opted_in = filter_opts_in(filter);
     let mut extra: Vec<Uuid> = Vec::new();
-    for id in named_channels(filter) {
+    let candidates: Vec<Uuid> = if opted_in {
+        readable_system.to_vec()
+    } else {
+        named_channels(filter).collect()
+    };
+    for id in candidates {
         if readable_system.contains(&id) && !implicit.contains(&id) && !extra.contains(&id) {
             extra.push(id);
         }
@@ -83,8 +116,8 @@ pub(crate) fn filter_channel_scope<'a>(
 
 /// Load the system channels the reader can read, narrowed to the token's
 /// channel scope, when at least one filter could use them. Returns an empty
-/// set, without a database read, when no filter names a channel outside the
-/// implicit set.
+/// set, without a database read, when no filter opts in or names a channel
+/// outside the implicit set.
 pub(crate) async fn readable_system_channels_for_filters(
     state: &AppState,
     community: CommunityId,
@@ -163,6 +196,7 @@ fn move_system_channels(
 #[cfg(test)]
 mod tests {
     use super::*;
+    use nostr::Keys;
 
     fn filter(json: serde_json::Value) -> Filter {
         serde_json::from_value(json).expect("filter")
@@ -234,6 +268,45 @@ mod tests {
         );
     }
 
+    #[test]
+    fn system_type_or_creator_on_group_state_kinds_opts_in() {
+        let open = Uuid::new_v4();
+        let system = vec![Uuid::new_v4(), Uuid::new_v4()];
+        let implicit = vec![open];
+        let all = [vec![open], system.clone()].concat();
+        let creator = Keys::generate().public_key().to_hex();
+
+        for opt_in in [
+            serde_json::json!({"kinds": [39000], "#t": ["system"]}),
+            serde_json::json!({"kinds": [39002], "#p": [creator], "#t": ["system", "x"]}),
+            serde_json::json!({"kinds": [39000, 39003], "#P": [creator]}),
+            serde_json::json!({"kinds": [39000], "#P": [creator], "#t": ["agent-attention"]}),
+        ] {
+            let f = filter(opt_in.clone());
+            assert!(filter_needs_system_channels(&f, &implicit), "{opt_in}");
+            assert_eq!(
+                filter_channel_scope(&f, &implicit, &system).as_ref(),
+                all.as_slice(),
+                "{opt_in}"
+            );
+        }
+        for no_opt_in in [
+            // Labels alone, or other kinds, do not opt in.
+            serde_json::json!({"kinds": [39000], "#t": ["agent-attention"]}),
+            serde_json::json!({"kinds": [39000, 9], "#t": ["system"]}),
+            serde_json::json!({"kinds": [9], "#P": [creator]}),
+            serde_json::json!({"#t": ["system"]}),
+        ] {
+            let f = filter(no_opt_in.clone());
+            assert!(!filter_needs_system_channels(&f, &implicit), "{no_opt_in}");
+            assert_eq!(
+                filter_channel_scope(&f, &implicit, &system).as_ref(),
+                &[open],
+                "{no_opt_in}"
+            );
+        }
+    }
+
     #[test]
     fn lookup_runs_only_when_a_filter_names_a_channel_outside_the_implicit_set() {
         let open = Uuid::new_v4();

crates/buzz-relay/src/nip11.rs

@@ -242,7 +242,12 @@ impl RelayInfo {
             supported_nips.push(NIP_RELAY_MEMBERSHIP);
         }
 
-        let mut supported_extensions = vec!["nip-er".to_string(), "nip-ar".to_string()];
+        let mut supported_extensions = vec![
+            "nip-er".to_string(),
+            "nip-ar".to_string(),
+            // NIP-CL: identity tags, labels and system channels.
+            "nip-cl".to_string(),
+        ];
         let gif = gif_provider.map(|provider| {
             supported_extensions.push("buzz-gif".to_string());
             GifDescriptor {
@@ -661,6 +666,25 @@ mod tests {
             .contains(&"buzz-gif".to_string()));
     }
 
+    /// NIP-CL: clients refuse labels and system channels unless the relay
+    /// says it supports them, so every document must advertise it.
+    #[test]
+    fn advertises_nip_cl_extension() {
+        let info = RelayInfo::build(
+            None,
+            None,
+            RelayCapabilityFlags::default(),
+            DEFAULT_MAX_FRAME_BYTES,
+            None,
+            None,
+            None,
+        );
+        assert!(info
+            .supported_extensions
+            .expect("extensions")
+            .contains(&"nip-cl".to_string()));
+    }
+
     /// NIP-WP → NIP-11 mirror: a set workspace icon is served in the standard
     /// `icon` field; no icon (or a cleared, empty icon) omits the field
     /// entirely so the JSON matches pre-icon documents byte-for-byte.

docs/nips/NIP-CL.md

@@ -190,6 +190,41 @@ The relay MUST NOT send push notifications or kind:44100/44101 membership
 notifications for system channels. A client that received one would show the
 channel.
 
+### Asking for system channels
+
+A filter whose kinds are all 39000–39003 also returns the system channels that
+the reader can read when it has:
+
+- `"#t"` with the value `system`; or
+- any `"#P"`. A query for the channels of named creators returns a small
+  result, so it does not fill generic channel lists.
+
+The filter then matches as usual. A label alone does not ask for system
+channels.
+
+| To find | Filter |
+| --- | --- |
+| My system channels | `{"kinds": [39002], "#p": ["<me>"], "#t": ["system"]}`, then their kind:39000 by `#d` |
+| Open system channels | `{"kinds": [39000], "#t": ["system"]}` |
+| One creator's channel with a label, system or not | `{"kinds": [39000], "#P": ["<creator>"], "#t": ["<label>"]}` |
+
+`"#t": ["system", "<label>"]` means `system` **or** the label. It does not
+mean "a system channel with this label". Use `#P` and the label instead.
+
+Private system channels stay hidden from non-members, as for every private
+channel.
+
+## Advertising support
+
+Until this draft has an upstream integer NIP number, relays MUST NOT advertise
+it in NIP-11 `supported_nips`. A relay that supports this NIP adds `"nip-cl"`
+to its NIP-11 `supported_extensions` string array.
+
+A relay without this NIP drops label tags on create and refuses
+`channel_type=system`. So a client SHOULD NOT create a channel with labels or
+a system channel on a relay whose NIP-11 document does not contain `"nip-cl"`.
+Lookups by label on such a relay find nothing, which is safe.
+
 ## Trust
 
 A client that relies on these tags MUST check that the relay's key signed the

scripts/run-tests.sh

@@ -193,6 +193,9 @@ run_unit_tests() {
   run_test_step "buzz-relay system channel scope tests" \
     cargo test -p buzz-relay --lib handlers::system_channels::tests:: -- --nocapture
 
+  run_test_step "buzz-relay NIP-11 document tests" \
+    cargo test -p buzz-relay --lib nip11::tests:: -- --nocapture
+
   run_test_step "buzz-relay storage snapshot tests" \
     cargo test -p buzz-relay --lib storage_sweep::tests:: -- --nocapture