From 8130cd9fc7d8fce315eb34fb14207d915841e8c5 Mon Sep 17 00:00:00 2001 From: Brad Seiler Date: Sun, 8 Nov 2026 13:55:55 -0500 Subject: [PATCH] fix(cli): preserve attachments across successive edits (#9226) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary Editing attachment messages through `buzz messages edit` can remove their files from the effective message. Preserve the latest live imeta set across successive edits and add `--clear-attachments` for explicit removal. Add an SDK builder for complete replacement attachment sets while preserving the existing API. The relay provides a versioned effective-edit lookup on `/query` for kinds 9/40002/40008/45001/45003; media reads fold the same set. It uses the existing normalized-target index, fresh channel authorization, and a primary snapshot. Generic `#e` matching is unchanged. Mixed-case historical edits cannot restore removed files, including when another reference would allow republication. ### Related issue Fixes #3022. Follows #9077. Related open attempts: #3062 and #3065. ### Testing Published head: `c630c4ebf0b20a9618f4844629eec1e666699cbe`, through all mandatory push hooks. The governed-five snapshot passed all four PostgreSQL/HTTP acceptance tests with `RUST_MIN_STACK` unset and 3,333 affected-package tests. The committed test-only array correction passed clippy and the preflight rerun. Independent mutations detected dropped read/lookup kinds, the CLI two-kind split, and preflight ordering, channel and live-edit errors. The published follow-up adds forum preview retention/suppression and a deleted-original preflight fixture; both surviving mutations now fail as intended. Independent review verified the fixes. All four acceptance tests pass; the full DB/relay run had two NIP-FI failures that passed in isolation. The actual standalone MCP workflow passed against this relay and clean local app `5b583b8c69596353bc37e9f3938eca1aabcd081e`: 15 MCP edits, 10 direct publications and 60 member GET/HEAD checks across five isolated blob pairs. Both tested heads are now published. Relay hosted checks report 80 successful and 31 skipped, with none pending or failed; app hosted checks and human upload → edit → clear → edit → GET/HEAD confirmation remain pending. No human acceptance is claimed. ### Rollout The lookup, CLI preservation/clear and media reads use kinds 9/40002/40008/45001/45003. Deploy the relay before releasing the CLI and paired [MCP producer](https://github.com/block/buzz-app/pull/2937). New clients abort without publishing against an unsupported relay. Old CLI edits emit empty imeta and now revoke diff/forum attachments that previously remained readable. Other channel-tagged kinds retain the historical plain-edit path. No additional schema change is needed beyond migration 0086. Readiness remains blocked by mobile edits that omit imeta, the app UI fold that can restore older files, unaudited SDK/direct callers and unmeasured historical exposure. Run `docs/sql/private-media-edit-preflight.sql` for each community; producer updates do not repair signed historical edits. No enforce activation is authorized. `off` skips policy checks, `shadow` changes policy metrics while allowing policy failures, and `enforce` denies inaccessible references. Observe `buzz_media_publication_access_total{mode="shadow",reason="denied"}`. Prefer rolling back the client release while retaining the additive lookup. A relay rollback makes new CLI/MCP edits fail closed; already-published imeta remains valid under the older reference rules. Concurrent edits still use `(created_at,id)` without compare-and-swap. Generated with Codex --------- Signed-off-by: Brad Seiler Co-authored-by: Honor --- Cargo.lock | 1 + Justfile | 4 +- crates/buzz-cli/TESTING.md | 3 + crates/buzz-cli/src/commands/messages.rs | 31 +- crates/buzz-cli/src/commands/messages/edit.rs | 132 ++++ .../src/commands/messages/edit/tests.rs | 309 ++++++++ crates/buzz-cli/src/lib.rs | 5 +- crates/buzz-core/src/media.rs | 4 + crates/buzz-db/src/store/media.rs | 85 +- .../message_edit_state_postgres_tests.rs | 309 ++++++++ crates/buzz-db/src/store/mod.rs | 2 + crates/buzz-relay/Cargo.toml | 1 + crates/buzz-relay/src/api/bridge.rs | 4 + .../src/api/bridge/effective_edit.rs | 88 +++ .../src/api/media_access_postgres_tests.rs | 1 + .../cli_edit_postgres_tests.rs | 734 ++++++++++++++++++ crates/buzz-sdk/src/builders.rs | 57 +- docs/effective-message-edits.md | 61 ++ docs/operator-private-media.md | 111 ++- docs/sql/private-media-edit-preflight.sql | 22 + 20 files changed, 1920 insertions(+), 44 deletions(-) create mode 100644 crates/buzz-cli/src/commands/messages/edit.rs create mode 100644 crates/buzz-cli/src/commands/messages/edit/tests.rs create mode 100644 crates/buzz-db/src/store/message_edit_state_postgres_tests.rs create mode 100644 crates/buzz-relay/src/api/bridge/effective_edit.rs create mode 100644 crates/buzz-relay/src/api/media_access_postgres_tests/cli_edit_postgres_tests.rs create mode 100644 docs/effective-message-edits.md create mode 100644 docs/sql/private-media-edit-preflight.sql diff --git a/Cargo.lock b/Cargo.lock index 41a068ff88..665b0448d9 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1443,6 +1443,7 @@ dependencies = [ "base64 0.22.1", "buzz-audit", "buzz-auth", + "buzz-cli", "buzz-conformance", "buzz-core", "buzz-datastore-tracing", diff --git a/Justfile b/Justfile index 66fe30e07b..b7914bfb8a 100644 --- a/Justfile +++ b/Justfile @@ -680,9 +680,9 @@ test-integration: ./scripts/run-tests.sh integration # Enforce-mode upload/publication/read journey (requires PostgreSQL and Redis). -# Uses the isolated PostgreSQL runner, including emoji entry replacement checks. +# Uses the isolated PostgreSQL runner, including CLI edits and the target index. test-media-access: - ./scripts/postgres-test-run.sh -p buzz-relay --lib -E 'test(=api::media_access_postgres_tests::private_media_revocation_http_journey)' + ./scripts/postgres-test-run.sh -p buzz-relay -p buzz-db --lib -E 'test(/^api::media_access_postgres_tests::/) | test(effective_edit_uses_target_index) | test(historical_edit_preflight)' # Regenerate the model-capability normative corpus from the production Rust # resolver. The corpus is a golden snapshot, never hand-edited: this runs the diff --git a/crates/buzz-cli/TESTING.md b/crates/buzz-cli/TESTING.md index b8e4640dd8..2581071bb2 100644 --- a/crates/buzz-cli/TESTING.md +++ b/crates/buzz-cli/TESTING.md @@ -235,6 +235,9 @@ buzz messages search --query "CLI test" --limit 5 | jq . # messages edit buzz messages edit --event "$EVENT_ID" --content "Edited by CLI test" | jq . +# Explicit removal stays removed in a subsequent text edit. +buzz messages edit --event "$EVENT_ID" --content "Remove attached files" --clear-attachments | jq . +buzz messages edit --event "$EVENT_ID" --content "Still no attachments" | jq . # messages delete buzz messages delete --event "$REPLY_ID" | jq . diff --git a/crates/buzz-cli/src/commands/messages.rs b/crates/buzz-cli/src/commands/messages.rs index 2bf9434c83..58c6de8e89 100644 --- a/crates/buzz-cli/src/commands/messages.rs +++ b/crates/buzz-cli/src/commands/messages.rs @@ -1,3 +1,5 @@ +mod edit; + use buzz_sdk::{DeleteMessageOptions, DiffMeta, ThreadRef, VoteDirection}; use nostr::PublicKey; use uuid::Uuid; @@ -895,29 +897,6 @@ pub async fn cmd_delete_message( Ok(()) } -/// Edit a message you previously sent. -pub async fn cmd_edit_message( - client: &BuzzClient, - event_id: &str, - content: &str, -) -> Result<(), CliError> { - validate_hex64(event_id)?; - validate_content_size(content)?; - - // Resolve channel_id from the event's h-tag - let channel_uuid = resolve_channel_id(client, event_id).await?; - let target_eid = parse_event_id(event_id)?; - - let builder = buzz_sdk::build_edit(channel_uuid, target_eid, content) - .map_err(|e| CliError::Other(format!("build_edit failed: {e}")))?; - - let event = client.sign_event(builder).await?; - - let resp = client.submit_event(event).await?; - println!("{}", normalize_write_response(&resp)); - Ok(()) -} - /// Vote on a forum post or comment. pub async fn cmd_vote_on_post( client: &BuzzClient, @@ -1014,7 +993,11 @@ pub async fn dispatch( ) .await } - MessagesCmd::Edit { event, content } => cmd_edit_message(client, &event, &content).await, + MessagesCmd::Edit { + event, + content, + clear_attachments, + } => edit::edit_message(client, &event, &content, clear_attachments).await, MessagesCmd::Delete { event, action_id, diff --git a/crates/buzz-cli/src/commands/messages/edit.rs b/crates/buzz-cli/src/commands/messages/edit.rs new file mode 100644 index 0000000000..1f02ff4468 --- /dev/null +++ b/crates/buzz-cli/src/commands/messages/edit.rs @@ -0,0 +1,132 @@ +//! Resolve replacement attachment state before publishing an edit. +use super::{channel_id_from_event, fetch_event}; +use crate::{ + client::BuzzClient, + error::CliError, + validate::{parse_event_id, validate_content_size, validate_hex64}, +}; +use nostr::{Event, Timestamp}; + +pub(super) async fn edit_message( + client: &BuzzClient, + event_id: &str, + content: &str, + clear_attachments: bool, +) -> Result<(), CliError> { + validate_hex64(event_id)?; + validate_content_size(content)?; + let target = parse_event_id(event_id)?; + let raw = fetch_event(client, &target.to_hex()).await?; + let channel = channel_id_from_event(event_id, &raw)?; + let original: Event = serde_json::from_value(raw) + .map_err(|e| CliError::Other(format!("invalid edit target: {e}")))?; + if original.id != target { + return Err(CliError::Other( + "relay returned the wrong edit target".into(), + )); + } + // Keep this boundary aligned with media_reference_query and the relay lookup. + // Other channel-tagged events retain the historical plain-edit behavior. + if !buzz_core::media::EDIT_GOVERNED_KINDS.contains(&original.kind.as_u16()) { + if clear_attachments { + return Err(CliError::Usage(format!( + "attachments on kind {} are not edit-governed", + original.kind.as_u16() + ))); + } + let builder = buzz_sdk::build_edit(channel, target, content) + .map_err(|e| CliError::Other(format!("build_edit failed: {e}")))?; + let event = client.sign_event(builder).await?; + let resp = client.submit_event(event).await?; + println!("{}", crate::client::normalize_write_response(&resp)); + return Ok(()); + } + let raw = client + .query(&serde_json::json!({ + "kinds": buzz_core::media::EDIT_GOVERNED_KINDS, "ids": [target.to_hex()], "#h": [channel.to_string()], + "effective_edit": 1, "consistency": "strong", + })) + .await?; + let snapshot: serde_json::Value = serde_json::from_str(&raw) + .map_err(|e| CliError::Other(format!("invalid effective edit response: {e}")))?; + if snapshot.get("effective_edit") != Some(&serde_json::json!(1)) + || snapshot.get("edit").is_none() + { + return Err(CliError::Other( + "relay does not support effective_edit v1; upgrade the relay before editing".into(), + )); + } + let original: Event = serde_json::from_value(snapshot["original"].clone()) + .map_err(|e| CliError::Other(format!("invalid effective original: {e}")))?; + if original.id != target + || !buzz_core::media::EDIT_GOVERNED_KINDS.contains(&original.kind.as_u16()) + || channel_id_from_event(event_id, &snapshot["original"])? != channel + { + return Err(CliError::Other( + "effective edit response has the wrong original".into(), + )); + } + let edits: Vec = if snapshot["edit"].is_null() { + vec![] + } else { + vec![serde_json::from_value(snapshot["edit"].clone()) + .map_err(|e| CliError::Other(format!("invalid effective edit: {e}")))?] + }; + let effective = latest_edit(&original, &edits); + if !edits.is_empty() && effective.id == original.id { + return Err(CliError::Other( + "effective edit response has the wrong replacement".into(), + )); + } + let media = if clear_attachments { + Vec::new() + } else { + effective + .tags + .iter() + .filter(|t| t.as_slice().first().map(String::as_str) == Some("imeta")) + .map(|t| t.as_slice().to_vec()) + .collect() + }; + let now = Timestamp::now().as_secs(); + // Make successive edits effective even when issued in the same second. + // Refuse an unexpectedly future timeline instead of extending it forever. + let head = effective.created_at.as_secs(); + if head > now.saturating_add(60) { + return Err(CliError::Other( + "latest edit is too far in the future; retry when the clock catches up".into(), + )); + } + let created_at = now.max(head + 1); + let builder = buzz_sdk::build_edit_with_media(channel, target, content, &media) + .map_err(|e| CliError::Other(format!("build_edit failed: {e}")))? + .custom_created_at(Timestamp::from(created_at)); + let event = client.sign_event(builder).await?; + let resp = client.submit_event(event).await?; + println!("{}", crate::client::normalize_write_response(&resp)); + Ok(()) +} + +fn latest_edit<'a>(original: &'a Event, edits: &'a [Event]) -> &'a Event { + let channel = |event: &Event| { + event + .tags + .iter() + .find(|t| t.as_slice().first().map(String::as_str) == Some("h")) + .and_then(|t| t.content()) + .and_then(|id| uuid::Uuid::parse_str(id).ok()) + }; + edits + .iter() + .filter(|edit| { + edit.kind.as_u16() == 40003 + && channel(edit) == channel(original) + && buzz_core::media::first_edit_target(edit).as_deref() + == Some(original.id.as_bytes().as_slice()) + }) + .max_by_key(|edit| (edit.created_at, edit.id)) + .unwrap_or(original) +} + +#[cfg(test)] +mod tests; diff --git a/crates/buzz-cli/src/commands/messages/edit/tests.rs b/crates/buzz-cli/src/commands/messages/edit/tests.rs new file mode 100644 index 0000000000..b6033b84d8 --- /dev/null +++ b/crates/buzz-cli/src/commands/messages/edit/tests.rs @@ -0,0 +1,309 @@ +use super::*; +use axum::{extract::State, routing::post, Json, Router}; +use nostr::{EventBuilder, Keys, Kind, Tag}; +use std::sync::{Arc, Mutex}; +use uuid::Uuid; + +fn message(keys: &Keys, channel: Uuid, media: &[Vec]) -> Event { + EventBuilder::new(Kind::Custom(9), "original") + .tags( + std::iter::once(Tag::parse(["h", &channel.to_string()]).unwrap()) + .chain(media.iter().map(|t| Tag::parse(t.clone()).unwrap())), + ) + .custom_created_at(Timestamp::from(Timestamp::now().as_secs() - 20)) + .sign_with_keys(keys) + .unwrap() +} +fn edit(keys: &Keys, original: &Event, at: u64, extra: &[Vec]) -> Event { + let channel = original + .tags + .iter() + .find(|t| t.as_slice()[0] == "h") + .unwrap() + .content() + .unwrap(); + EventBuilder::new(Kind::Custom(40003), "edit") + .tags( + std::iter::once(Tag::parse(["h", channel]).unwrap()) + .chain(extra.iter().map(|t| Tag::parse(t.clone()).unwrap())), + ) + .custom_created_at(Timestamp::from(at)) + .sign_with_keys(keys) + .unwrap() +} +fn imeta(name: &str) -> Vec { + vec![ + "imeta".into(), + format!("url https://example.com/{name}"), + "m image/png".into(), + "alt original label".into(), + "blurhash complete metadata".into(), + ] +} + +#[test] +fn latest_uses_server_order_target_and_channel() { + let keys = Keys::generate(); + let original = message(&keys, Uuid::new_v4(), &[imeta("original")]); + let target = vec!["e".into(), original.id.to_hex()]; + let now = Timestamp::now().as_secs(); + let older = edit(&keys, &original, now - 1, &[target.clone(), imeta("older")]); + let cleared = edit(&keys, &original, now, std::slice::from_ref(&target)); + let tied = edit(&keys, &original, now, &[target.clone(), imeta("tie")]); + let winner = if cleared.id > tied.id { + &cleared + } else { + &tied + }; + assert_eq!( + latest_edit(&original, &[older.clone(), cleared.clone(), tied.clone()]).id, + winner.id + ); + // First valid e wins; a later matching tag must not select this edit. + let decoy = edit( + &keys, + &original, + now + 1, + &[ + vec!["e".into(), "f".repeat(64)], + target.clone(), + imeta("decoy"), + ], + ); + let wrong_channel_original = message(&keys, Uuid::new_v4(), &[]); + let wrong_channel = edit( + &keys, + &wrong_channel_original, + now + 2, + std::slice::from_ref(&target), + ); + assert_eq!( + latest_edit(&original, &[older, cleared.clone(), decoy, wrong_channel]).id, + cleared.id + ); + let mixed = original + .id + .to_hex() + .chars() + .enumerate() + .map(|(i, c)| { + if i % 2 == 0 { + c.to_ascii_uppercase() + } else { + c + } + }) + .collect::(); + let valid_after_invalid = edit( + &keys, + &original, + now + 3, + &[vec!["e".into(), "invalid".into()], vec!["e".into(), mixed]], + ); + assert_eq!( + latest_edit(&original, std::slice::from_ref(&valid_after_invalid)).id, + valid_after_invalid.id + ); + assert_eq!(latest_edit(&original, &[]).id, original.id); +} + +#[derive(Clone)] +struct Relay { + events: Arc>>, + submitted: Arc>>, + fail_history: bool, + effective_queries: Arc, + ignore_extension: Arc, +} +async fn relay(events: Vec, fail_history: bool) -> (String, Relay) { + let state = Relay { + events: Arc::new(Mutex::new(events)), + submitted: Default::default(), + fail_history, + effective_queries: Default::default(), + ignore_extension: Default::default(), + }; + let app = Router::new() + .route("/query", post(|State(state): State, Json(filters): Json>| async move { + let filter = &filters[0]; + if filter.get("effective_edit").is_some() { + state.effective_queries.fetch_add(1, std::sync::atomic::Ordering::SeqCst); + } + if state.fail_history && filter.get("kinds").is_some() { + return (axum::http::StatusCode::FORBIDDEN, Json(serde_json::json!({"error":"denied"}))); + } + if filter.get("effective_edit").is_some() && !state.ignore_extension.load(std::sync::atomic::Ordering::SeqCst) { + assert_eq!(filter["consistency"], "strong"); + assert_eq!(filter, &serde_json::json!({ + "kinds":[9,40002,40008,45001,45003], "ids":filter["ids"], + "#h":filter["#h"], "effective_edit":1, "consistency":"strong" + })); + let events = state.events.lock().unwrap(); + let original = events.iter().find(|event| serde_json::json!(event.id.to_hex()) == filter["ids"][0]).unwrap(); + let latest = events.iter().filter(|event| event.kind.as_u16() == 40003 && buzz_core::media::first_edit_target(event).as_deref() == Some(original.id.as_bytes().as_slice())) + .max_by_key(|event| (event.created_at,event.id)); + return (axum::http::StatusCode::OK, Json(serde_json::json!({"effective_edit":1,"original":original,"edit":latest}))); + } + let events: Vec = state.events.lock().unwrap().iter().filter(|event| { + filter["ids"].as_array().is_none_or(|ids| ids.contains(&serde_json::json!(event.id.to_hex()))) + && filter["kinds"].as_array().is_none_or(|kinds| kinds.contains(&serde_json::json!(event.kind.as_u16()))) + }).cloned().collect(); + (axum::http::StatusCode::OK, Json(serde_json::json!(events))) + })) + .route("/events", post(|State(state): State, Json(event): Json| async move { + event.verify().unwrap(); + state.events.lock().unwrap().push(event.clone()); + state.submitted.lock().unwrap().push(event.clone()); + Json(serde_json::json!({"event_id":event.id.to_hex(), "accepted":true})) + })).with_state(state.clone()); + let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); + let url = format!("http://{}", listener.local_addr().unwrap()); + tokio::spawn(async move { + axum::serve(listener, app).await.unwrap(); + }); + (url, state) +} +fn client(url: String, keys: Keys) -> BuzzClient { + BuzzClient::new(url, crate::credentials::Credentials::from_keys(keys)).unwrap() +} + +#[tokio::test] +async fn command_retains_latest_full_set_then_keeps_clear_across_same_second_edits() { + let keys = Keys::generate(); + let channel = Uuid::new_v4(); + let original = message(&keys, channel, &[imeta("removed"), imeta("kept")]); + let replacement = edit( + &keys, + &original, + Timestamp::now().as_secs(), + &[vec!["e".into(), original.id.to_hex()], imeta("kept")], + ); + let (url, state) = relay(vec![original.clone(), replacement.clone()], false).await; + let client = client(url, keys); + for index in 0..4 { + edit_message( + &client, + &original.id.to_hex(), + &format!("edit {index}"), + index == 2, + ) + .await + .unwrap(); + } + let submitted = state.submitted.lock().unwrap(); + let mut previous = replacement.created_at; + for (index, event) in submitted.iter().enumerate() { + let tags: Vec<_> = event + .tags + .iter() + .filter(|t| t.as_slice()[0] == "imeta") + .map(|t| t.as_slice().to_vec()) + .collect(); + assert_eq!( + tags, + if index < 2 { + vec![imeta("kept")] + } else { + vec![] + } + ); + assert!(event.created_at > previous); + previous = event.created_at; + } +} + +#[tokio::test] +async fn command_aborts_on_unreadable_history_or_future_head() { + for fail_history in [true, false] { + let keys = Keys::generate(); + let original = message(&keys, Uuid::new_v4(), &[imeta("kept")]); + let poisoned = edit( + &keys, + &original, + Timestamp::now().as_secs() + 1000, + &[vec!["e".into(), original.id.to_hex()]], + ); + let (url, state) = relay(vec![original.clone(), poisoned], fail_history).await; + let client = client(url, keys); + assert!(edit_message(&client, &original.id.to_hex(), "new", false) + .await + .is_err()); + assert!(state.submitted.lock().unwrap().is_empty()); + } +} + +#[tokio::test] +async fn command_refuses_an_older_relay_that_ignores_the_extension() { + for kind in [9, 40002, 40008, 45001, 45003] { + let keys = Keys::generate(); + let channel = Uuid::new_v4(); + let original = EventBuilder::new(Kind::Custom(kind), "original") + .tags([ + Tag::parse(["h", &channel.to_string()]).unwrap(), + Tag::parse(imeta("kept")).unwrap(), + ]) + .sign_with_keys(&keys) + .unwrap(); + let (url, state) = relay(vec![original.clone()], false).await; + state + .ignore_extension + .store(true, std::sync::atomic::Ordering::SeqCst); + let client = client(url, keys); + let error = edit_message(&client, &original.id.to_hex(), "new", false) + .await + .unwrap_err(); + assert!( + error.to_string().contains("upgrade the relay"), + "kind {kind}" + ); + assert!(state.submitted.lock().unwrap().is_empty()); + } +} + +#[tokio::test] +async fn other_channel_kinds_keep_plain_edits_on_older_relays() { + for kind in [40003, 7, 45002, 12345] { + let keys = Keys::generate(); + let channel = Uuid::new_v4(); + let original = EventBuilder::new(Kind::Custom(kind), "original") + .tags([ + Tag::parse(["h", &channel.to_string()]).unwrap(), + Tag::parse(imeta("original")).unwrap(), + ]) + .sign_with_keys(&keys) + .unwrap(); + let (url, state) = relay(vec![original.clone()], false).await; + state + .ignore_extension + .store(true, std::sync::atomic::Ordering::SeqCst); + let client = client(url, keys); + for content in ["first edit", "second edit"] { + edit_message(&client, &original.id.to_hex(), content, false) + .await + .unwrap(); + } + let error = edit_message(&client, &original.id.to_hex(), "clear", true) + .await + .unwrap_err(); + assert!(error + .to_string() + .contains(&format!("attachments on kind {kind} are not edit-governed"))); + assert_eq!( + state + .effective_queries + .load(std::sync::atomic::Ordering::SeqCst), + 0 + ); + let submitted = state.submitted.lock().unwrap(); + assert_eq!(submitted.len(), 2, "clear must not publish"); + for (event, content) in submitted.iter().zip(["first edit", "second edit"]) { + assert_eq!(event.kind.as_u16(), 40003); + assert_eq!(event.content, content); + assert_eq!( + buzz_core::media::first_edit_target(event).as_deref(), + Some(original.id.as_bytes().as_slice()) + ); + assert!(!event.tags.iter().any(|tag| tag.as_slice()[0] == "imeta")); + } + } +} diff --git a/crates/buzz-cli/src/lib.rs b/crates/buzz-cli/src/lib.rs index 973fd0d2a5..fc9f99c30a 100644 --- a/crates/buzz-cli/src/lib.rs +++ b/crates/buzz-cli/src/lib.rs @@ -674,7 +674,7 @@ pub enum MessagesCmd { #[arg(long)] reply_to: Option, }, - /// Edit a previously sent message + /// Edit a previously sent message, retaining its current attachments Edit { /// Event ID of the message to edit (64-char hex) #[arg(long)] @@ -682,6 +682,9 @@ pub enum MessagesCmd { /// New message content #[arg(long)] content: String, + /// Remove all attachments on kinds 9/40002/40008/45001/45003 (later edits keep them removed) + #[arg(long)] + clear_attachments: bool, }, /// Delete a message by event ID Delete { diff --git a/crates/buzz-core/src/media.rs b/crates/buzz-core/src/media.rs index 1247f15719..d88c4337a7 100644 --- a/crates/buzz-core/src/media.rs +++ b/crates/buzz-core/src/media.rs @@ -2,6 +2,10 @@ use nostr::Event; use std::collections::BTreeSet; +/// Original event kinds whose latest live edit replaces attachment references. +/// Shared by clients, the effective-edit lookup and media-read authorization. +pub const EDIT_GOVERNED_KINDS: [u16; 5] = [9, 40002, 40008, 45001, 45003]; + /// Parse a local media URL, returning its parent hash (also for thumbnails). /// Only the tenant's canonical base and root-relative paths count. pub fn local_hash(value: &str, base: &str) -> Option { diff --git a/crates/buzz-db/src/store/media.rs b/crates/buzz-db/src/store/media.rs index e6218ae2c3..9f5551ab2e 100644 --- a/crates/buzz-db/src/store/media.rs +++ b/crates/buzz-db/src/store/media.rs @@ -15,10 +15,22 @@ pub(crate) const MEDIA_EDIT_TARGET_SQL: &str = r#"decode(lower(jsonb_path_query_ pub(super) const MEDIA_LOOKUP_SETTINGS_SQL: &str = "SET LOCAL statement_timeout = '2s'; SET LOCAL jit = off"; +fn edit_governed_kinds_sql() -> String { + buzz_core::media::EDIT_GOVERNED_KINDS + .iter() + .map(u16::to_string) + .collect::>() + .join(",") +} + fn edit_target(alias: &str) -> String { MEDIA_EDIT_TARGET_SQL.replacen("(tags,", &format!("({alias}.tags,"), 1) } +pub(super) fn latest_message_edit_sql() -> String { + format!("SELECT e.* FROM events e WHERE e.community_id=$1 AND e.channel_id=$2 AND e.kind=40003 AND e.deleted_at IS NULL AND {}=$3 ORDER BY e.created_at DESC,e.id DESC LIMIT 1", edit_target("e")) +} + /// Result of a complete live-reference lookup, for bounded denial metrics. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub enum MediaAccess { @@ -32,7 +44,69 @@ pub enum MediaAccess { Restricted, } +/// Original message and its latest live edit in one primary-database snapshot. +#[derive(Debug)] +pub struct MessageEditState { + /// Live original, authorized for the reader's explicitly named channel. + pub original: nostr::Event, + /// Complete replacement, if a live edit exists. + pub edit: Option, +} + impl Db { + /// Read effective edit metadata using the normalized-target + /// index. Channel membership, source liveness and edit selection share a + /// repeatable-read primary snapshot; no replica or membership cache is used. + pub async fn message_edit_state( + &self, + community: CommunityId, + channel: uuid::Uuid, + target: &[u8], + reader: &[u8], + ) -> Result> { + let mut tx = self.pool.begin().await?; + sqlx::query("SET TRANSACTION ISOLATION LEVEL REPEATABLE READ, READ ONLY") + .execute(&mut *tx) + .await?; + sqlx::raw_sql(MEDIA_LOOKUP_SETTINGS_SQL) + .execute(&mut *tx) + .await?; + // Mirrors channel_members::{get_accessible_channel_ids, + // get_readable_named_only_channels}. + // The channel is explicitly named by this read, so open unlisted and + // named-only channels follow the normal explicit-channel read policy. + let original_sql = format!("SELECT e.* FROM events e JOIN channels c ON c.community_id=e.community_id AND c.id=e.channel_id WHERE e.community_id=$1 AND e.channel_id=$2 AND e.id=$3 AND e.kind IN ({}) AND e.deleted_at IS NULL AND c.deleted_at IS NULL AND (c.visibility='open' OR EXISTS (SELECT 1 FROM channel_members cm WHERE cm.community_id=c.community_id AND cm.channel_id=c.id AND cm.pubkey=$4 AND cm.removed_at IS NULL))", edit_governed_kinds_sql()); + let original = sqlx::query(sqlx::AssertSqlSafe(original_sql)) + .bind(community.as_uuid()) + .bind(channel) + .bind(target) + .bind(reader) + .fetch_optional(&mut *tx) + .await?; + let Some(original) = original else { + tx.commit().await?; + return Ok(None); + }; + let original = super::event::row_to_stored_event(original)? + .ok_or_else(|| DbError::InvalidData("unreadable original message row".into()))? + .event; + let sql = latest_message_edit_sql(); + let edit = sqlx::query(sqlx::AssertSqlSafe(sql)) + .bind(community.as_uuid()) + .bind(channel) + .bind(target) + .fetch_optional(&mut *tx) + .await? + .map(|row| { + super::event::row_to_stored_event(row)? + .map(|stored| stored.event) + .ok_or_else(|| DbError::InvalidData("unreadable latest edit row".into())) + }) + .transpose()?; + tx.commit().await?; + Ok(Some(MessageEditState { original, edit })) + } + /// Record possession after the upload pipeline verified the actual bytes. /// Independent of optional audit and moderation records; admission fences deletion. #[datastore_span(name = "record_media_upload", system = "postgresql")] @@ -242,12 +316,13 @@ pub(super) fn media_reference_query( ); query .push_bind(pubkey) - .push( + .push(format!( ")))) AND (e.kind <> 40003 OR EXISTS (SELECT 1 FROM events original \ WHERE original.community_id=e.community_id AND original.channel_id=e.channel_id \ - AND original.deleted_at IS NULL AND original.kind IN (9,40002) \ + AND original.deleted_at IS NULL AND original.kind IN ({}) \ AND ", - ) + edit_governed_kinds_sql(), + )) .push(edit_target("e")) .push( "=original.id AND NOT EXISTS (SELECT 1 FROM events newer \ @@ -260,11 +335,11 @@ pub(super) fn media_reference_query( if probe == 3 { // The renderer retains snapshots unless the original or any live // edit suppresses them; later body edits never undo suppression. - query.push("AND e.kind IN (9,40002) AND NOT EXISTS (SELECT 1 FROM jsonb_array_elements(e.tags) marker(value) WHERE marker.value='[\"link-preview\",\"none\"]'::jsonb) AND EXISTS (SELECT 1 FROM events edit WHERE edit.community_id=e.community_id AND edit.channel_id=e.channel_id AND edit.kind=40003 AND edit.deleted_at IS NULL AND ") + query.push(format!("AND e.kind IN ({}) AND NOT EXISTS (SELECT 1 FROM jsonb_array_elements(e.tags) marker(value) WHERE marker.value='[\"link-preview\",\"none\"]'::jsonb) AND EXISTS (SELECT 1 FROM events edit WHERE edit.community_id=e.community_id AND edit.channel_id=e.channel_id AND edit.kind=40003 AND edit.deleted_at IS NULL AND ", edit_governed_kinds_sql())) .push(edit_target("edit")).push("=e.id) AND NOT EXISTS (SELECT 1 FROM events suppressed WHERE suppressed.community_id=e.community_id AND suppressed.channel_id=e.channel_id AND suppressed.kind=40003 AND suppressed.deleted_at IS NULL AND ") .push(edit_target("suppressed")).push("=e.id AND EXISTS (SELECT 1 FROM jsonb_array_elements(suppressed.tags) marker(value) WHERE marker.value='[\"link-preview\",\"none\"]'::jsonb))"); } else { - query.push("AND (e.kind NOT IN (9,40002) OR NOT EXISTS (SELECT 1 FROM events edit WHERE edit.community_id=e.community_id AND edit.channel_id=e.channel_id AND edit.kind=40003 AND edit.deleted_at IS NULL AND ") + query.push(format!("AND (e.kind NOT IN ({}) OR NOT EXISTS (SELECT 1 FROM events edit WHERE edit.community_id=e.community_id AND edit.channel_id=e.channel_id AND edit.kind=40003 AND edit.deleted_at IS NULL AND ", edit_governed_kinds_sql())) .push(edit_target("edit")).push("=e.id))"); } query diff --git a/crates/buzz-db/src/store/message_edit_state_postgres_tests.rs b/crates/buzz-db/src/store/message_edit_state_postgres_tests.rs new file mode 100644 index 0000000000..a9d99247d8 --- /dev/null +++ b/crates/buzz-db/src/store/message_edit_state_postgres_tests.rs @@ -0,0 +1,309 @@ +use super::media::latest_message_edit_sql; +use crate::{ + channel::{ChannelType, ChannelVisibility}, + Db, +}; +use nostr::{EventBuilder, Keys, Kind, Tag, Timestamp}; + +#[tokio::test] +#[ignore = "requires Postgres"] +async fn effective_edit_uses_target_index_and_fresh_channel_authorization() { + let pool = sqlx::PgPool::connect(&crate::test_support::database_url()) + .await + .unwrap(); + let db = Db::from_pool(pool.clone()); + let community = db + .ensure_configured_community("effective-edit.example") + .await + .unwrap() + .id; + let owner = Keys::generate(); + let outsider = Keys::generate(); + db.ensure_user(community, owner.public_key().as_bytes()) + .await + .unwrap(); + let channel = db + .create_channel( + community, + "edit", + ChannelType::Stream, + ChannelVisibility::Private, + None, + owner.public_key().as_bytes(), + None, + ) + .await + .unwrap() + .id; + let original = EventBuilder::new(Kind::Custom(45001), "original") + .tags([Tag::parse(["h", &channel.to_string()]).unwrap()]) + .sign_with_keys(&owner) + .unwrap(); + db.insert_event(community, &original, Some(channel)) + .await + .unwrap(); + let tags = vec![ + Tag::parse(["h", &channel.to_string()]).unwrap(), + Tag::parse(["e", &original.id.to_hex().to_uppercase()]).unwrap(), + ]; + let edit = EventBuilder::new(Kind::Custom(40003), "replacement") + .tags(tags) + .custom_created_at(Timestamp::from(Timestamp::now().as_secs() + 1)) + .sign_with_keys(&owner) + .unwrap(); + db.insert_event(community, &edit, Some(channel)) + .await + .unwrap(); + // More unrelated edits than the old channel-wide cap; they must not + // affect lookup cost or prevent editing this message. Synthetic IDs are + // intentional: these rows are unrelated fixtures and must never be read. + sqlx::query("INSERT INTO events (community_id,id,pubkey,created_at,kind,tags,content,sig,channel_id) SELECT $1,decode(lpad(to_hex(n),64,'0'),'hex'),$2,now(),40003,$3,'unrelated',$4,$5 FROM generate_series(1,10001) n") + .bind(community.as_uuid()).bind(owner.public_key().as_bytes().as_slice()) + .bind(serde_json::json!([["h",channel.to_string()],["e","f".repeat(64)]])) + .bind(edit.sig.serialize().as_slice()).bind(channel).execute(&pool).await.unwrap(); + sqlx::query("ANALYZE events").execute(&pool).await.unwrap(); + let snapshot = db + .message_edit_state( + community, + channel, + original.id.as_bytes(), + owner.public_key().as_bytes(), + ) + .await + .unwrap() + .unwrap(); + assert_eq!(snapshot.original.id, original.id); + assert_eq!(snapshot.edit.unwrap().id, edit.id); + assert!(db + .message_edit_state( + community, + channel, + original.id.as_bytes(), + outsider.public_key().as_bytes() + ) + .await + .unwrap() + .is_none()); + db.ensure_user(community, outsider.public_key().as_bytes()) + .await + .unwrap(); + db.add_member( + community, + channel, + outsider.public_key().as_bytes(), + crate::channel::MemberRole::Member, + Some(owner.public_key().as_bytes()), + ) + .await + .unwrap(); + assert!(db + .message_edit_state( + community, + channel, + original.id.as_bytes(), + outsider.public_key().as_bytes() + ) + .await + .unwrap() + .is_some()); + db.remove_member( + community, + channel, + outsider.public_key().as_bytes(), + owner.public_key().as_bytes(), + ) + .await + .unwrap(); + assert!(db + .message_edit_state( + community, + channel, + original.id.as_bytes(), + outsider.public_key().as_bytes() + ) + .await + .unwrap() + .is_none()); + let sql = format!("EXPLAIN (FORMAT JSON) {}", latest_message_edit_sql()); + let plan: serde_json::Value = sqlx::query_scalar(sqlx::AssertSqlSafe(sql)) + .bind(community.as_uuid()) + .bind(channel) + .bind(original.id.as_bytes().as_slice()) + .fetch_one(&pool) + .await + .unwrap(); + let plan = plan.to_string(); + let child_indexes: Vec = sqlx::query_scalar("SELECT child.relname FROM pg_inherits inheritance JOIN pg_class child ON child.oid=inheritance.inhrelid WHERE inheritance.inhparent='idx_events_media_edit_target'::regclass") + .fetch_all(&pool).await.unwrap(); + assert!( + child_indexes.iter().any(|index| plan.contains(index)), + "{plan}" + ); + assert!( + plan.contains("decode(lower("), + "target must be an index condition: {plan}" + ); + assert!(!plan.contains("Seq Scan"), "{plan}"); + db.soft_delete_event_and_update_thread(community, edit.id.as_bytes(), None, None) + .await + .unwrap(); + assert!(db + .message_edit_state( + community, + channel, + original.id.as_bytes(), + owner.public_key().as_bytes() + ) + .await + .unwrap() + .unwrap() + .edit + .is_none()); + db.soft_delete_event_and_update_thread(community, original.id.as_bytes(), None, None) + .await + .unwrap(); + assert!(db + .message_edit_state( + community, + channel, + original.id.as_bytes(), + owner.public_key().as_bytes() + ) + .await + .unwrap() + .is_none()); +} + +#[tokio::test] +#[ignore = "requires Postgres"] +async fn historical_edit_preflight_matches_selected_live_edit() { + let sql = include_str!("../../../../docs/sql/private-media-edit-preflight.sql"); + // The checked-in operator query uses the exact normalized target expression. + assert!(sql.contains(super::media::MEDIA_EDIT_TARGET_SQL)); + let pool = sqlx::PgPool::connect(&crate::test_support::database_url()) + .await + .unwrap(); + let db = Db::from_pool(pool.clone()); + let community = db + .ensure_configured_community("preflight.example") + .await + .unwrap() + .id; + let owner = Keys::generate(); + db.ensure_user(community, owner.public_key().as_bytes()) + .await + .unwrap(); + let mut channels = Vec::new(); + for name in ["originals", "decoys"] { + channels.push( + db.create_channel( + community, + name, + ChannelType::Stream, + ChannelVisibility::Private, + None, + owner.public_key().as_bytes(), + None, + ) + .await + .unwrap() + .id, + ); + } + let now = Timestamp::now().as_secs(); + for (index, kind, scenario, denied) in [ + (1, 45001, "empty", true), + (2, 45003, "later-retained", false), + (3, 40008, "deleted", false), + (4, 45001, "mixed", true), + (5, 45003, "cross-channel", false), + (6, 9, "control", true), + (7, 45001, "deleted-original", true), + ] { + let hash = format!("{index:064x}"); + let media = [vec![ + "imeta".into(), + format!("url https://preflight.example/media/{hash}"), + format!("x {hash}"), + ]]; + let original = EventBuilder::new(Kind::Custom(kind), scenario) + .tags([ + Tag::parse(["h", &channels[0].to_string()]).unwrap(), + Tag::parse(media[0].clone()).unwrap(), + ]) + .custom_created_at(Timestamp::from(now - 20)) + .sign_with_keys(&owner) + .unwrap(); + db.insert_event(community, &original, Some(channels[0])) + .await + .unwrap(); + let channel = if scenario == "cross-channel" { + channels[1] + } else { + channels[0] + }; + let target = if scenario == "mixed" { + original.id.to_hex().to_uppercase() + } else { + original.id.to_hex() + }; + let empty = EventBuilder::new(Kind::Custom(40003), "empty") + .tags([ + Tag::parse(["h", &channel.to_string()]).unwrap(), + Tag::parse(["e", &target]).unwrap(), + ]) + .custom_created_at(Timestamp::from(now - 2)) + .sign_with_keys(&owner) + .unwrap(); + db.insert_event(community, &empty, Some(channel)) + .await + .unwrap(); + if scenario == "later-retained" { + let retained = EventBuilder::new(Kind::Custom(40003), "retained") + .tags([ + Tag::parse(["h", &channels[0].to_string()]).unwrap(), + Tag::parse(["e", &original.id.to_hex()]).unwrap(), + Tag::parse(media[0].clone()).unwrap(), + ]) + .custom_created_at(Timestamp::from(now - 1)) + .sign_with_keys(&owner) + .unwrap(); + db.insert_event(community, &retained, Some(channels[0])) + .await + .unwrap(); + } + if scenario == "deleted" { + db.soft_delete_event_and_update_thread(community, empty.id.as_bytes(), None, None) + .await + .unwrap(); + } + if scenario == "deleted-original" { + db.soft_delete_event_and_update_thread(community, original.id.as_bytes(), None, None) + .await + .unwrap(); + } + let access = db + .media_access( + community, + &hash, + "", + "https://preflight.example/media", + owner.public_key().as_bytes(), + false, + |_, _| true, + ) + .await + .unwrap(); + assert_eq!( + access == super::media::MediaAccess::Allowed, + !denied, + "{scenario}" + ); + } + let counts: Vec<(i32, i64)> = sqlx::query_as(sql) + .bind(community.as_uuid()) + .fetch_all(&pool) + .await + .unwrap(); + assert_eq!(counts, vec![(45001, 2)]); +} diff --git a/crates/buzz-db/src/store/mod.rs b/crates/buzz-db/src/store/mod.rs index f9448aae51..d1cb647694 100644 --- a/crates/buzz-db/src/store/mod.rs +++ b/crates/buzz-db/src/store/mod.rs @@ -119,3 +119,5 @@ pub mod media; #[cfg(test)] mod media_postgres_tests; +#[cfg(test)] +mod message_edit_state_postgres_tests; diff --git a/crates/buzz-relay/Cargo.toml b/crates/buzz-relay/Cargo.toml index 53cf3f4dc7..08f542b4c6 100644 --- a/crates/buzz-relay/Cargo.toml +++ b/crates/buzz-relay/Cargo.toml @@ -95,6 +95,7 @@ async-compression = { version = "0.4.42", features = ["tokio", "gzip"] } dev = ["buzz-auth/dev"] [dev-dependencies] +buzz-cli = { path = "../buzz-cli" } # Generate ephemeral P-256 signing fixtures; already in the runtime dependency graph. ring = "0.17" # The mesh-llm smoke harnesses moved to `crates/buzz-mesh-smoke`. A diff --git a/crates/buzz-relay/src/api/bridge.rs b/crates/buzz-relay/src/api/bridge.rs index 0b6c481a88..f5d7f18cdd 100644 --- a/crates/buzz-relay/src/api/bridge.rs +++ b/crates/buzz-relay/src/api/bridge.rs @@ -3,6 +3,7 @@ //! These endpoints provide HTTP access to the relay's Nostr protocol, //! authenticated via NIP-98 signed events. +mod effective_edit; mod membership_snapshot; mod read_state_snapshot; pub(crate) mod search_slots; @@ -1788,6 +1789,9 @@ async fn query_member_events( })? .map(|events| Json(Value::Array(events))); } + if effective_edit::requested(&raw_filters) { + return effective_edit::query(state, tenant, &pubkey, &raw_filters).await; + } if read_state_snapshot::requested(&raw_filters) { return read_state_snapshot::query(state, tenant, &pubkey, &raw_filters).await; } diff --git a/crates/buzz-relay/src/api/bridge/effective_edit.rs b/crates/buzz-relay/src/api/bridge/effective_edit.rs new file mode 100644 index 0000000000..b7c84284ff --- /dev/null +++ b/crates/buzz-relay/src/api/bridge/effective_edit.rs @@ -0,0 +1,88 @@ +//! Explicit, versioned effective-edit lookup; generic #e matching stays exact. +use crate::{ + api::{api_error, db_read_error}, + state::AppState, +}; +use axum::{http::StatusCode, Json}; +use buzz_core::TenantContext; +use nostr::{EventId, PublicKey}; +use serde_json::{json, Value}; +use uuid::Uuid; + +pub(super) fn requested(filters: &[Value]) -> bool { + filters.iter().any(|f| f.get("effective_edit").is_some()) +} + +fn request(filters: &[Value]) -> Result<(Uuid, EventId), &'static str> { + let filter = filters.first().ok_or("missing effective_edit filter")?; + let channel = filter["#h"][0] + .as_str() + .and_then(|v| Uuid::parse_str(v).ok()) + .ok_or("invalid channel")?; + let target = filter["ids"][0] + .as_str() + .and_then(|v| EventId::from_hex(v).ok()) + .ok_or("invalid target")?; + if filters + != [ + json!({"kinds":buzz_core::media::EDIT_GOVERNED_KINDS, "ids":[target.to_hex()], "#h":[channel.to_string()], "effective_edit":1, "consistency":"strong"}), + ] + { + return Err("effective_edit requires one exact version-1 original-message filter with strong consistency"); + } + Ok((channel, target)) +} + +pub(super) async fn query( + state: &AppState, + tenant: &TenantContext, + reader: &PublicKey, + filters: &[Value], +) -> Result, (StatusCode, Json)> { + let (channel, target) = + request(filters).map_err(|message| api_error(StatusCode::BAD_REQUEST, message))?; + let snapshot = state + .db + .message_edit_state( + tenant.community(), + channel, + target.as_bytes(), + reader.as_bytes(), + ) + .await + .map_err(|error| db_read_error("effective edit lookup", &error))? + .filter(|snapshot| { + crate::handlers::req::event_visible_to_reader(&snapshot.original, reader.as_bytes()) + && snapshot.edit.as_ref().is_none_or(|edit| { + crate::handlers::req::event_visible_to_reader(edit, reader.as_bytes()) + }) + }) + .ok_or_else(|| api_error(StatusCode::NOT_FOUND, "message not found"))?; + Ok(Json( + json!({"effective_edit":1, "original":snapshot.original, "edit":snapshot.edit}), + )) +} + +#[cfg(test)] +mod tests { + use super::*; + #[test] + fn exact_versioned_shape_is_required() { + let channel = Uuid::new_v4(); + let valid = json!({"kinds":[9,40002,40008,45001,45003],"ids":["a".repeat(64)],"#h":[channel.to_string()],"effective_edit":1,"consistency":"strong"}); + assert!(request(std::slice::from_ref(&valid)).is_ok()); + for (key, value) in [ + ("effective_edit", json!(2)), + ("consistency", json!("eventual")), + ("#h", json!([])), + ("ids", json!([])), + ("kinds", json!([40003])), + ("limit", json!(1)), + ] { + let mut invalid = valid.clone(); + invalid[key] = value; + assert!(request(&[invalid]).is_err(), "{key}"); + } + assert!(request(&[valid.clone(), valid]).is_err()); + } +} diff --git a/crates/buzz-relay/src/api/media_access_postgres_tests.rs b/crates/buzz-relay/src/api/media_access_postgres_tests.rs index 0fe6b7b985..b2a8f7d9c5 100644 --- a/crates/buzz-relay/src/api/media_access_postgres_tests.rs +++ b/crates/buzz-relay/src/api/media_access_postgres_tests.rs @@ -173,6 +173,7 @@ async fn post( .await } +mod cli_edit_postgres_tests; mod emoji_reference_postgres_tests; mod reference_budget_postgres_tests; mod reference_formats_postgres_tests; diff --git a/crates/buzz-relay/src/api/media_access_postgres_tests/cli_edit_postgres_tests.rs b/crates/buzz-relay/src/api/media_access_postgres_tests/cli_edit_postgres_tests.rs new file mode 100644 index 0000000000..e961ada4c8 --- /dev/null +++ b/crates/buzz-relay/src/api/media_access_postgres_tests/cli_edit_postgres_tests.rs @@ -0,0 +1,734 @@ +//! Exercise the real CLI, HTTP bridge, ingest, and media reads in enforce mode. +use super::*; + +// Keep the large ingest future off the enclosing test thread's stack. +async fn publish( + state: &Arc, + tenant: &buzz_core::TenantContext, + keys: &Keys, + event: nostr::Event, +) -> Result { + Box::pin(post(state, tenant, keys, event)).await +} + +async fn lookup( + state: &Arc, + host: &str, + keys: &Keys, + channel: Uuid, + target: nostr::EventId, +) -> (StatusCode, serde_json::Value) { + let body = serde_json::to_vec(&serde_json::json!([{"kinds":[9,40002,40008,45001,45003],"ids":[target.to_hex()],"#h":[channel.to_string()],"effective_edit":1,"consistency":"strong"}])).unwrap(); + let proof = EventBuilder::new(Kind::Custom(27235), "") + .tags([ + Tag::parse(["u", &format!("http://{host}/query")]).unwrap(), + Tag::parse(["method", "POST"]).unwrap(), + Tag::parse(["payload", &hex::encode(Sha256::digest(&body))]).unwrap(), + Tag::parse(["nonce", &Uuid::new_v4().to_string()]).unwrap(), + ]) + .sign_with_keys(keys) + .unwrap(); + let request = Request::builder() + .method("POST") + .uri("/query") + .header(header::HOST, host) + .header(header::CONTENT_TYPE, "application/json") + .header( + header::AUTHORIZATION, + format!( + "Nostr {}", + base64::engine::general_purpose::STANDARD.encode(proof.as_json()) + ), + ) + .body(Body::from(body)) + .unwrap(); + let response = crate::router::build_router(state.clone()) + .oneshot(request) + .await + .unwrap(); + let status = response.status(); + ( + status, + serde_json::from_slice(&to_bytes(response.into_body(), 1_000_000).await.unwrap()).unwrap(), + ) +} + +async fn cli_edit(url: &str, keys: &Keys, id: nostr::EventId, clear: bool, content: &str) { + let mut args = vec![ + "buzz".to_owned(), + "--relay".into(), + url.into(), + "--private-key".into(), + keys.secret_key().to_secret_hex(), + "messages".into(), + "edit".into(), + "--event".into(), + id.to_hex(), + "--content".into(), + content.into(), + ]; + if clear { + args.push("--clear-attachments".into()); + } + assert_eq!( + Box::pin(buzz_cli::run_from_args(args)).await, + 0, + "CLI edit failed" + ); +} +async fn readable( + state: &Arc, + host: &str, + hashes: &[String], + keys: &[&Keys], + expected: &[StatusCode], +) { + for (hash, status) in hashes.iter().zip(expected) { + for key in keys { + for method in ["GET", "HEAD"] { + let response = router(state.clone()) + .oneshot(get(key, host, hash, hash, method, None)) + .await + .unwrap(); + assert_eq!(response.status(), *status, "{method} {hash}"); + assert_eq!( + response.headers()[header::CACHE_CONTROL], + "private, no-store" + ); + } + } + } +} + +#[tokio::test] +#[ignore = "requires Postgres"] +async fn cli_successive_edits_preserve_and_revoke_attachments() { + let bucket = Bucket::default(); + let s3_listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); + let endpoint = format!("http://{}", s3_listener.local_addr().unwrap()); + let service = Router::new() + .route("/{*key}", any(object)) + .with_state(bucket); + let s3_server = tokio::spawn(async move { + axum::serve(s3_listener, service).await.unwrap(); + }); + let state = fixture( + &endpoint, + super::super::media_access::MediaReadMode::Enforce, + ) + .await; + let http_listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); + let host_string = http_listener.local_addr().unwrap().to_string(); + let host = host_string.as_str(); + state.db.ensure_configured_community(host).await.unwrap(); + let tenant = crate::tenant::bind_community(&state.db, host) + .await + .unwrap(); + let owner = Keys::generate(); + let member = Keys::generate(); + let outsider = Keys::generate(); + for key in [&owner, &member, &outsider] { + state + .db + .ensure_user(tenant.community(), key.public_key().as_bytes()) + .await + .unwrap(); + } + let channel = state + .db + .create_channel( + tenant.community(), + "cli-edit", + ChannelType::Stream, + ChannelVisibility::Private, + None, + owner.public_key().as_bytes(), + None, + ) + .await + .unwrap() + .id; + state + .db + .add_member( + tenant.community(), + channel, + member.public_key().as_bytes(), + MemberRole::Member, + Some(owner.public_key().as_bytes()), + ) + .await + .unwrap(); + let mut media = Vec::new(); + let mut hashes = Vec::new(); + for bytes in [ + b"cli retained file".as_slice(), + b"cli removed file".as_slice(), + ] { + let hash = hex::encode(Sha256::digest(bytes)); + let upload = Request::builder() + .method("PUT") + .uri("/upload") + .header(header::HOST, host) + .header(header::AUTHORIZATION, proof(&owner, host, &hash, "upload")) + .header("X-SHA-256", &hash) + .body(Body::from(bytes)) + .unwrap(); + let response = router(state.clone()).oneshot(upload).await.unwrap(); + assert_eq!(response.status(), StatusCode::OK); + let descriptor: serde_json::Value = + serde_json::from_slice(&to_bytes(response.into_body(), 100_000).await.unwrap()) + .unwrap(); + media.push(vec![ + "imeta".into(), + format!("url {}", descriptor["url"].as_str().unwrap()), + format!("x {hash}"), + "m application/octet-stream".into(), + format!("size {}", bytes.len()), + "alt original label".into(), + ]); + hashes.push(hash); + } + let original = EventBuilder::new(Kind::Custom(9), "two files") + .tags( + std::iter::once(Tag::parse(["h", &channel.to_string()]).unwrap()) + .chain(media.iter().map(|t| Tag::parse(t.clone()).unwrap())), + ) + .custom_created_at(Timestamp::from(Timestamp::now().as_secs() - 20)) + .sign_with_keys(&owner) + .unwrap(); + publish(&state, &tenant, &owner, original.clone()) + .await + .unwrap(); + let url = format!("http://{host}"); + let http_app = crate::router::build_router(state.clone()); + let http_server = tokio::spawn(async move { + axum::serve(http_listener, http_app).await.unwrap(); + }); + for index in 0..2 { + cli_edit( + &url, + &owner, + original.id, + false, + &format!("typo fix {index}"), + ) + .await; + readable( + &state, + host, + &hashes, + &[&owner, &member], + &[StatusCode::OK; 2], + ) + .await; + readable( + &state, + host, + &hashes, + &[&outsider], + &[StatusCode::NOT_FOUND; 2], + ) + .await; + } + // Model a desktop/SDK edit removing just the second file. Keep its URL in + // the body to prove text cannot grant access or revive removed imeta. + let body = format!("retained one, removed URL {}", media[1][1]); + let partial = buzz_sdk::build_edit_with_media(channel, original.id, &body, &media[..1]) + .unwrap() + .custom_created_at(Timestamp::from(Timestamp::now().as_secs() + 5)) + .sign_with_keys(&owner) + .unwrap(); + publish(&state, &tenant, &owner, partial.clone()) + .await + .unwrap(); + for _ in 0..2 { + cli_edit(&url, &owner, original.id, false, &body).await; + readable( + &state, + host, + &hashes, + &[&owner, &member], + &[StatusCode::OK, StatusCode::NOT_FOUND], + ) + .await; + } + // A newer edit that restores both files is then deleted. Resolve the + // latest live edit, not this deleted replacement or the original. + let deleted = + buzz_sdk::build_edit_with_media(channel, original.id, "deleted replacement", &media) + .unwrap() + .custom_created_at(Timestamp::from(Timestamp::now().as_secs() + 20)) + .sign_with_keys(&owner) + .unwrap(); + // Seed a historical deleted replacement without publishing a new live + // restoration. The uploader still has authority to republish these bytes. + state + .db + .insert_event(tenant.community(), &deleted, Some(channel)) + .await + .unwrap(); + state + .db + .soft_delete_event_and_update_thread(tenant.community(), deleted.id.as_bytes(), None, None) + .await + .unwrap(); + cli_edit(&url, &owner, original.id, false, &body).await; + readable( + &state, + host, + &hashes, + &[&owner, &member], + &[StatusCode::OK, StatusCode::NOT_FOUND], + ) + .await; + cli_edit(&url, &owner, original.id, true, &body).await; + cli_edit(&url, &owner, original.id, false, &body).await; + readable( + &state, + host, + &hashes, + &[&owner, &member], + &[StatusCode::NOT_FOUND; 2], + ) + .await; + + // Mixed-case removal must stay removed from this message even when a + // second live reference would authorize republication of the same bytes. + let second = EventBuilder::new(Kind::Custom(9), "another reference") + .tags( + std::iter::once(Tag::parse(["h", &channel.to_string()]).unwrap()) + .chain(media[..1].iter().map(|t| Tag::parse(t.clone()).unwrap())), + ) + .sign_with_keys(&owner) + .unwrap(); + publish(&state, &tenant, &owner, second.clone()) + .await + .unwrap(); + let restored = buzz_sdk::build_edit_with_media(channel, original.id, "both", &media) + .unwrap() + .custom_created_at(Timestamp::from(Timestamp::now().as_secs() + 30)) + .sign_with_keys(&owner) + .unwrap(); + publish(&state, &tenant, &owner, restored).await.unwrap(); + let mixed = original + .id + .to_hex() + .chars() + .enumerate() + .map(|(i, c)| { + if i % 2 == 0 { + c.to_ascii_uppercase() + } else { + c + } + }) + .collect::(); + let removal = EventBuilder::new(Kind::Custom(40003), "only second") + .tags([ + Tag::parse(["h", &channel.to_string()]).unwrap(), + Tag::parse(["e", &mixed]).unwrap(), + Tag::parse(media[1].clone()).unwrap(), + ]) + .custom_created_at(Timestamp::from(Timestamp::now().as_secs() + 31)) + .sign_with_keys(&owner) + .unwrap(); + publish(&state, &tenant, &owner, removal).await.unwrap(); + cli_edit(&url, &owner, original.id, false, "after mixed-case removal").await; + let current = state + .db + .message_edit_state( + tenant.community(), + channel, + original.id.as_bytes(), + owner.public_key().as_bytes(), + ) + .await + .unwrap() + .unwrap() + .edit + .unwrap(); + let carried: Vec<_> = current + .tags + .iter() + .filter(|t| t.as_slice()[0] == "imeta") + .map(|t| t.as_slice().to_vec()) + .collect(); + assert_eq!( + carried, + vec![media[1].clone()], + "must not resurrect the first file" + ); + readable( + &state, + host, + &hashes, + &[&owner, &member], + &[StatusCode::OK; 2], + ) + .await; + state + .db + .soft_delete_event_and_update_thread(tenant.community(), second.id.as_bytes(), None, None) + .await + .unwrap(); + readable( + &state, + host, + &hashes, + &[&owner, &member], + &[StatusCode::NOT_FOUND, StatusCode::OK], + ) + .await; + + // An agent owner outside this private channel has no attachment read or + // publication authority. A direct retaining edit must also be rejected. + state + .db + .set_agent_owner( + tenant.community(), + owner.public_key().as_bytes(), + outsider.public_key().as_bytes(), + ) + .await + .unwrap(); + let owner_edit = + buzz_sdk::build_edit_with_media(channel, original.id, "owner edit", &media[1..]) + .unwrap() + .sign_with_keys(&outsider) + .unwrap(); + let Err(error) = publish(&state, &tenant, &outsider, owner_edit).await else { + panic!("owner edit was accepted"); + }; + assert!( + matches!(error, crate::handlers::ingest::IngestError::Rejected(ref reason) + if reason == "restricted: attachment is not accessible"), + "{error:?}" + ); + assert_ne!( + Box::pin(buzz_cli::run_from_args(vec![ + "buzz".to_string(), + "--relay".into(), + url.clone(), + "--private-key".into(), + outsider.secret_key().to_secret_hex(), + "messages".into(), + "edit".into(), + "--event".into(), + original.id.to_hex(), + "--content".into(), + "owner edit".into() + ])) + .await, + 0 + ); + + // Knowing a private hash never lets another member attach it to their own + // message/edit. This exercises the common publication check on kind 40003. + let open = state + .db + .create_channel( + tenant.community(), + "attacker", + ChannelType::Stream, + ChannelVisibility::Open, + None, + outsider.public_key().as_bytes(), + None, + ) + .await + .unwrap() + .id; + let harmless = EventBuilder::new(Kind::Custom(9), "no files") + .tags([Tag::parse(["h", &open.to_string()]).unwrap()]) + .sign_with_keys(&outsider) + .unwrap(); + publish(&state, &tenant, &outsider, harmless.clone()) + .await + .unwrap(); + let copied = buzz_sdk::build_edit_with_media(open, harmless.id, "stolen", &media) + .unwrap() + .sign_with_keys(&outsider) + .unwrap(); + assert!(publish(&state, &tenant, &outsider, copied).await.is_err()); + readable( + &state, + host, + &hashes, + &[&outsider], + &[StatusCode::NOT_FOUND; 2], + ) + .await; + let additional_targets = Box::pin(governed_kinds_journey( + &state, &tenant, host, &url, channel, open, &owner, &member, &outsider, + )) + .await; + for kind in [40003, 7, 45002] { + let unsupported = EventBuilder::new(Kind::Custom(kind), "unsupported lookup target") + .tags([Tag::parse(["h", &open.to_string()]).unwrap()]) + .sign_with_keys(&outsider) + .unwrap(); + state + .db + .insert_event(tenant.community(), &unsupported, Some(open)) + .await + .unwrap(); + assert_eq!( + lookup(&state, host, &member, open, unsupported.id).await.0, + StatusCode::NOT_FOUND + ); + } + let forbidden = lookup(&state, host, &outsider, channel, original.id).await; + let missing = lookup( + &state, + host, + &outsider, + channel, + nostr::EventId::from_byte_array([0x42; 32]), + ) + .await; + assert_eq!( + forbidden, missing, + "inaccessible and missing states must be indistinguishable" + ); + assert_eq!(forbidden.0, StatusCode::NOT_FOUND); + state + .db + .remove_member( + tenant.community(), + channel, + member.public_key().as_bytes(), + owner.public_key().as_bytes(), + ) + .await + .unwrap(); + assert_eq!( + lookup(&state, host, &member, channel, original.id).await, + missing, + "fresh snapshot must reject a removed member" + ); + for target in additional_targets { + assert_eq!( + lookup(&state, host, &member, channel, target).await, + missing, + "removed member must not read added target kind" + ); + } + assert_eq!( + lookup(&state, host, &member, open, harmless.id).await.0, + StatusCode::OK, + "open-channel reader needs no membership" + ); + http_server.abort(); + s3_server.abort(); +} + +// Keep this larger journey's future separately allocated on the default stack. +#[allow(clippy::too_many_arguments)] +async fn governed_kinds_journey( + state: &Arc, + tenant: &buzz_core::TenantContext, + host: &str, + url: &str, + channel: Uuid, + open: Uuid, + owner: &Keys, + member: &Keys, + outsider: &Keys, +) -> Vec { + let mut targets = Vec::new(); + for kind in [9, 40002, 40008, 45001, 45003] { + // Each original has isolated A/B bytes. Only the signer uploaded them; + // the member reader has no upload grant or alternate live reference. + let mut tags = Vec::new(); + let mut hashes = Vec::new(); + for label in ["A", "B"] { + let bytes = format!("kind {kind} {label} {}", Uuid::new_v4()).into_bytes(); + let hash = hex::encode(Sha256::digest(&bytes)); + let response = router(state.clone()) + .oneshot( + Request::builder() + .method("PUT") + .uri("/upload") + .header(header::HOST, host) + .header(header::AUTHORIZATION, proof(owner, host, &hash, "upload")) + .header("X-SHA-256", &hash) + .body(Body::from(bytes.clone())) + .unwrap(), + ) + .await + .unwrap(); + assert_eq!(response.status(), StatusCode::OK); + let descriptor: serde_json::Value = + serde_json::from_slice(&to_bytes(response.into_body(), 100_000).await.unwrap()) + .unwrap(); + tags.push(vec![ + "imeta".into(), + format!("url {}", descriptor["url"].as_str().unwrap()), + format!("x {hash}"), + "m application/octet-stream".into(), + format!("size {}", bytes.len()), + ]); + hashes.push(hash); + } + // Creation schemas differ by surface; seed signed originals, then use + // the real HTTP lookup, CLI, ingest and media handlers for every edit. + let source = EventBuilder::new(Kind::Custom(kind), "original A") + .tags([ + Tag::parse(["h", &channel.to_string()]).unwrap(), + Tag::parse(tags[0].clone()).unwrap(), + ]) + .custom_created_at(Timestamp::from(Timestamp::now().as_secs() - 20)) + .sign_with_keys(owner) + .unwrap(); + state + .db + .insert_event(tenant.community(), &source, Some(channel)) + .await + .unwrap(); + targets.push(source.id); + let denied = lookup(state, host, outsider, channel, source.id).await; + let absent = lookup( + state, + host, + outsider, + channel, + nostr::EventId::from_byte_array([0x43; 32]), + ) + .await; + assert_eq!(denied, absent, "kind {kind} leaks private state"); + assert_eq!(denied.0, StatusCode::NOT_FOUND); + let retaining = + buzz_sdk::build_edit_with_media(channel, source.id, "unauthorized owner", &tags[..1]) + .unwrap() + .sign_with_keys(outsider) + .unwrap(); + let Err(error) = publish(state, tenant, outsider, retaining).await else { + panic!("owner gained media access for kind {kind}") + }; + assert!( + matches!(error,crate::handlers::ingest::IngestError::Rejected(ref reason) if reason=="restricted: attachment is not accessible"), + "{error:?}" + ); + readable( + state, + host, + &hashes, + &[member], + &[StatusCode::OK, StatusCode::NOT_FOUND], + ) + .await; + let replacement = + buzz_sdk::build_edit_with_media(channel, source.id, "replace A with B", &tags[1..]) + .unwrap() + .custom_created_at(Timestamp::from(Timestamp::now().as_secs() - 1)) + .sign_with_keys(owner) + .unwrap(); + publish(state, tenant, owner, replacement).await.unwrap(); + for (clear, content, expected) in [ + (false, "retain B", &tags[1..]), + (true, "clear", &tags[0..0]), + (false, "still empty", &tags[0..0]), + ] { + cli_edit(url, owner, source.id, clear, content).await; + let (status, snapshot) = lookup(state, host, member, channel, source.id).await; + assert_eq!(status, StatusCode::OK); + let edit: nostr::Event = serde_json::from_value(snapshot["edit"].clone()).unwrap(); + assert_eq!(edit.content, content); + let carried: Vec<_> = edit + .tags + .iter() + .filter(|t| t.as_slice()[0] == "imeta") + .map(|t| t.as_slice().to_vec()) + .collect(); + assert_eq!(carried, expected, "kind {kind} metadata"); + readable( + state, + host, + &hashes, + &[member], + &[ + StatusCode::NOT_FOUND, + if expected.is_empty() { + StatusCode::NOT_FOUND + } else { + StatusCode::OK + }, + ], + ) + .await; + } + if kind == 45001 { + // A is no longer referenced by the attachment journey. Give only + // this forum original a snapshot reference, with no reader grant. + let image_url = tags[0][1].strip_prefix("url ").unwrap(); + let preview = EventBuilder::new(Kind::Custom(kind), "forum preview") + .tags([ + Tag::parse(["h", &channel.to_string()]).unwrap(), + Tag::parse([ + "link-preview", + "snapshot", + "1", + "https://example.org/page", + "title", + "site", + "description", + image_url, + &hashes[0], + "", + "", + ]) + .unwrap(), + ]) + .custom_created_at(Timestamp::from(Timestamp::now().as_secs() - 20)) + .sign_with_keys(owner) + .unwrap(); + state + .db + .insert_event(tenant.community(), &preview, Some(channel)) + .await + .unwrap(); + readable(state, host, &hashes[..1], &[member], &[StatusCode::OK]).await; + cli_edit(url, owner, preview.id, false, "keep forum preview").await; + readable(state, host, &hashes[..1], &[member], &[StatusCode::OK]).await; + let suppressed = + buzz_sdk::build_edit_with_media(channel, preview.id, "hide preview", &[]) + .unwrap() + .tags([Tag::parse(["link-preview", "none"]).unwrap()]) + .sign_with_keys(owner) + .unwrap(); + publish(state, tenant, owner, suppressed).await.unwrap(); + readable( + state, + host, + &hashes[..1], + &[member], + &[StatusCode::NOT_FOUND], + ) + .await; + cli_edit(url, owner, preview.id, false, "preview stays hidden").await; + readable( + state, + host, + &hashes[..1], + &[member], + &[StatusCode::NOT_FOUND], + ) + .await; + } + let public = EventBuilder::new(Kind::Custom(kind), "public original") + .tags([Tag::parse(["h", &open.to_string()]).unwrap()]) + .sign_with_keys(outsider) + .unwrap(); + state + .db + .insert_event(tenant.community(), &public, Some(open)) + .await + .unwrap(); + assert_eq!( + lookup(state, host, member, open, public.id).await.0, + StatusCode::OK + ); + } + targets +} diff --git a/crates/buzz-sdk/src/builders.rs b/crates/buzz-sdk/src/builders.rs index 9970e2f000..6c5a306562 100644 --- a/crates/buzz-sdk/src/builders.rs +++ b/crates/buzz-sdk/src/builders.rs @@ -425,16 +425,43 @@ pub fn build_diff_message( } /// Build an edit event targeting an existing message (kind 40003). +/// +/// This compatibility helper removes all attachments. Use +/// [`build_edit_with_media`] with the latest effective imeta to retain them. pub fn build_edit( channel_id: Uuid, target_event_id: nostr::EventId, new_content: &str, +) -> Result { + build_edit_with_media(channel_id, target_event_id, new_content, &[]) +} + +/// Build an edit with the complete replacement attachment set (kind 40003). +/// +/// Pass the latest live edit's imeta tags, or the original's when no edit +/// exists, to retain attachments. An empty set explicitly removes attachments; +/// URLs in the content alone do not retain media access. The relay still +/// checks the signer's publication rights for every local attachment. +pub fn build_edit_with_media( + channel_id: Uuid, + target_event_id: nostr::EventId, + new_content: &str, + media_tags: &[Vec], ) -> Result { check_content(new_content, 64 * 1024)?; - let tags = vec![ + let mut tags = vec![ tag(&["h", &channel_id.to_string()])?, tag(&["e", &target_event_id.to_hex()])?, ]; + // Only attachment metadata belongs here: a caller must not inject a + // competing target/channel or authorization tag through media_tags. + if media_tags + .iter() + .any(|t| t.first().map(String::as_str) != Some("imeta")) + { + return Err(SdkError::InvalidTag("expected imeta attachment tag".into())); + } + imeta_tags(media_tags, &mut tags)?; Ok(EventBuilder::new(Kind::Custom(40003), new_content).tags(tags)) } @@ -3309,6 +3336,34 @@ mod tests { assert!(has_tag(&ev, "e", &eid.to_hex())); } + #[test] + fn edit_replaces_complete_media_set_and_rejects_non_media_tags() { + let cid = uuid(); + let eid = event_id(); + let media = vec![vec![ + "imeta".into(), + "url https://example.com/a.png".into(), + "x ".to_owned() + &"a".repeat(64), + "alt full label".into(), + "image https://example.com/poster.png".into(), + ]]; + let event = sign(build_edit_with_media(cid, eid, "new", &media).unwrap()); + assert_eq!( + event + .tags + .iter() + .find(|t| t.as_slice()[0] == "imeta") + .unwrap() + .as_slice(), + media[0] + ); + let cleared = sign(build_edit_with_media(cid, eid, "clear", &[]).unwrap()); + assert!(!cleared.tags.iter().any(|t| t.as_slice()[0] == "imeta")); + assert!(build_edit_with_media(cid, eid, "bad", &[vec!["e".into(), eid.to_hex()]]).is_err()); + assert!(build_edit_with_media(cid, eid, "bad", &[vec![]]).is_err()); + assert!(build_edit_with_media(cid, eid, &"x".repeat(64 * 1024 + 1), &media).is_err()); + } + #[test] fn edit_content_too_large() { let cid = uuid(); diff --git a/docs/effective-message-edits.md b/docs/effective-message-edits.md new file mode 100644 index 0000000000..0f18140114 --- /dev/null +++ b/docs/effective-message-edits.md @@ -0,0 +1,61 @@ +# Effective message edits + +For kinds 9/40002/40008/45001/45003, text edits replace a message's complete attachment set. To retain files, a +writer needs the latest live edit's imeta, or the original's when no edit exists. +A plain `#e` query is an exact tag match and can miss valid historical targets +whose hex casing differs. This extension leaves generic NIP-01 matching intact. + +Other channel-tagged kinds keep the historical plain-edit behavior, without the +lookup or a relay-version dependency. The CLI rejects `--clear-attachments` for +those kinds because their attachments are not edit-governed. + +## Request and response + +An authenticated `POST /query` accepts one exact filter: + +```json +[{"kinds":[9,40002,40008,45001,45003],"ids":[""],"#h":[""],"effective_edit":1,"consistency":"strong"}] +``` + +The version-1 envelope is: + +```json +{"effective_edit":1,"original":{"id":"...","kind":9,"tags":[],"content":"...","pubkey":"...","created_at":0,"sig":"..."},"edit":null} +``` + +`original` and a non-null `edit` are complete signed Nostr events. `edit` is the +latest live kind 40003 in the same channel targeting the live original of kind +9, 40002, 40008, 45001 or 45003. Targets use the first valid 64-character hex `e` tag in signed order, +ignoring casing. The greatest `(created_at,id)` wins. Upload proof can still authorize +republication by the uploader after removal; correct state resolution prevents +accidental restoration even when publication would be authorized. An empty imeta set on that +edit is authoritative; it never falls back to original attachments. + +Original liveness, fresh channel authorization, and edit selection share a +repeatable-read transaction on the primary. Ordinary per-event visibility gates +also apply. Missing and inaccessible targets both return the same 404. Lookup +failure returns an error, never an empty state. The normalized-target index from +migration 0086 bounds the latest-edit probe; unrelated channel edits are not read. + +The CLI requires the versioned envelope, checks the returned target/channel, and +aborts without signing or publishing when an older relay returns an ordinary +array. Deploy the relay first, then release the CLI. Old clients retain their +existing behavior against the new relay. No schema change is needed beyond 0086. + +The CLI stamps its replacement after the observed state, including same-second +edits. A head more than 60 seconds ahead of its clock causes an error until the +clock catches up. Concurrent writers still use `(created_at,id)` ordering; the +lookup does not create a compare-and-swap precondition on publication. + +The same five kinds govern the lookup, CLI attachment preservation and media +read authorization. A superseded original no longer grants access; the latest +live edit's full imeta set controls that reference. A file remains readable if +another live readable reference or upload authority grants access. An empty +replacement set does not fall back to the original. + +The v1 request uses the exact five-kind filter above. + +Deploy the relay before releasing the CLI or paired MCP producer. Edits of any +of these five kinds fail closed on community relays that lack the lookup. The +CLI and MCP both preserve the latest metadata, including an empty set; the CLI's +`--clear-attachments` covers all five kinds. diff --git a/docs/operator-private-media.md b/docs/operator-private-media.md index dc37d37d52..8634f9e536 100644 --- a/docs/operator-private-media.md +++ b/docs/operator-private-media.md @@ -90,17 +90,51 @@ media access through the ordinary media endpoint. 4. Repair unsupported/historical URLs or republish them through an authorized uploader before restarting every serving pod in `enforce`. -**Enforce blocker: SDK/CLI attachment edits remain unsupported.** -`buzz_sdk::build_edit` and `buzz messages edit` currently omit attachment imeta. -Editing an attachment message through these clients can make its file unreadable, -including to its sender. This review item is deferred, not fixed. Enforce stays blocked unless the operator can rule out SDK/CLI attachment edits -in the community. The agent harness edit producer is unverified and remains -inside this blocker until its metadata behavior is traced and tested. To remove -the blocker, update both SDK/CLI paths to preserve the -latest effective attachment metadata across successive edits without restoring -files removed by an earlier edit. Require an upload → successive edits → GET/HEAD -regression proving retained files stay readable and removed files stay denied. -Keeping a URL in the edited body does not preserve access. +**Enforce prerequisite: upgrade every edit producer.** The updated +`buzz messages edit` resolves the latest live attachment state on the primary and +carries its complete imeta set for kinds 9/40002/40008/45001/45003. For those kinds, +`--clear-attachments` explicitly removes all attachments; later text edits +preserve that empty set. Other channel-tagged kinds keep their original media references after edits, +and the CLI rejects `--clear-attachments` for them. SDK +integrations must use `buzz_sdk::build_edit_with_media` with the latest +effective metadata. The compatibility `build_edit` helper still emits an empty +attachment set and therefore removes attachments on kinds 9/40002/40008/45001/45003. A URL in the +body does not retain access. + +Deploy the relay with the [versioned effective-edit +lookup](effective-message-edits.md) before releasing the new CLI. When editing +kinds 9/40002/40008/45001/45003, a new CLI against an older relay aborts without publishing; an +old CLI against a new relay emits empty imeta and revokes attachments on all +five kinds, including diff/forum attachments that previously remained readable. +Other kinds use plain edits without a lookup or a relay-version dependency. The +`buzz-dev-mcp` multicall named `buzz` delegates to the same CLI entrypoint. The +PostgreSQL regression `cli_successive_edits_preserve_and_revoke_attachments` +exercises that entrypoint through HTTP and checks GET/HEAD, removal, and +mixed-case edit targets. + +**Remaining enforce blockers include the separate `block/buzz-app` MCP edit producer +and the active producers listed below.** +Its direct publisher at `src/buzz-mcp/tools.ts` copies the original event's +imeta rather than resolving successive edits. It must adopt effective state +and pass the same removal regression before enabling enforce for its users. +Audit other third-party SDK/direct publishers too; upgrading the CLI alone +cannot change their signed events. + +The relay rejects a retaining edit when its signer cannot publish any carried +attachment. Agent ownership does not grant private-channel media access: an +owner outside that channel cannot retain its files. Restore access or explicitly +remove the attachments with `--clear-attachments`; the CLI itself also requires +read access to the original. Concurrent edits remain ordered by `(created_at,id)`; +coordinate writers when attachment changes overlap. There is no compare-and-swap +precondition on message edits. + +This client change has no feature flag. Observe +`buzz_media_publication_access_total{mode="shadow",reason="denied"}` before +enforcement. Roll back the CLI by reverting its release; retain the additive +relay lookup. Rolling the relay back first causes new CLI/MCP edits on all five kinds to +abort without publishing. Already-published imeta edits remain valid; rollback +restores the older relay's media-reference rules, so recheck access before +resuming enforcement. There is no grant backfill: existing stored references are evaluated directly, and signed event tags are never rewritten. Legacy unreferenced/orphan blobs @@ -189,3 +223,58 @@ instance has no usable signing key. The error names affected instances; restore their keys, or remove/unlink those instances, then retry. The previous persona and instance avatars remain unchanged, so the same requested avatar is retryable. Keys may be permanently lost; waiting alone does not repair a lost key. + +The relay folds effective attachment state for the same five kinds accepted by +the lookup: 9/40002/40008/45001/45003. CLI and MCP text edits preserve that state; +`--clear-attachments` replaces it with an empty set on all five. After removal, +access through that reference is revoked; another live reference or upload +proof may independently authorize the blob. An older community relay without +the lookup refuses these edits through the new CLI/MCP, so upgrade relays first. + +## Historical diff and forum edits + +**Readiness blocker:** before enforcement, audit existing kind 40008/45001/45003 +originals and every active edit producer. Expanding read folding applies to +stored events, including old CLI edits that emitted no imeta. Upgrading clients +does not repair those signed historical edits. An authorized writer must review +and republish the intended full set; never infer that an empty edit meant to +retain the original's files. No automatic historical rewrite is included. + +Run the read-only [preflight query](sql/private-media-edit-preflight.sql), binding +`$1` to a community UUID. For psql, prepare its SELECT as +`PREPARE private_media_edit_preflight(uuid) AS ...;`, then execute +`EXECUTE private_media_edit_preflight('');` inside a read-only +repeatable-read transaction. Use a bounded statement timeout. Each row counts +live added-kind originals with imeta whose selected live edit omits imeta. The +selection uses the same normalized first-valid target, same channel/community, +and descending `(created_at,id)` ordering as the read path. Missing kinds have +zero matches. These are affected references, not necessarily inaccessible blobs: +another live reference or upload proof can independently authorize the bytes. +Partial attachment changes require separate review. No production count is +claimed by this PR. + +| Producer | Current behavior and rollout requirement | +| --- | --- | +| Old CLI (through `96af488d`) | Plain edits emit no imeta; on the new relay they revoke diff/forum attachments that previously remained readable. | +| Updated CLI | Resolves the full effective set for all five kinds; explicit clear stays empty. Release after the relay. | +| SDK | `build_edit` emits no imeta. `build_edit_with_media` copies the supplied set; callers must resolve latest state. Third-party callers remain a prerequisite. | +| Legacy desktop chat | Carries rendered attachment state through `submitMessageEdit.ts` and `events.rs`; its local edit selection does not provide the primary lookup's full ordering guarantees. Audit deployed versions. | +| Legacy native generic edit | Copies supplied media, defaulting to empty; no effective lookup. Its forum readers keep showing original metadata and only fold preview suppression. Audit any direct generic callers before enforcement. | +| Mobile editor | `message_actions.dart` passes only custom emoji tags to `editMessage`; no imeta is retained. This remains an uncorrected producer blocker. | +| App MCP | Current main copies original metadata; the paired MCP fix adopts latest state. The fix must ship before enforcement for its users. | +| App UI | `fold.ts` selects the latest edit having imeta, falling back to the original. The UI editor can restore older files after an empty removal; it remains an uncorrected producer blocker. | +| ACP | Edit routing consumes events; generic publication forwards already-built signed events. It does not resolve attachments for producers. | + +Source paths are relative to this repository, or to `block/buzz-app` for app +entries. App audit snapshot: `cae3922574`; paired MCP implementation is tracked +in [the counterpart PR](https://github.com/block/buzz-app/pull/2937). This audit +identifies source behavior, not installed versions. + +Mode effects are precise: `off` skips reference checks for media reads and +publication. `shadow` evaluates the five-kind policy and records its metrics, +but does not deny reads/publication for policy failures; it also records sticky +publication boundaries best effort. `enforce` denies reads without a current +readable reference and rejects inaccessible carried attachments. Thus historical +empty edits change shadow observations immediately, and deny their superseded +original reference in enforce. Storage and tenant admission still apply in all +modes. The lookup and new CLI behavior apply independently of read mode. diff --git a/docs/sql/private-media-edit-preflight.sql b/docs/sql/private-media-edit-preflight.sql new file mode 100644 index 0000000000..f2d81dbefe --- /dev/null +++ b/docs/sql/private-media-edit-preflight.sql @@ -0,0 +1,22 @@ +-- Bind $1 to the community UUID. Counts newly edit-governed originals whose +-- selected live edit omits imeta; independent grants can still authorize blobs. +SELECT original.kind, count(*) AS affected_originals +FROM events original +JOIN channels c ON c.community_id = original.community_id + AND c.id = original.channel_id AND c.deleted_at IS NULL +JOIN LATERAL ( + SELECT e.tags + FROM events e + WHERE e.community_id = original.community_id + AND e.channel_id = original.channel_id + AND e.kind = 40003 AND e.deleted_at IS NULL + AND decode(lower(jsonb_path_query_first(tags, 'strict $[*] ? (@[0] == "e" && @[1] like_regex "^[0-9A-Fa-f]{64}$")[1]', '{}'::jsonb, true) #>> '{}'), 'hex') = original.id + ORDER BY e.created_at DESC, e.id DESC + LIMIT 1 +) latest ON true +WHERE original.community_id = $1 + AND original.kind IN (40008,45001,45003) AND original.deleted_at IS NULL + AND EXISTS (SELECT 1 FROM jsonb_array_elements(original.tags) t WHERE t->>0 = 'imeta') + AND NOT EXISTS (SELECT 1 FROM jsonb_array_elements(latest.tags) t WHERE t->>0 = 'imeta') +GROUP BY original.kind +ORDER BY original.kind;