mirror of
https://github.com/block/buzz.git
synced 2026-11-07 01:41:06 +00:00
## Summary Fixes two CI flakes from the 21–28 Oct 2026 audit, both in test code with no product change: - The relay E2E helper `collect_until_eose` now drops repeat copies of an event id. - Two Windows shell tests drop a 5 s cap that a cold Git Bash start can exceed. This is one of seven PRs split out of the `flakey-tests` branch (siblings below). ### Related issue None found for these two. The buzz-terminal `passwd_shell()` `getpwuid` race (`env_fence_tests::resolve_shell_falls_through_to_passwd_not_the_default`, run 38590095289) is left to open PR #7687, which already replaces `getpwuid`. #8636 and #7976 edit `buzz-test-client/src/lib.rs`, and #7607 and #7915 edit `buzz-dev-mcp/src/shell.rs`, but none of them changes `collect_until_eose` or these timeouts. ### Flaky tests fixed | Test | Failing runs | Cause | Change | |---|---|---|---| | Relay E2E `e2e_project::test_project_tombstone_deletes_coordinate_and_spares_members` (e2e_project.rs:338, "left: 2 right: 1") | 40408678412#1, job 121879630394 (PR #8620, schema-only) | The EVENT handler sends OK, then fans out in a spawned task. `handle_req` registers the live subscription before its history query, and `req.rs`'s `seen_ids` dedups history rows only. If fan-out runs after the REQ is registered, the event arrives once live and once from history, both before EOSE, and `collect_until_eose` kept every frame. | `collect_until_eose` keeps the first copy of each event id, as production clients do. Its doc comment says a test of the relay's own dedup must read frames with `recv_event`. A new unit test, `collect_until_eose_keeps_one_copy_of_each_event_id`, uses a fake relay that sends the duplicate. | | Windows `buzz-dev-mcp` `shell::tests::basic_echo` (shell.rs:1026, exit 124) | 39675698081#1, job 119485363399 (green on re-run) | 124 is the shell tool's own timeout exit code. `basic_echo` capped the shell at 5 s, and it is one of the first tests in the process to start Git Bash, which can take longer than 5 s on a cold runner. | `timeout_ms: None` (the production 120 s default) in `basic_echo` and in `workdir_is_honored`, which has the same exposure. `timeout_fires` still covers timeouts. | Pre-check: no e2e test relies on the relay's own history dedup. Every exact-count assertion uses a single filter. The NIP-33 "only newest remains" assertions would see a failed replacement as two different ids, and this dedup keeps both. Local evidence is from the original `flakey-tests` commit (fc4518d39): - The new unit test: FAIL before, PASS after. - 13 of the 14 e2e files that call the helper, plus `e2e_thread_roots`, against an isolated local relay. All pass, except `e2e_relay` `test_unarchive_emits_member_added_notification`. That test fails by itself 3/5 with the original helper too, and CI's Relay E2E lane doesn't select it. - The Windows failure can't be reproduced on macOS. The Windows Rust lane is the check. Follow-up (out of scope): make the relay deliver each event id at most once per subscription, keeping the live/history overlap. ### Verification This branch was pushed without local re-verification, by the user's choice: local test runs were overloading the machine and failing for that reason. **Hosted CI is the check.** It was pushed with `git push --no-verify`, because the pre-push hook runs clippy and unit tests. - Both files are byte-identical to their final versions on `flakey-tests` (67185df6a). - Checked for this PR: `git diff --check`, `rustfmt --check`, and a conflict-free merge with all six siblings onto `main`. ### Readiness Test-only change. AGENTS.md's readiness steps (agent review, then human confirmation) are still outstanding, so `buzz-review-completed` is not set. ### Merge order Independent of the other six. It merges cleanly onto `main`, and with every sibling, in any order. ### Sibling PRs Split from `flakey-tests` (one PR per area): #8676 (PostgreSQL lease + lane), #8678 (mobile), #8679 (Tauri apt cache + Windows Rust cap), #8680 (desktop product races), #8681 (desktop E2E state gates), #8682 (desktop E2E composer + row actions). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Signed-off-by: Matt Toohey <contact@matttoohey.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
420 lines
15 KiB
Rust
420 lines
15 KiB
Rust
#![deny(unsafe_code)]
|
|
#![warn(missing_docs)]
|
|
|
|
//! Minimal NIP-01 WebSocket test client for the Buzz relay.
|
|
|
|
use std::collections::HashSet;
|
|
use std::time::Duration;
|
|
|
|
use nostr::{Event, EventBuilder, Filter, Keys, Kind, Tag};
|
|
use serde_json::{json, Value};
|
|
use thiserror::Error;
|
|
use tracing::debug;
|
|
|
|
use buzz_ws_client::NostrWsConnection;
|
|
pub use buzz_ws_client::{parse_relay_message, OkResponse, RelayMessage, WsClientError};
|
|
|
|
/// Scenario-driven load generator behind the `buzz-loadgen` binary.
|
|
/// The scenario schema is documented in `perf/LOADGEN.md`.
|
|
#[allow(missing_docs)]
|
|
pub mod loadgen;
|
|
|
|
/// Errors returned by [`BuzzTestClient`] operations.
|
|
#[derive(Debug, Error)]
|
|
pub enum TestClientError {
|
|
/// A WebSocket transport error occurred.
|
|
#[error("WebSocket error: {0}")]
|
|
WebSocket(tokio_tungstenite::tungstenite::Error),
|
|
|
|
/// A JSON serialization or deserialization error occurred.
|
|
#[error("JSON error: {0}")]
|
|
Json(serde_json::Error),
|
|
|
|
/// Failed to build a Nostr event.
|
|
#[error("Nostr event builder error: {0}")]
|
|
EventBuilder(String),
|
|
|
|
/// Failed to parse a URL.
|
|
#[error("URL parse error: {0}")]
|
|
Url(String),
|
|
|
|
/// The relay did not respond within the expected time.
|
|
#[error("Timeout waiting for relay message")]
|
|
Timeout,
|
|
|
|
/// The WebSocket connection was closed before the operation completed.
|
|
#[error("Connection closed unexpectedly")]
|
|
ConnectionClosed,
|
|
|
|
/// The relay sent a message that was not expected at this point.
|
|
#[error("Unexpected relay message: {0}")]
|
|
UnexpectedMessage(String),
|
|
|
|
/// NIP-42 authentication was rejected by the relay.
|
|
#[error("Authentication failed: {0}")]
|
|
AuthFailed(String),
|
|
|
|
/// The relay rejected the submitted event.
|
|
#[error("Event rejected by relay: {0}")]
|
|
EventRejected(String),
|
|
|
|
/// No NIP-42 AUTH challenge was received from the relay.
|
|
#[error("No AUTH challenge received from relay")]
|
|
NoAuthChallenge,
|
|
}
|
|
|
|
impl From<WsClientError> for TestClientError {
|
|
fn from(e: WsClientError) -> Self {
|
|
match e {
|
|
WsClientError::WebSocket(e) => TestClientError::WebSocket(e),
|
|
WsClientError::Json(e) => TestClientError::Json(e),
|
|
WsClientError::EventBuilder(s) => TestClientError::EventBuilder(s),
|
|
WsClientError::Url(s) => TestClientError::Url(s),
|
|
WsClientError::Timeout => TestClientError::Timeout,
|
|
WsClientError::ConnectionClosed => TestClientError::ConnectionClosed,
|
|
WsClientError::UnexpectedMessage(s) => TestClientError::UnexpectedMessage(s),
|
|
WsClientError::AuthFailed(s) => TestClientError::AuthFailed(s),
|
|
WsClientError::EventRejected(s) => TestClientError::EventRejected(s),
|
|
WsClientError::NoAuthChallenge => TestClientError::NoAuthChallenge,
|
|
}
|
|
}
|
|
}
|
|
|
|
impl From<nostr::event::builder::Error> for TestClientError {
|
|
fn from(e: nostr::event::builder::Error) -> Self {
|
|
TestClientError::EventBuilder(e.to_string())
|
|
}
|
|
}
|
|
|
|
/// WebSocket test client for integration testing against a running Buzz relay.
|
|
pub struct BuzzTestClient {
|
|
inner: NostrWsConnection,
|
|
}
|
|
|
|
impl BuzzTestClient {
|
|
/// Connects to the relay at `url` and performs NIP-42 authentication with `keys`.
|
|
pub async fn connect(url: &str, keys: &Keys) -> Result<Self, TestClientError> {
|
|
let mut client = Self::connect_unauthenticated(url).await?;
|
|
client.authenticate(keys).await?;
|
|
Ok(client)
|
|
}
|
|
|
|
/// Connects to the relay at `url` without performing authentication.
|
|
pub async fn connect_unauthenticated(url: &str) -> Result<Self, TestClientError> {
|
|
let inner = NostrWsConnection::connect(url).await?;
|
|
debug!("connected to relay at {url}");
|
|
Ok(Self { inner })
|
|
}
|
|
|
|
/// Performs NIP-42 authentication using `keys` against the connected relay.
|
|
pub async fn authenticate(&mut self, keys: &Keys) -> Result<(), TestClientError> {
|
|
self.inner.authenticate(keys, None).await?;
|
|
Ok(())
|
|
}
|
|
|
|
/// Performs NIP-42 authentication with a NIP-OA `auth` tag, establishing
|
|
/// the agent→owner relationship in the relay's DB. Call this as the
|
|
/// *agent* connection; the tag must be signed by `owner_keys`.
|
|
pub async fn authenticate_with_nip_oa(
|
|
&mut self,
|
|
agent_keys: &Keys,
|
|
auth_tag: &Tag,
|
|
) -> Result<(), TestClientError> {
|
|
self.inner.authenticate(agent_keys, Some(auth_tag)).await?;
|
|
Ok(())
|
|
}
|
|
|
|
/// Sends a signed event to the relay and waits for the OK response.
|
|
pub async fn send_event(&mut self, event: Event) -> Result<OkResponse, TestClientError> {
|
|
Ok(self.inner.send_event(event).await?)
|
|
}
|
|
|
|
/// Builds and sends a text message event to `channel_id` using the given `kind`.
|
|
pub async fn send_text_message(
|
|
&mut self,
|
|
keys: &Keys,
|
|
channel_id: &str,
|
|
content: &str,
|
|
kind: u16,
|
|
) -> Result<OkResponse, TestClientError> {
|
|
let h_tag = Tag::parse(["h", channel_id])
|
|
.map_err(|e| TestClientError::EventBuilder(e.to_string()))?;
|
|
let event = EventBuilder::new(Kind::Custom(kind), content)
|
|
.tags([h_tag])
|
|
.sign_with_keys(keys)?;
|
|
self.send_event(event).await
|
|
}
|
|
|
|
/// Sends a REQ message to open a subscription with the given `sub_id` and `filters`.
|
|
pub async fn subscribe(
|
|
&mut self,
|
|
sub_id: &str,
|
|
filters: Vec<Filter>,
|
|
) -> Result<(), TestClientError> {
|
|
let mut msg: Vec<Value> = Vec::with_capacity(2 + filters.len());
|
|
msg.push(json!("REQ"));
|
|
msg.push(json!(sub_id));
|
|
for f in filters {
|
|
msg.push(serde_json::to_value(&f).map_err(WsClientError::Json)?);
|
|
}
|
|
self.inner.send_raw(&Value::Array(msg)).await?;
|
|
Ok(())
|
|
}
|
|
|
|
/// Sends a CLOSE message to cancel the subscription identified by `sub_id`.
|
|
pub async fn close_subscription(&mut self, sub_id: &str) -> Result<(), TestClientError> {
|
|
self.inner.send_raw(&json!(["CLOSE", sub_id])).await?;
|
|
Ok(())
|
|
}
|
|
|
|
/// Sends a raw JSON value as a WebSocket text frame.
|
|
pub async fn send_raw(&mut self, value: &serde_json::Value) -> Result<(), TestClientError> {
|
|
self.inner.send_raw(value).await?;
|
|
Ok(())
|
|
}
|
|
|
|
/// Receives the next relay message, waiting up to `timeout_dur`.
|
|
pub async fn recv_event(
|
|
&mut self,
|
|
timeout_dur: Duration,
|
|
) -> Result<RelayMessage, TestClientError> {
|
|
Ok(self.inner.next_event(timeout_dur).await?)
|
|
}
|
|
|
|
/// Collects the events for `sub_id` until EOSE is received, waiting up to
|
|
/// `timeout_dur`, keeping the first copy of each event id.
|
|
///
|
|
/// The relay registers a REQ's live subscription before it queries
|
|
/// history, so an event whose live fan-out runs late can arrive twice
|
|
/// before EOSE. Production clients drop the repeat, and so does this
|
|
/// helper. A test of the relay's own per-subscription dedup must read
|
|
/// frames with [`Self::recv_event`] instead.
|
|
pub async fn collect_until_eose(
|
|
&mut self,
|
|
sub_id: &str,
|
|
timeout_dur: Duration,
|
|
) -> Result<Vec<Event>, TestClientError> {
|
|
let deadline = tokio::time::Instant::now() + timeout_dur;
|
|
let mut events = Vec::new();
|
|
let mut seen = HashSet::new();
|
|
|
|
loop {
|
|
let remaining = deadline
|
|
.checked_duration_since(tokio::time::Instant::now())
|
|
.unwrap_or(Duration::ZERO);
|
|
|
|
if remaining.is_zero() {
|
|
return Err(TestClientError::Timeout);
|
|
}
|
|
|
|
match self.inner.next_event(remaining).await? {
|
|
RelayMessage::Event {
|
|
subscription_id,
|
|
event,
|
|
} if subscription_id == sub_id => {
|
|
if seen.insert(event.id) {
|
|
events.push(*event);
|
|
}
|
|
}
|
|
RelayMessage::Eose { subscription_id } if subscription_id == sub_id => {
|
|
return Ok(events);
|
|
}
|
|
_ => {}
|
|
}
|
|
}
|
|
}
|
|
|
|
/// Closes the WebSocket connection gracefully.
|
|
pub async fn disconnect(self) -> Result<(), TestClientError> {
|
|
self.inner.disconnect().await?;
|
|
Ok(())
|
|
}
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
use nostr::{EventBuilder, Keys, Kind, RelayUrl, Tag};
|
|
|
|
#[test]
|
|
fn parse_relay_messages() {
|
|
struct Case {
|
|
json: &'static str,
|
|
check: fn(RelayMessage),
|
|
}
|
|
|
|
let cases = vec![
|
|
Case {
|
|
json: r#"["OK","abc123",true,""]"#,
|
|
check: |msg| match msg {
|
|
RelayMessage::Ok(ok) => {
|
|
assert_eq!(ok.event_id, "abc123");
|
|
assert!(ok.accepted);
|
|
assert_eq!(ok.message, "");
|
|
}
|
|
_ => panic!("expected Ok"),
|
|
},
|
|
},
|
|
Case {
|
|
json: r#"["OK","def456",false,"blocked: not authorized"]"#,
|
|
check: |msg| match msg {
|
|
RelayMessage::Ok(ok) => {
|
|
assert_eq!(ok.event_id, "def456");
|
|
assert!(!ok.accepted);
|
|
assert_eq!(ok.message, "blocked: not authorized");
|
|
}
|
|
_ => panic!("expected Ok"),
|
|
},
|
|
},
|
|
Case {
|
|
json: r#"["EOSE","sub1"]"#,
|
|
check: |msg| match msg {
|
|
RelayMessage::Eose { subscription_id } => assert_eq!(subscription_id, "sub1"),
|
|
_ => panic!("expected Eose"),
|
|
},
|
|
},
|
|
Case {
|
|
json: r#"["NOTICE","hello from relay"]"#,
|
|
check: |msg| match msg {
|
|
RelayMessage::Notice { message } => assert_eq!(message, "hello from relay"),
|
|
_ => panic!("expected Notice"),
|
|
},
|
|
},
|
|
Case {
|
|
json: r#"["AUTH","deadbeef1234"]"#,
|
|
check: |msg| match msg {
|
|
RelayMessage::Auth { challenge } => assert_eq!(challenge, "deadbeef1234"),
|
|
_ => panic!("expected Auth"),
|
|
},
|
|
},
|
|
Case {
|
|
json: r#"["CLOSED","sub2","auth-required: must authenticate"]"#,
|
|
check: |msg| match msg {
|
|
RelayMessage::Closed {
|
|
subscription_id,
|
|
message,
|
|
} => {
|
|
assert_eq!(subscription_id, "sub2");
|
|
assert_eq!(message, "auth-required: must authenticate");
|
|
}
|
|
_ => panic!("expected Closed"),
|
|
},
|
|
},
|
|
];
|
|
|
|
for case in cases {
|
|
let msg = parse_relay_message(case.json).expect(case.json);
|
|
(case.check)(msg);
|
|
}
|
|
}
|
|
|
|
#[test]
|
|
fn parse_unknown_message_type_errors() {
|
|
let result = parse_relay_message(r#"["UNKNOWN","data"]"#);
|
|
assert!(result.is_err());
|
|
}
|
|
|
|
#[test]
|
|
fn auth_event_has_relay_and_challenge_tags() {
|
|
let keys = Keys::generate();
|
|
let relay_url: RelayUrl = "ws://localhost:3000".parse().unwrap();
|
|
let event = EventBuilder::auth("test-challenge", relay_url)
|
|
.sign_with_keys(&keys)
|
|
.unwrap();
|
|
|
|
assert_eq!(event.kind, Kind::Authentication);
|
|
|
|
let tags: Vec<Vec<String>> = event
|
|
.tags
|
|
.iter()
|
|
.map(|t| t.as_slice().iter().map(|s| s.to_string()).collect())
|
|
.collect();
|
|
|
|
assert!(
|
|
tags.iter().any(|t| t.len() >= 2 && t[0] == "relay"),
|
|
"missing relay tag"
|
|
);
|
|
assert!(
|
|
tags.iter()
|
|
.any(|t| t.len() >= 2 && t[0] == "challenge" && t[1] == "test-challenge"),
|
|
"missing challenge tag"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn text_event_carries_h_tag() {
|
|
let keys = Keys::generate();
|
|
let channel_id = "my-channel-123";
|
|
let h_tag = Tag::parse(["h", channel_id]).unwrap();
|
|
let event = EventBuilder::new(Kind::Custom(9), "hello")
|
|
.tags([h_tag])
|
|
.sign_with_keys(&keys)
|
|
.unwrap();
|
|
|
|
assert_eq!(event.kind, Kind::Custom(9));
|
|
let tags: Vec<Vec<String>> = event
|
|
.tags
|
|
.iter()
|
|
.map(|t| t.as_slice().iter().map(|s| s.to_string()).collect())
|
|
.collect();
|
|
|
|
assert!(
|
|
tags.iter()
|
|
.any(|t| t.len() >= 2 && t[0] == "h" && t[1] == channel_id),
|
|
"missing h tag"
|
|
);
|
|
}
|
|
|
|
/// The relay registers a REQ's live subscription before its history
|
|
/// query, so an event whose fan-out task runs late arrives once live and
|
|
/// once from history, both before EOSE.
|
|
#[tokio::test]
|
|
async fn collect_until_eose_keeps_one_copy_of_each_event_id() {
|
|
use futures_util::SinkExt;
|
|
use tokio_tungstenite::tungstenite::Message;
|
|
|
|
let keys = Keys::generate();
|
|
let raced = EventBuilder::new(Kind::TextNote, "raced")
|
|
.sign_with_keys(&keys)
|
|
.unwrap();
|
|
let other = EventBuilder::new(Kind::TextNote, "other")
|
|
.sign_with_keys(&keys)
|
|
.unwrap();
|
|
let frames = [
|
|
json!(["EVENT", "sub", raced]),
|
|
json!(["EVENT", "elsewhere", other]),
|
|
json!(["EVENT", "sub", other]),
|
|
json!(["EVENT", "sub", raced]),
|
|
json!(["EOSE", "sub"]),
|
|
];
|
|
|
|
let listener = tokio::net::TcpListener::bind("127.0.0.1:0")
|
|
.await
|
|
.expect("bind fake relay");
|
|
let address = listener.local_addr().expect("fake relay address");
|
|
let relay = tokio::spawn(async move {
|
|
let (stream, _) = listener.accept().await.expect("accept client");
|
|
let mut ws = tokio_tungstenite::accept_async(stream)
|
|
.await
|
|
.expect("fake relay handshake");
|
|
for frame in frames {
|
|
ws.send(Message::Text(frame.to_string().into()))
|
|
.await
|
|
.expect("send frame");
|
|
}
|
|
ws
|
|
});
|
|
|
|
let mut client = BuzzTestClient::connect_unauthenticated(&format!("ws://{address}"))
|
|
.await
|
|
.expect("connect to fake relay");
|
|
let events = client
|
|
.collect_until_eose("sub", Duration::from_secs(5))
|
|
.await
|
|
.expect("collect until EOSE");
|
|
let ids: Vec<_> = events.iter().map(|event| event.id).collect();
|
|
assert_eq!(ids, [raced.id, other.id]);
|
|
drop(relay.await.expect("join fake relay"));
|
|
}
|
|
}
|