mirror of
https://github.com/block/buzz.git
synced 2026-10-27 12:40:30 +00:00
f7708d0694dbbd92bb6f4cd2b6872f4bbdc3cebb
16 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Logan Johnson
|
7852b1670c |
fix(search): normalize profile name tiers in one place (#8334)
🤖 ## Summary Profile name search (`POST /profiles/search`, added in #8301) sorts results into tiers: an exact name first, then a name that starts with the query, then a name with a later word that starts with it. This PR stacks on #8301 and fixes two review items. **Tier normalization in one place.** Before this change, the query was lower-cased in Rust (`to_lowercase`), and the names were lower-cased in SQL (`lower()`). These can disagree. Under a `C` collation, PostgreSQL's `lower()` changes only ASCII letters, while Rust also lower-cases letters like `É`. So a search for `Émile` did not put the profile named `Émile` in the exact tier. Now the query and both name fields go through the same SQL expression: lower-case, collapse whitespace runs to one space, then trim. The query is bound once and both sides compare equal by construction. The trim also moved after the collapse, so surrounding whitespace that the regex turns into a space is now trimmed too. **Ranking wording.** The doc comment and `docs/profile-search.md` said "ties keep the NIP-50 order". That was wrong. Ties are ordered by `ts_rank_cd` over the two name fields only (not the whole profile, as in NIP-50), then newest first, then event id. Out of scope, not changed here: matching names that contain escaped Unicode or newlines, and any new name index. ## Related issue Review on #8301: https://github.com/block/buzz/pull/8301#pullrequestreview-5524491230 ## Testing - New case in `profile_names_match_name_words_only`: profiles `Émile` and a newer `Émilee`, query `Émile`. The exact match must come first. - Ran the Postgres integration tests against a `postgres:17-alpine` database created with `--locale=C`: - this branch: `profile_names_match_name_words_only` and `profile_names_survive_unparsable_content` pass; - #8301 code with the new case: `profile_names_match_name_words_only` fails (`Émilee` sorts before `Émile`). - Nine other tests in that file fail locally on both this branch and #8301 because they reuse one database (duplicate community hosts). CI gives each test its own database. - `cargo fmt --check`, `cargo clippy -p buzz-search --all-targets -D warnings`, and `cargo test -p buzz-search`: clean. - The pre-push hook's `rust-tests` and `desktop-tauri-checks` passed. Signed-off-by: Logan Johnson <loganj@squareup.com> Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz> |
||
|
Logan Johnson
|
f148c0f22c |
feat(relay): add profile search over name and display_name only (#8301)
🤖 > **Review as a pair with block/buzz-app#1084** (the app change that uses this endpoint). Either can merge first: the app falls back to the old search on relays without this endpoint. ## Summary Searching for a person by name over the relay's general search returns profiles that only *mention* the text somewhere else: in an avatar image embedded as text, in the bio, or in the verification address. A search for "wes" could fill every result page with profiles whose avatar data happens to contain "Wes", and never return the person actually named Wes. This adds a relay endpoint, `POST /profiles/search`, that looks only at a profile's **name** and **display name**. Every other profile field is ignored. **Matching and paging are the same as the relay's existing full-text profile search** (NIP-50 search over kind:0 profiles with `search_mode: "prefix"`). A client can switch to this endpoint without a visible change in which people a query finds: - Each query word must match a whole word of the name; the last word can be a prefix. Words can be in any order, so "johnson wes" finds "Wes Johnson". Case is ignored. - Pages work the same way: up to 100 results per page, up to page 1000. - The results are exactly the old results minus profiles that match only outside their names. The order is better for people search: exact name first, then names that start with the query, then names with a later word that starts with it, then other matches. Ties are ordered by `ts_rank_cd` over the two name fields only (not the whole profile, as in NIP-50), then newest first, then event id. The response is a JSON array of the original signed profile events (Nostr kind:0), so clients can verify them as they do today. The buzz-app client change is block/buzz-app#1084. ## Details - Request: `{"query": "wes", "limit": 20, "page": 1}`. `query` is 1–100 characters after trimming; `limit` is 1–100 (default 20); `page` is 1–1000 (default 1); unknown fields are rejected with 400. - The search reuses the full-text search's own query builder (`buzz-search`), so the word rule cannot drift from it. The existing full-text index on events stays the access path; the name check runs only on rows the index already matched. There is no new index. One small migration (`0061`) adds the function `profile_name_fields()`; it reads no table data, so it runs instantly. - Only current profiles count: a replaced profile stops matching. Profiles that are not a JSON object, or whose name fields are not strings, never match. Content that PostgreSQL cannot read as `jsonb` (for example a `\^@` escape or very deep nesting) also never matches, and it cannot make the search fail for everyone else. - The handler loads the matched events through the normal routed event read and applies the same reader visibility check as other reads, in search order. - Access uses the same checks as the relay's GIF search endpoint: a signed HTTP request (NIP-98), identity admission (NIP-FI), rate limit, replay protection, and relay membership. That code moved out of `gifs.rs` into one shared helper in `bridge.rs`; GIF behavior is unchanged. - The route is added to the existing protected-route and NIP-FI shadow/enforce route tables. Docs: `docs/profile-search.md`. - Two test-only lines get `sadscan:disable` comments, as other fixtures already do: an existing public ES256 test key in `router.rs`, and the local Postgres test URL in the search integration test. ### Related issue None found. ### Testing - Postgres test for the search query (`profile_names_match_name_words_only`): only name fields match (avatar data URL, `about`, `nip05`, `banner`, non-string names and non-object content are excluded); words in any order and across both name fields; whole words only for completed words; ranking order; pages split without overlap or gaps; replaced profile; other community; `%` literal. - Postgres end-to-end test through the router: an admitted member gets only the name-matching signed event, and the returned event still verifies. - Request validation unit tests; existing GIF admission tests and router route-table tests pass. - Local end-to-end run with buzz-app#1084 against a relay built from this head, with about 195 seeded profiles. For every query tried, the endpoint's results were a subset of the full-text search's results; the only profiles missing were avatar-only matches. --------- Signed-off-by: Logan Johnson <loganj@squareup.com> Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz> |
||
|
Kalvin C
|
cdfc38ecbd |
feat(search): add newest-first sort to bridge search (#8371)
## Summary Adds an opt-in newest-first order to bridge search, so clients can offer "Most recent" alongside relevance. - `buzz-search`: `SearchQuery` gains `sort: SearchSort`. `Relevance` keeps the existing `ts_rank_cd` ordering. `Newest` orders hits by `created_at DESC, id`, and `page` steps through that order. Newest skips the exact-profile-lexeme prioritization, which only applies to relevance ranking. - Bridge `POST /query`: search filters accept `search_sort: "newest"`. When the field is missing or has another value, results stay in relevance order. Community scope, channel intersection and per-hit authorization are unchanged. - NIP-11 lists `buzz-search-sort` in `supported_extensions`. Clients only send the field to relays that advertise it, so they never get relevance results labelled as newest. - WebSocket NIP-50 search stays relevance-only. `NOSTR.md` documents the bridge extension. ### Related issue N/A ### Testing - New Postgres integration test `newest_sort_pages_by_time_then_id_within_scope`. It pages a channel in newest order, including an equal-timestamp tie split across a page boundary and a newer hit outside the channel scope. It checks that the pages join up in `(created_at desc, id asc)` order and that relevance still ranks the strongest match first. - Unit tests: - `bridge_search_sort_extension_selects_newest_only_when_named` - `advertises_nip_cl_and_search_sort_extensions` - Ran the buzz-search Postgres integration suite, the relay `nip11` and `bridge_search` unit tests, `cargo fmt --check`, and `cargo clippy -p buzz-search -p buzz-relay --all-targets -D warnings` on Linux. - Local relay exercised through signed `POST /query`: - NIP-11 lists `buzz-search-sort`. - Relevance and newest return different orders. - Newest pages 1–3 return 25 hits with no gaps or duplicates, including same-second ties across page boundaries. - A non-member gets no hits from a private channel; a member gets all of them. - Client integration is in a companion block/buzz-app change. It was exercised end to end against this relay through the production broker: switching sort, paging, scope changes and capability fallback. Deployment note: clients only offer newest sorting after a relay with this change advertises `buzz-search-sort`. --------- Signed-off-by: Kalvin Chau <kalvin@block.xyz> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Bradley Axen
|
12670bd0f0 |
feat(relay): implement NIP-AR channel artifacts (#7919)
## Summary Implements NIP-AR channel artifacts (spec merged in #7791) on the relay, after a simplicity pass over the earlier two-agent implementation. An artifact is an editable record (kind 45010) with a stable identity (`d`), one home channel (`h`), and full-snapshot revisions chained by `prev`: - **Writes:** one transaction per write. It locks the artifact's head row, requires `prev` to name the current revision, records the revision in a ledger that survives retention, and advances the head. Two competing edits can't both win. A stale edit gets `conflict: artifact head changed`. Resubmitting an accepted event succeeds without applying it twice. - **Permissions:** writes pass the same gates as a kind 9 message in `h`. A move also passes them in the source channel. The move stores the destination snapshot and a relay-signed kind 45011 removal marker for the source in the same transaction, so both channels recover by replay. The marker never names the destination. - **Moderation:** redaction is the ordinary kind 9005 removal. The ledger keeps the ID so `prev` checks still work. Redacting the current revision retires the artifact. - **Reads:** HTTP `/query` and `/count` accept explicit `artifact: current|history` filters with exact multi-character tag matching (e.g. `#assignee`). Generic REQ and search return only current revisions. Kind 5 against artifacts is rejected. NIP-11 advertises the limits. - **Spec:** NIP-AR updated to match: kind 9 permission checks, 9005 redaction and retirement, no creator-only delete rule. A separate small commit routes `insert_channel_head_checked` through the metered writer pool (`acquire_writer`), like every other event write in that module. ### Related issue Follows #7791. No separate issue found. ### Testing - Clippy clean. - Unit lane passes. Failures seen along the way came from agent env leaking into `buzz-acp` tests (they pass in a clean env) and known timing flakes (`keepalive_resets_idle_past_deadline`, two `buzz-agent` databricks tests that passed on retry). This diff doesn't touch those crates. - Postgres lane passes, including new `artifact_postgres_tests` in `buzz-db` and `buzz-relay`. Three presence tests need `REDIS_URL` set. - Docker smoke against a fresh local relay, 29/29: create and idempotent resubmit, update and stale conflict, current, `#assignee` and history queries, generic REQ, kind 5 rejection, delete and restore, move with source marker, private-channel isolation, and 9005 retirement. - A pasteable curl walkthrough of the same flows was shared with the reviewer. Draft until the human hands-on check (AGENTS.md step 3) is confirmed. The pre-push hook was skipped because every lane above had already run on this tree. CI is the gate. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Honey <8e307ae0076a4dab6b94b036ea3edc7e08f823a625269c1e6919e881a048b4d2@buzz.block.builderlab.xyz> Signed-off-by: murderbot <3754f8729004d95654c46dbab3129e4ab9ef05cc2534e2a3fbfc155983bd637b@buzz.block.builderlab.xyz> Co-authored-by: Honey <8e307ae0076a4dab6b94b036ea3edc7e08f823a625269c1e6919e881a048b4d2@buzz.block.builderlab.xyz> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: Smartie <fb3185faacfc3760b6b4fe0085a8878d8dd537f7a7671c7d3184fe1aa1b04df4@buzz.block.builderlab.xyz> Co-authored-by: Fizz <400e8babadcee6a7f420103f10a2849d84c4a9c71d5bd04f3948c814216648a3@buzz.block.builderlab.xyz> Co-authored-by: murderbot <3754f8729004d95654c46dbab3129e4ab9ef05cc2534e2a3fbfc155983bd637b@buzz.block.builderlab.xyz> |
||
|
Luke Tornquist
|
bd73490418 |
ci: run PostgreSQL tests in isolated lane (#6730)
## Why PR #6660 introduced four PostgreSQL-backed persistence tests and a focused CI selector, but an exact test-name list does not automatically cover future database tests. Broad ignored-test execution also exposed shared-schema races and ambiguity between desired-state and migration-applied schema expectations. ## What - Establish a discoverable convention: PostgreSQL unit modules use `postgres_tests`, PostgreSQL integration binaries use a `postgres_` prefix, and unrelated external-infrastructure tests use an `external_infra_` prefix. - Add a dedicated nextest PostgreSQL profile and archive-backed CI job covering the relevant crates without enumerating test names. - Create a run-scoped desired-state source database and a unique PostgreSQL database per test process for parallel-safe isolation, with cleanup on success, failure, or interruption. - Route destructive migration and migration-parity tests to clean `template0` databases while desired-state tests clone the desired-state database. - Document discovery, schema modes, required role privileges, and a bounded portable local runner. - Keep the existing infrastructure-free unit-test jobs unchanged. This is intentionally limited to test and CI harness behavior. It does not move production database code, change persistence semantics, or implement issue #20's broader shared test-utility refactor. ## Risk Low production risk because all changes are confined to tests, CI configuration, documentation, and test harness scripts. Remaining operational risks: - Cleanup retries `dropdb --force` five times across roughly four seconds. Exhaustion warns with the database name but deliberately does not mask the test result; individual PostgreSQL diagnostics are suppressed. - The seven-package boundary is duplicated between the runner and CI archive and must remain synchronized if PostgreSQL tests move to a new crate. - Portable SHA fallback branches were exercised on Linux; no macOS Blox workstation was needed for this Linux CI artifact. - GitHub's unchanged generic Unit Tests job currently fails while cold-linking `buzz-voice` because `sherpa-onnx-c-api` is absent. The branch changes no voice/build/toolchain inputs; the exact parent previously passed that job, and Blox passes the same infrastructure-free suite (1,450/1,450). A prior-head one-job retry and the final-head run both reproduced the hosted-runner failure. ## Verification Author Blox workstation `2020088`: - `cargo fmt --all -- --check` - `just clippy` - `just test-unit`: 1,450 passed; PostgreSQL tests remain skipped in fast jobs - PR #6660 focused tests: 4/4 passed through the final runner - Full lane: 282/282 passed in 10.087s with a non-superuser role limited to `CREATEDB`, `CREATEROLE`, and `pg_read_all_stats` - 272 desired-state tests - 10 migration-applied tests - 6 explicitly filtered external-infrastructure tests - Cleanup fault injection: a deliberately underprivileged 282-test run produced the expected 277 passes and 5 failures, and the post-run catalog audit found zero lane databases after retry cleanup. The restored successful run also left zero lane databases. GitHub exact final head `92c231e48f299ea2af23817763f5ae80ac013d68`: - [PostgreSQL Tests](https://github.com/block/buzz/actions/runs/32789030203/job/97630827823): 282/282 passed across 10 binaries in 98.886s; 1,240 skipped, including 6 via the profile filter - The shared relay/PostgreSQL archive predecessor completed successfully - [Unit Tests](https://github.com/block/buzz/actions/runs/32789030203/job/97626930420): unrelated `sherpa-onnx-c-api` native-link failure described above - Independent exact-head review on separate Blox workstation `2022762`: no substantive findings after all initial findings were addressed. The reviewer independently verified cleanup retry behavior, exit-status preservation, shell portability, syntax, and a clean exact-head worktree. ## References - Stacked on #6660 at exact head `561de54be4d9c2b622b7c2aa5b61bc3068f47e2a` - https://github.com/TheSentinel454/buzz/issues/20 - https://github.com/TheSentinel454/buzz/issues/20#issuecomment-5398526792 Generated with Codex. ## Update — August 24, 2026 review follow-up - Centralized the repeated PostgreSQL test URL resolution in crate-local, test-only helpers without introducing the broader shared utility refactor from issue #20. - Restored descriptive hybrid/Redis test function names. Structural `external_infra_*_tests` modules now own exclusion, and the nextest filter only recognizes module path segments. - Added a three-second source guard that scans every Rust file and fails CI when an ignored PostgreSQL test would be omitted or an external-infrastructure test would be included. Fixture tests cover accepted modules/binaries and both failure modes. - Removed the two implementation-plan documents. Performance profiling on Blox workstation `2027352` (same prebuilt archive and cargo-nextest 0.9.143 for every comparison): - Current per-test database model, 8 workers: 10.15–10.40s across three runs. The 283 `createdb` calls and 284 `dropdb` calls consumed 28.85–29.46 aggregate seconds, about 39–40% of aggregate test-process duration. - Reusing one database per worker without cleanup: 6.55–6.84s, about 35% faster, but all three runs failed because global matcher-queue tests inherited a quiesced community from an earlier test; the failing test varied with scheduling. - Reusing workers with `TRUNCATE … RESTART IDENTITY CASCADE`: all 282 tests passed, but total time regressed to 10.47–10.67s. The 265 truncate calls consumed 24.34–25.45 aggregate seconds, so truncation merely replaced most clone/drop cost. - Raising concurrency to 16 workers reduced one isolated-database run to 8.79s, but 24 workers exposed a cluster-global `pg_stat_activity` race. The profile remains at the proven-safe 8 workers. Decision: retain unique per-test databases. Harness cleanup uses `dropdb --if-exists --force`; it drops the database rather than truncating tables or deleting rows. Verification at local branch head `60086c1f6ee34a193c342255a5e8e6293b60e988`: - Discovery inventory: exactly 282 intended tests; all 6 hybrid/Redis tests present and structurally excluded. - PR #6660 focused persistence tests: 4/4 passed. - Full PostgreSQL lane: 282/282 passed three times; 1,240 skipped each run; wall time 9.72–9.96s without profiling shims. - Non-ignored tests with a desired-state database: 1,234/1,234 passed; one mesh-demo timeout from the first run passed immediately in isolation. - `cargo fmt --all -- --check` and all-target/all-feature clippy for `buzz-db`, `buzz-deletion`, and `buzz-relay` passed. Final independent re-review at exact head `42097c0136ab3fa8f0efe5720ae829c04575b9b6` on separate Blox workstation `2028455`: no substantive residual findings. The reviewer independently exercised ordinary strings, zero/one/three-hash raw strings, line/block comments, and string-contained lookalikes, then reran the full 353-file scan, shell syntax, Python AST parsing, and diff check. Independent review found that the source guard's initial regular expression could miss raw-string ignore reasons and treat a commented attribute as real. Final head `42097c0136ab3fa8f0efe5720ae829c04575b9b6` parses valid ordinary/raw Rust string literals only at attributes found in comment-sanitized source; regression fixtures cover both cases. The fixture suite, Python compilation, shell syntax, diff check, and full 353-file repository scan pass on Blox. ## Update — August 25, 2026 restack PR #6660 merged, so this follow-up was rebased from its former exact parent onto current `origin/main` (`8d2d0ff5ad42733e9949442c4b6358d0ba87f9a8`). The final candidate head is `05dcc2ab28948e3ab79bb44839a69f2ba44648a2`; the PR no longer carries #6660's pre-squash history. Fresh exact-head verification on Blox workstation `2028572`: - Discovery guard passed across 353 Rust files; inventory remained exactly 282 intended PostgreSQL tests. - PR #6660 focused persistence tests: 4/4 passed. - Full PostgreSQL lane: 282/282 passed three times; 1,243 skipped each run; wall time 9.70–9.87s. - `cargo fmt --all -- --check` and all-target/all-feature clippy for `buzz-db`, `buzz-deletion`, and `buzz-relay` passed. ## Update — August 25, 2026 final hosted verification - [GitHub PostgreSQL Tests](https://github.com/block/buzz/actions/runs/32801679226/job/97667835080): 282/282 passed in 53.787s; 1,243 skipped. - The final guard scanned all 353 Rust files successfully before the hosted lane ran. - The shared relay/archive prerequisite and infrastructure-free Unit Tests job both passed at exact final head `42097c0136ab3fa8f0efe5720ae829c04575b9b6`. --------- Signed-off-by: Luke Tornquist <tornquist@squareup.com> Signed-off-by: tornquist <tornquist@squareup.com> |
||
|
Tyler
|
b3baa56ba8 |
fix(db): exclude kind:30179 ciphertext from brownfield FTS (#6822)
## Problem Kind `30179` (NIP-PMA private managed agent — NIP-44 ciphertext carrying the agent nsec, env vars, prompt) joined `AUTHOR_ONLY_KINDS` in #4593, but only **fresh** installs stopped indexing it: - `migrations/0008` installs the positive FTS allowlist only when `events` is empty; - `migrations/0014` wraps the retained brownfield expression for `30350` alone; - `schema/schema.sql` still carries the legacy negative skip-set without `30179`. So a relay upgraded in place keeps tokenizing `30179` ciphertext into `events.search_tsv`. Found by Wren while reviewing #4999 (which activates publication of 30179 from Desktop). No readable leak: `/query` applies `search_hit_accepted` + `event_visible_to_reader` before serialization and live foreign searches against builderlab returned `[]` — this is the storage-layer privacy invariant (`docs/nips/NIP-PMA.md` deployment step 2) plus wasted FTS work. ## Fix - `migrations/0033_private_managed_agent_fts.sql` — same shape as 0014: capture the current generated expression via `pg_get_expr`, drop/re-add `search_tsv` wrapped with `kind = 30179 → NULL`. Every other kind keeps whatever policy the database already had (fresh allowlist or brownfield skip-set). Rebuilds the GIN index; no heap-wide policy rewrite. - `schema/schema.sql` — add `30179` to the desired-state skip-set. ## Tests - `populated_upgrade_preserves_search_policy_except_for_private_kinds` (renamed from `…_push_leases`): provisions the **legacy negative expression** (migrations 1–7), inserts kind 1 / 30179 / 30350 rows, checkpoints after 0014 (30179 still searchable, 30350 not), then runs to head and asserts 30179 is NULL. **Proven red** with the 0033 file removed: `left: [(1, Some(true)), (30179, Some(true)), (30350, None)]`. - `embedded_migrator_contains_consolidated_initial_schema`: count 32→33, asserts 0033 shape and `schema.sql` parity. - `buzz-search/tests/fts_integration.rs` setup applies 0033 so the FTS tripwires run against the real chain. ## Verification (HEAD |
||
|
Luke Tornquist
|
113a33b7e4 |
Add database pressure observability (#6700)
## Why Database pressure currently collapses several distinct delays into one symptom. This adds the evidence layer needed to distinguish pool acquisition wait, logical database operation time, advisory-lock wait, and selected transaction duration before changing timeout or retry policy. This is the phase 2 Lane A observability bundle for [#26](https://github.com/TheSentinel454/buzz/issues/26), [#28](https://github.com/TheSentinel454/buzz/issues/28), and [#33](https://github.com/TheSentinel454/buzz/issues/33). It is stacked on #6668. ## What - Record explicit reader/writer checkout wait and acquisition outcomes with `buzz_db_pool_acquire_wait_seconds` and `buzz_db_pool_acquisitions_total`. - Extend the compile-time `#[datastore_span(name = "...")]` seam with `buzz_db_operation_duration_seconds`, so operation labels remain static source literals instead of request data. - Route correctness-critical replacement, membership, push-gate, deletion, and migration/schema-safety advisory locks through one observer without changing their SQL, order, scope, or blocking behavior. - Measure six internally owned transaction lifetimes with `buzz_db_transaction_duration_seconds`, starting after `BEGIN` succeeds and ending after explicit commit/rollback or scope exit. - Emit root slow-operation warnings at 500 ms, logging the first slow completion and then 1/100 per call site with only `operation`, `outcome`, and `elapsed_ms`. - Document names, units, fixed label vocabularies, measurement boundaries, and blind spots in this PR description. Fixed labels are deliberately small: - `pool_role`: `writer`, `reader` - `lock_type`: `replacement`, `membership`, `push_gate`, `deletion`, `migration_schema_safety` - `outcome`: `success`, `error`, `timeout` where SQLx/PostgreSQL can distinguish it accurately - `operation`: compile-time datastore names plus the six closed transaction operation names documented in the runbook No metric or slow warning contains community IDs, event IDs, event kinds, coordinates, d-tags, SQL/query text, query IDs, returned errors, or event content. ## Coverage boundaries - Operation duration is the complete annotated logical function body, not pure SQL execution; it may include implicit checkout, lock wait, nested operations, and application work. Cancelled futures do not reach its completion hook. - Pool timing covers explicit helper checkouts, including proved-reader routing and selected writer-owned transactions. Implicit SQLx checkout through `&PgPool` remains folded into operation duration. - Lock timing covers application-side blocking locks in the five named families. Trigger/stored-procedure locks, channel-TTL locking, the usage try-lock, and the audit service session lock remain outside this slice. - Transaction timing covers only the six wholly owned boundaries documented in the runbook. It excludes pool wait, `BEGIN`, asynchronous rollback cleanup after an early return, and caller-owned `Db::begin_transaction` lifetime. ## Relationship to #6229 #6229 is the incident-driven timeout precursor. This PR does not add or change `statement_timeout`, `lock_timeout`, `idle_in_transaction_session_timeout`, retries, audit durability, or client-visible conflicts. It provides the missing distributions needed to evaluate those policies later and intentionally leaves #6229's open audit retry/durability finding untouched. The branches overlap in `crates/buzz-db/src/lib.rs` and `crates/buzz-db/src/migration.rs`, so a later rebase may need textual conflict resolution, but the behavior is complementary rather than duplicated. ## Risk assessment Moderate-low. The primary risk is instrumentation overhead and added static series. Cardinality is source-bounded, slow logs are sampled/redacted root events, and the lock/transaction changes wrap existing awaits without changing policy or ordering. ## Verification Author workstation: `buzz-tornquist-db-pressure-observability` (`2010927`), exact head `d7cf833e26c528adfcde3917ded80daf6f4ddac9`, parent `6f50e6b2b2a996349149af61d35bdd6a355f77fd`. - `cargo fmt --all --check` — passed - `cargo clippy -p buzz-datastore-tracing -p buzz-db -p buzz-audit -p buzz-search -p buzz-relay --all-targets -- -D warnings` — passed - `cargo test -p buzz-datastore-tracing --quiet` — 4 passed - `cargo test -p buzz-db --quiet` — 109 passed, 200 ignored - `cargo test -p buzz-audit -p buzz-search --quiet` — 16 passed, 25 ignored - `cargo test -p buzz-relay --lib --quiet -- --test-threads=1` — 906 passed, 48 ignored - Native PostgreSQL focused tests for pool success/timeout/error, lock success/contention/timeout/error, replacement, membership serialization, push ordering, deletion fencing, migration/schema exclusion, and reader fallback — 8 passed The default-parallel relay run passed once; subsequent runs exposed the existing load-sensitive `api::mesh_demo::tests::demo_join_forwarded_arm_round_trips_echo` 504 at the end of the suite. That test passes in isolation and the full relay suite passes serially. Independent exact-head review workstation: `buzz-tornquist-db-pressure-observability-review` (`2013067`). Formatting, the same all-target clippy command, datastore instrumentation tests, DB unit tests, source privacy guards, and diff/non-goal audits passed; no review findings. Generated with Codex --------- Signed-off-by: tornquist <tornquist@squareup.com> |
||
|
Dave Grochowski
|
397796c5f3 |
feat(tracing): add PostgreSQL tracing spans (#3678)
## Why Expose PostgreSQL datastore latency within existing request traces so slow logical database operations can be identified without recording tenant data or query arguments. ## What - Add client spans around logical PostgreSQL operations across the database facade, search, audit, replica fencing, and command persistence - Use a dedicated `buzz_datastore` target and `db.system.name = "postgresql"` for filtering and backend classification - Exclude health-check database calls and scrub raw identifiers and errors from newly traced paths ## Risk Assessment Medium — this instruments frequently used datastore paths and increases trace volume when enabled, but does not change SQL execution or datastore behavior. Existing OpenTelemetry filtering controls export. ## References - Pre-push clippy and fast unit-test hooks passed Generated with Amp --------- Signed-off-by: David Grochowski <dgrochowski@squareup.com> Co-authored-by: Amp <amp@ampcode.com> |
||
|
Wes
|
3c76f682c3 |
fix(search): surface exact short profile names (#5480)
## Summary - prioritize exact whole-lexeme matches within short kind-0 prefix searches - preserve the existing prefix result set, pagination, community/channel scope, hydration, and authorization path - add a Postgres regression where newer noisy `jm…` profiles saturate the bounded page ## Why Desktop mention autocomplete starts searching after one character. The `jm` profile is indexed and matches both `jm:*` prefix search and standard full-text search, but production prefix search returns a full 50-result page without it. Raw profile JSON supplies enough unrelated `jm…` lexemes that newer equal-rank matches fill the bounded page before the exact short display name. Changing clients would leave deployed Desktop 0.5.8 installations broken. This shared search-layer compatibility fix changes ordering only for `Prefix + kinds:[0] + query length <= 2`; message search, longer profile typeahead, and agent eligibility are untouched. ## Validation At commit `ff88761135d5045139aeb3da14d08cbfba203169` with a clean worktree: - `BUZZ_TEST_DATABASE_URL=postgres://buzz:buzz_dev@localhost:5432/buzz cargo test -p buzz-search --tests -- --include-ignored` — 22 passed (3 unit + 19 Postgres integration) - `cargo clippy -p buzz-search --tests -- -D warnings` - `cargo fmt --all -- --check` - mutation check: disabling exact-lexeme priority makes `short_kind0_prefix_prioritizes_exact_lexeme_on_a_noisy_page` fail - mandatory pre-push hooks: branch-skew, Rust tests, and Desktop/Tauri checks passed ## Risk Low. The extra ordering predicate applies only to one- or two-character prefix searches restricted exactly to kind 0. It does not add candidates, bypass filters, or alter access control. Exact matches move ahead of broader prefix matches; all remaining ordering stays relevance, recency, then event ID. Signed-off-by: Wes <wesbillman@users.noreply.github.com> Co-authored-by: Carl <c7ebe626f000404285d3686e1dc74cc07cc60a9754a150041ba132e14bd3e2ec@buzz.block.builderlab.xyz> |
||
|
Tyler
|
1c006822e4 |
feat(push): add public APNs gateway (#1770)
Signed-off-by: npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757 <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@sprout-oss.stage.blox.sqprod.co> Co-authored-by: npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757 <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@sprout-oss.stage.blox.sqprod.co> |
||
|
Tyler
|
1b4703021d |
Bound NIP-RS retention and search indexing (#1771)
Signed-off-by: npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757 <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@sprout-oss.stage.blox.sqprod.co> Co-authored-by: npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757 <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@sprout-oss.stage.blox.sqprod.co> |
||
|
Will Pfleger
|
71265ca361 |
feat(nips,relay,acp): NIP-AM durable encrypted agent turn metrics (kind 44200) (#1441)
Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@sprout-oss.stage.blox.sqprod.co> Co-authored-by: npub1fgdl5qqnh3k3f2xkqrvt7cujalhm623x4s7fdjdj5yrtp5fzjl9qrjpucw <4a1bfa0013bc6d14a8d600d8bf6392efefbd2a26ac3c96c9b2a106b0d12297ca@sprout-oss.stage.blox.sqprod.co> |
||
|
Tyler
|
14fba21e57 |
Multi-tenant Buzz relay: community_id as a server-resolved key (comprehensive rewrite) (#1321)
Signed-off-by: tlongwell-block <109685178+tlongwell-block@users.noreply.github.com> Signed-off-by: npub1jh9wn95s0472h86ahapupaf7m6kx4v9sx2n0atj2hltcfer8k06s5n3pyf <95cae996907d7cab9f5dbf43c0f53edeac6ab0b032a6feae4abfd784e467b3f5@sprout-oss.stage.blox.sqprod.co> Signed-off-by: Tyler Longwell <tlongwell@block.xyz> Signed-off-by: npub1t2tgm7d8f995uqvmnm8h88sg3wnpp9a5xysjf6dg3tjmgt3ltulqdp8ehr <5a968df9a7494b4e019b9ecf739e088ba61097b4312124e9a88ae5b42e3f5f3e@sprout-oss.stage.blox.sqprod.co> Signed-off-by: npub17jjz49l9jjmhhk7cac63j8yt9z555n9cw8vk7v5jz4vzw4ppld5qgj57cc <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Eva <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Mari <95cae996907d7cab9f5dbf43c0f53edeac6ab0b032a6feae4abfd784e467b3f5@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Sami <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Max <d8473ee32b973aa31a21a65adddcc4b69cc2a8a4dee8121ecd51926e0cddbc02@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Quinn <96f056ad5f2305c8ddf637dc65d048aa4c12d7daeb8867690e34fca46b0ef64c@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Dawn <c6237ef84fa537c78dcee78efd2d4e59f728859c7f194da42ac51ededfa0be05@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Tyler Longwell <tlongwell@block.xyz> Co-authored-by: Sami <sami@sprout-oss.stage.blox.sqprod.co> Co-authored-by: npub1t2tgm7d8f995uqvmnm8h88sg3wnpp9a5xysjf6dg3tjmgt3ltulqdp8ehr <5a968df9a7494b4e019b9ecf739e088ba61097b4312124e9a88ae5b42e3f5f3e@sprout-oss.stage.blox.sqprod.co> |
||
|
Tyler
|
73cc31cc52 |
chore: remove LLM-slop comments across the codebase (#1277)
Signed-off-by: Tyler Longwell <tlongwell@block.xyz> Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Max <d8473ee32b973aa31a21a65adddcc4b69cc2a8a4dee8121ecd51926e0cddbc02@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Mari <95cae996907d7cab9f5dbf43c0f53edeac6ab0b032a6feae4abfd784e467b3f5@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Quinn <96f056ad5f2305c8ddf637dc65d048aa4c12d7daeb8867690e34fca46b0ef64c@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Sami <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Perci <5a968df9a7494b4e019b9ecf739e088ba61097b4312124e9a88ae5b42e3f5f3e@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Tyler Longwell <tlongwell@block.xyz> |
||
|
Tyler
|
65ccb1262f |
Parse Typesense multi_search errors (#1208)
Signed-off-by: npub1t2tgm7d8f995uqvmnm8h88sg3wnpp9a5xysjf6dg3tjmgt3ltulqdp8ehr <5a968df9a7494b4e019b9ecf739e088ba61097b4312124e9a88ae5b42e3f5f3e@sprout-oss.stage.blox.sqprod.co> Co-authored-by: npub1t2tgm7d8f995uqvmnm8h88sg3wnpp9a5xysjf6dg3tjmgt3ltulqdp8ehr <5a968df9a7494b4e019b9ecf739e088ba61097b4312124e9a88ae5b42e3f5f3e@sprout-oss.stage.blox.sqprod.co> |
||
|
Will Pfleger
|
d99ad131f1 |
refactor: rename sprout backend to buzz (#958)
Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <wpfleger@block.xyz> Signed-off-by: Will Pfleger <wpfleger@squareup.com> Signed-off-by: Will Pfleger <wpfleger96@gmail.com> Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@sprout-oss.stage.blox.sqprod.co> Co-authored-by: npub1fgdl5qqnh3k3f2xkqrvt7cujalhm623x4s7fdjdj5yrtp5fzjl9qrjpucw <4a1bfa0013bc6d14a8d600d8bf6392efefbd2a26ac3c96c9b2a106b0d12297ca@sprout-oss.stage.blox.sqprod.co> Co-authored-by: npub16v54tttfqacx9ycvc3k0ut0npj564ahcuajzy6qjvh57ntmsf4uq4806j2 <d32955ad69077062930cc46cfe2df30ca9aaf6f8e76422681265e9e9af704d78@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Will Pfleger <wpfleger96@gmail.com> |