mirror of
https://github.com/block/buzz.git
synced 2026-11-01 15:01:12 +00:00
84de626963bcbb87a68226192f091c94de0541a0
3 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
npub1jmc9dt2lyvzu3h0kxlwxt5zg4fxp9476awyxw6gwxn72g6cw7exqs64whm
|
ce406c7160 |
fix(search): restore privacy kind exclusions at the FTS storage layer
The Typesense→Postgres FTS rewrite replaced out-of-band indexing with
`search_tsv TSVECTOR GENERATED ALWAYS AS (to_tsvector('simple', content))
STORED` over every row. The old relay (handlers/event.rs:287 on main)
deliberately skipped search-indexing for three kind classes, and the new
search query layer has no kind exclusion — so gift wraps, DM-visibility
snapshots, and event reminders were all in the FTS index.
Fix at the storage layer (option A — single source of truth, zero
app-layer drift across multiple search call sites): make the generated
column yield `NULL::tsvector` for excluded kinds via a CASE expression.
A NULL tsvector never matches `@@`, so excluded rows are structurally
unsearchable.
Excluded set, parity with main's `handlers/event.rs:287-290`:
- 1059 KIND_GIFT_WRAP (NIP-17 ciphertext)
- 30300 KIND_EVENT_REMINDER (AUTHOR_ONLY_KINDS — defense in depth)
- 30622 KIND_DM_VISIBILITY (per-viewer private hide state)
Constants are inlined in the migration with a comment naming the
`buzz_core::kind` names: sqlx migrations are frozen SQL and can't
`use buzz_core::kind`; importing core into a migration would be worse
drift than the inline-with-comment shape.
Three coupled layers:
1. Schema CASE in migrations/0001_initial_schema.sql. The 0001 schema
was consolidated by Max in
|
||
|
tlongwell-block
|
38708b84ad |
rewrite(search): ChannelScope enum closes ChannelLessOnly fence hole
The legacy 2x2 `(channel_ids: Option<Vec<Uuid>>, include_channel_less: bool)` shape could not unambiguously express "channel-less events only" — both `Some(vec![]) + true` and `None + true` fell into the no-constraint branch, silently broadening to all community channels rather than restricting to `channel_id IS NULL`. That matched the legacy Typesense `channel_id:=__global__` sentinel one way (per-channel + global) but not the other (global only).
Replace with a single `ChannelScope` enum whose four variants are 1-to-1 with the legacy `(accessible_channels, include_global)` matrix:
- non-empty + true -> ChannelsOrChannelLess(accessible)
- non-empty + false -> Channels(accessible)
- empty + true -> ChannelLessOnly (the variant the old shape could not express)
- empty + false -> caller short-circuits to EOSE, doesn't call search
Emitted SQL fragments are byte-identical to the legacy match for the three carry-over cases; `ChannelLessOnly` adds `AND channel_id IS NULL` — the fence the old type could not express.
Verification:
- Full package `cargo test -p buzz-search -- --include-ignored --test-threads=1`: 9/9 green (8 existing + 1 new `channel_less_only_excludes_per_channel_events`).
- Adversarial mutation: replaced the `ChannelLessOnly` SQL emission with a no-op (the buggy semantic the old shape produced); new test went RED with 3 hits instead of 1, restored, green again. The fix is the emitted predicate, not the variant name.
- clippy -D warnings clean; fmt clean.
- Empty-vec edge cases are intentionally not special-cased: `Channels(vec![])` emits `channel_id = ANY('{}')` (false-for-all, zero hits, preserves the old early-return semantic via SQL); `ChannelsOrChannelLess(vec![])` is equivalent to `ChannelLessOnly`.
Coordinated with Eva ahead of relay-wiring sweep at req.rs and bridge.rs so call sites land against the final type, not the buggy one.
(cherry picked from commit
|
||
|
tlongwell-block
|
c2477ba34f |
rewrite(search): Postgres FTS, community-scoped, drop Typesense
The Lane-0 freeze landed `events.search_tsv TSVECTOR GENERATED ALWAYS AS
(to_tsvector('simple', content)) STORED` + `GIN (search_tsv)` directly in
the schema. With that in place the entire Typesense apparatus is dead
weight: there is nothing to index out-of-band, no consistency window to
reason about, no client-forgeable index/content drift. Indexing is the
SQL write.
This rewrites `crates/buzz-search/` from scratch around that:
- `query.rs`: one SQL builder. `community_id = $ctx` is the first
predicate of every executed statement and is unconditional —
`SearchQuery` requires a `CommunityId` at the type level (no
construction path omits it). `search_tsv @@ websearch_to_tsquery(...)`
is the FTS predicate; `ts_rank_cd DESC, created_at DESC, id` is the
order. Channel scope replaces today's `__global__` sentinel with
`channel_id IS NULL`. Empty query short-circuits without a roundtrip.
- `lib.rs`: thin `SearchService { pool }`. Takes `&PgPool` directly so
the crate stays a leaf — no buzz-db dependency. Re-exports
`CommunityId` for callers that need to mint the fence.
- `error.rs`: collapsed to one variant (`Db(sqlx::Error)`); empty
queries are not errors.
- Deleted `collection.rs` and `index.rs` (Typesense HTTP client and
indexer). Dropped `reqwest`/`serde`/`serde_json`/`chrono`/`nostr`
from `Cargo.toml`.
- Added `tests/fts_integration.rs` — 8 integration tests against real
Postgres, each on its own throwaway schema applying the frozen
`migrations/0001_initial_schema.sql` via `include_str!`. The
load-bearing one is `search_does_not_return_other_community_events`:
mutating the `community_id = $ctx` predicate to `1=1` makes that
test go red (verified, then reverted) — the fence bites where it
has to.
Conformance row 50 — search re-auth and one-shot NIP-50 — is unchanged
in shape: the relay refetches canonical events per hit through buzz-db's
scoped fetcher and runs the access predicate. Search is never the
access boundary; this crate just returns candidate ids. The row's
Typesense prose rewrite is owned by Eva's integration lane (one writer
per path).
EXPLAIN ANALYZE evidence on a 200k-row community confirms the planner
picks `Bitmap Index Scan on events_p<...>_search_tsv_idx` for the
populated partition (full plan in RESEARCH/SEARCH_LANE_FTS_EXPLAIN.md
in the workspace). Single-column `GIN (search_tsv)` is sufficient at
this scale — no `btree_gin` needed (Max's caveat holds).
Cross-lane removals owed to Eva (relay-wiring lane, not this commit):
- relay state.rs: remove `search_index_tx` mpsc + worker
- relay main.rs: remove `search.ensure_collection()` call
- relay handlers/event.rs: remove `search_index_tx.send()`
- relay api/bridge.rs::handle_bridge_search: rewrite to new API
- relay handlers/req.rs::handle_search_req: rewrite to new API
- relay handlers/req.rs::build_search_channel_scope_filter: delete
- relay bin/reindex_kind0.rs: delete
- docker-compose.yml: drop typesense service + volume
- docs/multi-tenant-conformance.md row 50: rewrite Typesense prose
Tests: `cargo test -p buzz-search --test fts_integration --
--include-ignored --test-threads=1` — 8 passed, 0 failed.
Clippy: `cargo clippy -p buzz-search --all-targets -- -D warnings` — clean.
(cherry picked from commit
|