mirror of
https://github.com/block/buzz.git
synced 2026-10-29 05:21:31 +00:00
fix(search): restore privacy kind exclusions at the FTS storage layer
The Typesense→Postgres FTS rewrite replaced out-of-band indexing with
`search_tsv TSVECTOR GENERATED ALWAYS AS (to_tsvector('simple', content))
STORED` over every row. The old relay (handlers/event.rs:287 on main)
deliberately skipped search-indexing for three kind classes, and the new
search query layer has no kind exclusion — so gift wraps, DM-visibility
snapshots, and event reminders were all in the FTS index.
Fix at the storage layer (option A — single source of truth, zero
app-layer drift across multiple search call sites): make the generated
column yield `NULL::tsvector` for excluded kinds via a CASE expression.
A NULL tsvector never matches `@@`, so excluded rows are structurally
unsearchable.
Excluded set, parity with main's `handlers/event.rs:287-290`:
- 1059 KIND_GIFT_WRAP (NIP-17 ciphertext)
- 30300 KIND_EVENT_REMINDER (AUTHOR_ONLY_KINDS — defense in depth)
- 30622 KIND_DM_VISIBILITY (per-viewer private hide state)
Constants are inlined in the migration with a comment naming the
`buzz_core::kind` names: sqlx migrations are frozen SQL and can't
`use buzz_core::kind`; importing core into a migration would be worse
drift than the inline-with-comment shape.
Three coupled layers:
1. Schema CASE in migrations/0001_initial_schema.sql. The 0001 schema
was consolidated by Max in 4b7654a1c (Lane-0 contract) and is
pre-deploy; editing in place rather than adding a new migration
matches the agreed shape and is covered by
`run_migrations_applies_consolidated_initial_schema_on_fresh_database`.
2. buzz-search/tests/fts_integration.rs — new test
`excluded_kinds_are_storage_level_unsearchable` inserts kind:1059 +
kind:30300 + kind:30622 + kind:9 with the same unique token; asserts
only the kind:9 control surfaces. Each excluded kind has its own
load-bearing negative assertion with a diagnostic message naming the
regression, plus a tight exactly-one-hit bound.
Mutate-bite verified: dropping the CASE's NULL branch (revert to
`to_tsvector('simple', content)`) makes excluded kinds searchable
and the test fails RED with the designed message
"kind:1059 MUST NOT be searchable — privacy regression in search_tsv
generated column". Restored → 10/10 green.
3. crates/buzz-test-client/tests/e2e_nostr_interop.rs — rewrote
`test_nip17_gift_wrap_not_searchable` to use the relay's actual
NIP-50 search seam (`Filter::new().search(token)`,
`collect_until_eose`) instead of the now-removed Typesense
`/multi_search`. Pattern cribbed from
`test_nip50_search_returns_results_and_eose`. Pointer comment to
the underlying mutate-bite in fts_integration.rs.
Verification on `quinn/search-kind-exclusions` off `34ffb8ab3`:
- `cargo test -p buzz-search --tests -- --include-ignored`: 10/10
- `cargo test -p buzz-db -- --include-ignored`: 99/99 (incl. migration
lints + consolidated-schema fresh-DB)
- `cargo check -p buzz-test-client --tests`: clean
- `cargo clippy -p buzz-search -p buzz-db --all-targets -- -D warnings`:
clean
- `cargo fmt --all -- --check`: clean
Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
This commit is contained in:
parent
34ffb8ab37
commit
3fd16cc1f7
@@ -203,7 +203,23 @@ CREATE TABLE events (
|
||||
-- community-leading btree filters BitmapAnd-ed with the GIN probe, so the
|
||||
-- GIN index itself stays the minimal `GIN (search_tsv)` (Max's caveat:
|
||||
-- avoid btree_gin unless EXPLAIN proves it buys something).
|
||||
search_tsv TSVECTOR GENERATED ALWAYS AS (to_tsvector('simple', content)) STORED,
|
||||
--
|
||||
-- Privacy kind exclusions (parity with the pre-rewrite Typesense feed —
|
||||
-- old relay's `handlers/event.rs:287-290` skip set):
|
||||
-- 1059 = KIND_GIFT_WRAP (NIP-17 ciphertext)
|
||||
-- 30300 = KIND_EVENT_REMINDER (AUTHOR_ONLY_KINDS — defense in depth)
|
||||
-- 30622 = KIND_DM_VISIBILITY (per-viewer private hide state)
|
||||
-- NULL tsvector never matches `@@`, so excluded rows are storage-level
|
||||
-- unsearchable. Constants kept in `buzz_core::kind` (KIND_GIFT_WRAP,
|
||||
-- KIND_EVENT_REMINDER, KIND_DM_VISIBILITY); inlined here because a sqlx
|
||||
-- migration is frozen SQL and cannot import the Rust constant. If a new
|
||||
-- privacy-sensitive kind is added there, update this list and add a
|
||||
-- regression test in `buzz-search/tests/fts_integration.rs`.
|
||||
search_tsv TSVECTOR GENERATED ALWAYS AS (
|
||||
CASE WHEN kind IN (1059, 30300, 30622) THEN NULL::tsvector
|
||||
ELSE to_tsvector('simple', content)
|
||||
END
|
||||
) STORED,
|
||||
sig BYTEA NOT NULL,
|
||||
received_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
channel_id UUID,
|
||||
|
||||
Reference in New Issue
Block a user