mirror of
https://github.com/block/buzz.git
synced 2026-11-05 17:01:35 +00:00
4d8fe9a5e0
Adds a Postgres full-text search backend as an alternative to Typesense for
NIP-50 search, gated behind BUZZ_SEARCH_BACKEND=typesense|postgres|disabled
(default typesense — no behavior change for existing deployments).
The replacement is structural: NIP-50 search is the only Typesense call site,
and the read path already refetches canonical events from Postgres by id, so
Typesense was just a lookup index in front of the DB that owns the data. A
generated stored tsvector column + GIN index gives the same shape with zero
write-path code change.
Changes
- migrations/0004_search_fts.sql: events.content_tsv GENERATED ALWAYS AS
to_tsvector('simple', content) STORED, GIN index, cascades to partitions.
- crates/buzz-search: SearchBackend enum (Typesense | Postgres | Disabled),
SearchService::with_postgres / ::disabled, postgres.rs backend impl,
backend-neutral SearchQuery (structured kinds/authors/channel_ids/since/until;
each backend renders its own filter).
- crates/buzz-relay/src/config.rs: BUZZ_SEARCH_BACKEND env wired with strict
parsing (unknown value → ConfigError::InvalidValue, no silent fallback) +
3 unit tests.
- crates/buzz-relay/src/main.rs: dispatch on backend; Postgres → with_postgres
using db.pool(); Disabled → no-op; Typesense → existing path. ensure_collection
only runs for the Typesense backend.
- crates/buzz-relay/src/{handlers/req.rs, api/bridge.rs}: swap to the new
SearchService surface. Caller code shrinks — filter parts were already
structured.
- crates/buzz-db/src/lib.rs: Db::pool() accessor for the PG backend.
Validation (against parent 2e426b2f, PG17 side-deployed):
- buzz-search lib: 29/29 pass.
- buzz-relay config tests: 11/11 (incl. 3 new).
- NIP-50 e2e on Typesense backend: 5/5 pass (regression baseline).
- NIP-50 e2e on Postgres backend: 5/5 pass — including
test_nip50_search_relevance_order, confirming ts_rank_cd ranks correctly
for the NIP-50 query shape and the 'simple' tokenizer config is acceptable.
- Wider e2e_nostr_interop sweep on Postgres: 19/23. The 4 failures reproduce
identically on Typesense backend on this branch — pre-existing test-fixture
coupling to a hard-coded 'events' collection name, not a regression.
This is additive: Typesense remains default; nothing in the existing path is
removed. Operators flip BUZZ_SEARCH_BACKEND per release to A/B/rollback.
Signed-off-by: Tyler <109685178+tlongwell-block@users.noreply.github.com>
Co-authored-by: Sami <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@sprout-oss.stage.blox.sqprod.co>
36 lines
1.8 KiB
SQL
36 lines
1.8 KiB
SQL
-- Add a generated full-text-search column + GIN index to the events table so
|
|
-- the relay can serve NIP-50 search directly from Postgres, eliminating the
|
|
-- Typesense dependency.
|
|
--
|
|
-- `content_tsv` is `GENERATED ALWAYS AS (to_tsvector('simple', content)) STORED`
|
|
-- so every INSERT/UPDATE populates it automatically — no application-level
|
|
-- index maintenance needed (matches the pattern Typesense filled today via the
|
|
-- worker pipeline in `buzz-relay/src/state.rs`).
|
|
--
|
|
-- Tokenizer choice: `simple` does no stemming and preserves identifiers like
|
|
-- agent handles, nip05 strings, and slugs. The `english` config would stem
|
|
-- ("running" → "run") but mangle handles ("alice42" tokenizes fine, but
|
|
-- something like "agents" → "agent" would break exact-handle search). Chat
|
|
-- content is heterogeneous; `simple` is the safer default for v1.
|
|
--
|
|
-- kind:0 metadata flattening: the existing Typesense pipeline appends parsed
|
|
-- display_name/name/nip05 to event content before indexing
|
|
-- (`buzz-search/src/index.rs::flatten_kind0_for_indexing`). With FTS on raw
|
|
-- `content`, those strings still tokenize because they live in the kind:0 JSON
|
|
-- body — `to_tsvector('simple', '{"name":"alice"}')` matches `q=alice` after
|
|
-- json-aware tokenization. Validated by the NIP-50 e2e suite.
|
|
--
|
|
-- `events` is partitioned by RANGE (created_at); ADD COLUMN on the parent
|
|
-- cascades the generated column to every partition, and CREATE INDEX on the
|
|
-- parent builds a partitioned GIN index that propagates to each partition.
|
|
-- Partition pruning on since/until queries narrows the GIN scan further than
|
|
-- Typesense's full-collection scan does today.
|
|
--
|
|
-- Managed by sqlx migrations.
|
|
|
|
ALTER TABLE events
|
|
ADD COLUMN content_tsv tsvector
|
|
GENERATED ALWAYS AS (to_tsvector('simple', content)) STORED;
|
|
|
|
CREATE INDEX idx_events_content_tsv ON events USING GIN (content_tsv);
|