mirror of
https://github.com/block/buzz.git
synced 2026-11-03 07:51:22 +00:00
bbf125dab2
Replace the GENERATED ALWAYS ... STORED tsvector column + column-backed
GIN index with a single expression index on to_tsvector('simple', content).
Why: the expression index is maintained by Postgres on every INSERT/UPDATE
exactly like a column index (no write-path work), but avoids the stored
column's ALTER TABLE row rewrite / ACCESS EXCLUSIVE backfill — so the
migration build is online-safe on a fresh/small relay and the same-named
index (idx_events_content_fts) is pre-buildable out of band on a large
populated relay (CREATE INDEX CONCURRENTLY + ATTACH per the live-relay
runbook). IF NOT EXISTS makes the migration idempotent against that path.
The query path renders the identical to_tsvector('simple', content)
expression so the planner uses the index. Rank SQL (ts_rank_cd) is
unchanged in behavior.
- migrations/0004_search_fts.sql: single CREATE INDEX IF NOT EXISTS expr index
- schema/schema.sql: drop generated column; expr index for fresh installs
- crates/buzz-search/src/postgres.rs: query refs -> to_tsvector(...) expr
- crates/buzz-db/src/migration.rs: assertions match new shape
- crates/buzz-search/src/lib.rs, crates/buzz-relay/src/main.rs: doc wording
Tests: cargo test -p buzz-db -p buzz-search (incl. ignored PG tests):
118 passed, 0 failed.
Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
41 lines
2.3 KiB
SQL
41 lines
2.3 KiB
SQL
-- Add a full-text-search GIN index to the events table so the relay can serve
|
|
-- NIP-50 search directly from Postgres, eliminating the Typesense dependency.
|
|
--
|
|
-- Shape: an EXPRESSION index on `to_tsvector('simple', content)` rather than a
|
|
-- `GENERATED ... STORED` column. Postgres maintains an expression index on every
|
|
-- INSERT/UPDATE exactly like a column index, so the write path needs no
|
|
-- application-level index maintenance (this replaces the Typesense worker
|
|
-- pipeline) — and there is no stored column, so no `ALTER TABLE ... STORED` row
|
|
-- rewrite and no `ACCESS EXCLUSIVE` backfill. The index build is therefore
|
|
-- online-safe on a fresh/small relay (this file) and pre-buildable out of band
|
|
-- on a large populated relay (operator runbook, below).
|
|
--
|
|
-- Tokenizer choice: `simple` does no stemming and preserves identifiers like
|
|
-- agent handles, nip05 strings, and slugs. The `english` config would stem
|
|
-- ("running" -> "run") and mangle handles. Chat content is heterogeneous;
|
|
-- `simple` is the safer default for v1. The query path in
|
|
-- `buzz-search/src/postgres.rs` renders the identical `to_tsvector('simple', content)`
|
|
-- expression so the planner matches this index.
|
|
--
|
|
-- kind:0 metadata: the old Typesense pipeline appended parsed
|
|
-- display_name/name/nip05 to event content before indexing. With FTS over raw
|
|
-- `content`, those strings still tokenize because they live in the kind:0 JSON
|
|
-- body — `to_tsvector('simple', '{"name":"alice"}')` matches `q=alice`.
|
|
-- Validated by the NIP-50 e2e suite.
|
|
--
|
|
-- `events` is partitioned by RANGE (created_at). `CREATE INDEX ... ON events`
|
|
-- builds a partitioned GIN index whose per-partition child indexes propagate to
|
|
-- existing and future partitions. Partition pruning on since/until narrows the
|
|
-- GIN scan further than Typesense's full-collection scan does today.
|
|
--
|
|
-- IF NOT EXISTS makes this migration idempotent against the operator runbook:
|
|
-- on a large relay the index is built per-child with CREATE INDEX CONCURRENTLY
|
|
-- and ATTACHed to a parent named `idx_events_content_fts` BEFORE this code
|
|
-- deploys, so this statement is a no-op. See
|
|
-- GUIDES/BUZZ_POSTGRES_FTS_LIVE_RELAY_RUNBOOK.md.
|
|
--
|
|
-- Managed by sqlx migrations.
|
|
|
|
CREATE INDEX IF NOT EXISTS idx_events_content_fts
|
|
ON events USING GIN (to_tsvector('simple', content));
|