Files
buzz/migrations/0033_private_managed_agent_fts.sql
Tyler b3baa56ba8 fix(db): exclude kind:30179 ciphertext from brownfield FTS (#6822)
## Problem

Kind `30179` (NIP-PMA private managed agent — NIP-44 ciphertext carrying
the agent nsec, env vars, prompt) joined `AUTHOR_ONLY_KINDS` in #4593,
but only **fresh** installs stopped indexing it:

- `migrations/0008` installs the positive FTS allowlist only when
`events` is empty;
- `migrations/0014` wraps the retained brownfield expression for `30350`
alone;
- `schema/schema.sql` still carries the legacy negative skip-set without
`30179`.

So a relay upgraded in place keeps tokenizing `30179` ciphertext into
`events.search_tsv`. Found by Wren while reviewing #4999 (which
activates publication of 30179 from Desktop). No readable leak: `/query`
applies `search_hit_accepted` + `event_visible_to_reader` before
serialization and live foreign searches against builderlab returned `[]`
— this is the storage-layer privacy invariant (`docs/nips/NIP-PMA.md`
deployment step 2) plus wasted FTS work.

## Fix

- `migrations/0033_private_managed_agent_fts.sql` — same shape as 0014:
capture the current generated expression via `pg_get_expr`, drop/re-add
`search_tsv` wrapped with `kind = 30179 → NULL`. Every other kind keeps
whatever policy the database already had (fresh allowlist or brownfield
skip-set). Rebuilds the GIN index; no heap-wide policy rewrite.
- `schema/schema.sql` — add `30179` to the desired-state skip-set.

## Tests

- `populated_upgrade_preserves_search_policy_except_for_private_kinds`
(renamed from `…_push_leases`): provisions the **legacy negative
expression** (migrations 1–7), inserts kind 1 / 30179 / 30350 rows,
checkpoints after 0014 (30179 still searchable, 30350 not), then runs to
head and asserts 30179 is NULL. **Proven red** with the 0033 file
removed: `left: [(1, Some(true)), (30179, Some(true)), (30350, None)]`.
- `embedded_migrator_contains_consolidated_initial_schema`: count 32→33,
asserts 0033 shape and `schema.sql` parity.
- `buzz-search/tests/fts_integration.rs` setup applies 0033 so the FTS
tripwires run against the real chain.

## Verification (HEAD 6154cc33d, same shell)

- `cargo test -p buzz-db -- --include-ignored --test-threads=1`: 305
passed / 3 failed — the 3
(`create_community_with_owner_enforces_per_owner_limit`,
`insert_mentions_indexes_rosters_past_bind_parameter_cap`,
`transfer_ownership_returns_limit_reached_for_maxed_transferee`) fail
identically on pristine `main` f24971033 with a fresh DB; unrelated to
this diff.
- `cargo test -p buzz-search -- --include-ignored`: 19 passed (incl.
`author_only_kinds_are_storage_level_unsearchable`,
`p_gated_persistent_kinds_have_storage_null_tsvector`).
- `desired_state_schema_bootstrap_progresses_beyond_fencing` +
`run_migrations_applies_consolidated_initial_schema_on_fresh_database`:
pass.
- `cargo fmt --all --check` clean; `cargo clippy -p buzz-db -p
buzz-search --all-targets -D warnings` clean.

Independent of #4999 — both main and #4999 are brownfield-exposed today;
this lands either order.

---------

Signed-off-by: Meli <5aaa86bce934fc3445fc254aab560a40923f10252f92107e665073dede0e04d3@buzz.block.builderlab.xyz>
Signed-off-by: Eva <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Co-authored-by: Meli <5aaa86bce934fc3445fc254aab560a40923f10252f92107e665073dede0e04d3@buzz.block.builderlab.xyz>
Co-authored-by: Eva <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
2026-08-27 10:47:35 -04:00

47 lines
2.1 KiB
SQL

-- NIP-PMA kind:30179 carries the owner's private managed-agent payload
-- (agent nsec, env vars, prompt) as NIP-44 ciphertext and is author-only.
-- Exclude it from full-text search without changing the search policy of
-- existing installations. Fresh installs are already safe via the positive
-- allowlist from migration 0008; this closes the brownfield gap where a
-- populated database still runs the legacy negative skip-set (0001/0005)
-- and would tokenize the ciphertext into search_tsv.
--
-- Same shape as 0014 (kind:30350): PostgreSQL cannot alter a generated
-- expression in place, so capture the current expression, drop the column,
-- and re-add it wrapped with the new exclusion. Every other kind keeps
-- whatever policy the database had before.
--
-- Operational cost: this is not free on large databases. DROP COLUMN +
-- ADD ... GENERATED ... STORED rewrites the entire events heap and then
-- rebuilds the GIN index, all under an ACCESS EXCLUSIVE lock inside the
-- migration transaction (CREATE INDEX CONCURRENTLY is not possible
-- here), with no lock_timeout. Expect relay downtime proportional to
-- the size of events. The index is recreated from the stock definition
-- below; any non-stock indexes or storage parameters on search_tsv are
-- not captured or replayed. 0014 set this precedent on smaller tables;
-- operators with large brownfield databases should schedule a window.
DO $$
DECLARE
existing_expression TEXT;
BEGIN
SELECT pg_get_expr(d.adbin, d.adrelid)
INTO existing_expression
FROM pg_attrdef d
JOIN pg_attribute a
ON a.attrelid = d.adrelid
AND a.attnum = d.adnum
WHERE d.adrelid = 'events'::regclass
AND a.attname = 'search_tsv';
IF existing_expression IS NULL THEN
RAISE EXCEPTION 'events.search_tsv generated expression not found';
END IF;
ALTER TABLE events DROP COLUMN search_tsv;
EXECUTE format(
'ALTER TABLE events ADD COLUMN search_tsv TSVECTOR GENERATED ALWAYS AS (CASE WHEN kind = 30179 THEN NULL::tsvector ELSE (%s) END) STORED',
existing_expression
);
CREATE INDEX idx_events_search_tsv ON events USING GIN (search_tsv);
END $$;