## Problem Migrations `0076`, `0078` and `0081` raise an exception on any table with at least one row unless its index was prebuilt with `CREATE INDEX CONCURRENTLY`. A relay that runs migrations at startup (`BUZZ_AUTO_MIGRATE`, on by default in the Helm chart) and has data in those tables therefore crash-loops on upgrade: ``` migration error: while executing migration 81: error returned from database: prebuild idx_nip_fi_profile_heads_names concurrently; see docs/profile-search.md ``` `0081` (#9019) hits every auto-migrating relay that has published NIP-FI verified profiles. `0076`/`0078` (#8523) hit any relay with artifact data or channel members. Each of these tables is small enough on such relays that a plain index build takes milliseconds. The root cause is `docs/database.md` Rule 6. It told authors that an index on a populated table MUST be built concurrently by an operator ahead of time, so migrations were written to refuse populated tables. With auto-migration that refusal is a crash loop, not a safe stop. ## Fix The migrations are already applied on some databases, so they can't be edited: sqlx would reject the changed checksums. Instead, `run_migrations_locked` builds their missing indexes first. For each entry in `INLINE_INDEX_BUILDS` (`0076`, `0078`, `0081`), it runs migrations through the version before it, then builds any missing index. Each build is a plain `CREATE INDEX` in its own transaction, under the same 1s `lock_timeout` and 5s `statement_timeout` the migrations use. That is the bounded inline build `0049` already does: - **Small populated databases** upgrade the same way empty ones do. - **Large databases** still fail closed. Only a lock or statement timeout (`55P03` / `57014`) gets the "prebuild concurrently" hint, pointing at that migration's own doc. Other errors pass through unchanged. The blocked migration is not recorded. - **Prebuilt indexes** are found with `to_regclass` and skipped before `CREATE INDEX`, so deployments that prebuilt concurrently never request the writer-conflicting lock. - **Invalid prebuilt indexes** are left for the migration to reject, as before. `CONCURRENTLY` is not used in this path. The runner holds the migration advisory lock, and another replica blocked on that lock holds a snapshot that a concurrent build would wait on. ### Rules - `docs/database.md` Rule 6: with `BUZZ_AUTO_MIGRATE` on, every migration must apply its own change on a populated database and must never refuse one because it has rows. A new index is built inline under the migration's budgets. Only deployments that apply migrations out of band (auto-migration off) prebuild concurrently. - Root `AGENTS.md` (step 4) and `migrations/AGENTS.md` repeat the never-refuse rule. - `docs/artifact-sync-deployment.md` and `docs/profile-search.md` describe the startup behavior first and the concurrent prebuild as the out-of-band path. ## Tests - `populated_database_auto_migrates_through_every_migration` (Postgres) migrates to `0075`, seeds artifact rows, and checks four phases: 1. **Lock held:** while a writer lock is held, the `0076` build fails closed. 2. **Index prebuilt:** with the `0076` index prebuilt, the build is skipped. 3. **Partial `0078`:** a blocked `0078` build keeps the builds before it committed. 4. **Verified profile seeded:** with a verified profile head inserted before `0081`, a final startup succeeds. It asserts that the applied versions equal every embedded migration and that all six indexes are valid, ready and live. - `frozen_prebuild_guards_have_inline_builds` (no database) scans `migrations/`. It fails if any migration refuses a populated table (`RAISE EXCEPTION 'prebuild …`) without an inline build, and if a new guard is added. - `migration_execution_cannot_bypass_schema_destruction_lock` allows exactly one production `run_to`, inside `run_migrations_locked`. ## What got simpler - Auto-migrating relays no longer need a manual, operator-only prebuild step for small deployments. That restores the chart README's contract that `helm upgrade` is the whole upgrade procedure. - Rule 6 no longer steers authors toward refusing populated tables, and a test enforces that. - The change adds one table-driven step for three frozen migrations and no new configuration. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Luke Tornquist <tornquist@squareup.com> Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
4.3 KiB
Profile name search
POST /profiles/search finds community members by profile name. It matches
only the name and display_name string fields of each current global kind:0
profile and, where the community advertises NIP-FI verified profiles
(federated_identity.public_profile.searchable in NIP-11), the
display_name, username and full_name tags of each current kind:30382
verified profile signed by the relay key. It does not search picture
(including data URLs), about, nip05, banner, or any other profile field. NIP-50 search over kind:0 matches the
whole profile JSON, so it can return profiles whose avatar data or bio happens
to contain the query text.
Apart from the fields it reads, the endpoint matches and pages like a NIP-50
kind:0 search with search_mode: "prefix". A client can switch from one to the
other without a visible change in which people a query finds.
The endpoint requires NIP-98 authentication over the request body and relay
membership, like other relay-owned member endpoints. A relay without the
endpoint answers 404; clients fall back to NIP-50 search for that request.
Request:
{ "query": "wes", "limit": 20, "page": 1 }
query: 1 through 100 characters after trimming. Required.limit: results per page, 1 through 100. Defaults to 20.page: 1-based page oflimitresults, 1 through 1000. Defaults to 1.- Unknown fields are rejected with
400.
Response: a JSON array of signed events for one page of people, best match
first. Each person contributes their current kind:0, when they have one, then
their current relay-signed kind:30382, when verified profiles are advertised
and they have one, whichever of the two matched. A page with fewer than
limit people is the last page. Clients that read only kind:0 keep working;
they also find people who match only by verified name.
Matching is the NIP-50 prefix rule, case-insensitive. The query is split on
whitespace. Every word except the last must equal a word of name or
display_name; the last word may be a word prefix. Words may appear in any
order and may be split across the two fields, so "johnson wes" finds
"Wes Johnson". A verified profile matches the same way over its three name
tags, except that its last word must be at least three characters, as in
NIP-50. The two sources match separately: a query whose words are split
between a person's kind:0 and their verified profile does not match.
Accents are not folded.
People are ordered by their best match across both sources:
- a name equal to the query;
- a name that starts with the query;
- a name with a later word that starts with the query;
- any other match.
For these tiers the query and the names are lower-cased, with whitespace
runs collapsed to one space and trimmed, by the same database expression.
Ties go by relevance of the query to the names alone (not the whole
profile, as in NIP-50), then newest first, then pubkey, so pages do not
overlap. Profile content that is not a JSON object, or a name field that
is not a string, never matches. Content that PostgreSQL cannot read as jsonb
(for example a \u0000 escape or very deep nesting) has no names, so it
never matches and never makes the search fail
(schema/functions/profile_name_fields.sql).
Deployment
Verified-name matching reads idx_nip_fi_profile_heads_names. Relay startup
migration and buzz-admin migrate build it before migration 0081 under a
one-second lock and five-second statement budget. Applied directly as SQL,
0081 builds it only while nip_fi_profile_heads is empty. A deployment that
applies migrations out of band (auto-migration off) builds it first, outside a
transaction:
CREATE INDEX CONCURRENTLY idx_nip_fi_profile_heads_names
ON public.nip_fi_profile_heads USING GIN (
to_tsvector('simple',
COALESCE(jsonb_path_query_first(event, 'strict $.tags[*] ? (@[0] == "display_name")[1]', '{}', true) #>> '{}', '')
|| ' ' || COALESCE(jsonb_path_query_first(event, 'strict $.tags[*] ? (@[0] == "username")[1]', '{}', true) #>> '{}', '')
|| ' ' || COALESCE(jsonb_path_query_first(event, 'strict $.tags[*] ? (@[0] == "full_name")[1]', '{}', true) #>> '{}', ''))
);
If the build fails, drop the invalid index and build it again. The migration then checks that the index is valid and has this definition.