Files
buzz/docs/profile-search.md
Luke Tornquist ff15c347c2 fix(db): auto-migrate prebuild-guarded indexes on small databases (#8802)
## Problem

Migrations `0076`, `0078` and `0081` raise an exception on any table
with at least one row unless its index was prebuilt with `CREATE INDEX
CONCURRENTLY`. A relay that runs migrations at startup
(`BUZZ_AUTO_MIGRATE`, on by default in the Helm chart) and has data in
those tables therefore crash-loops on upgrade:

```
migration error: while executing migration 81: error returned from database:
prebuild idx_nip_fi_profile_heads_names concurrently; see docs/profile-search.md
```

`0081` (#9019) hits every auto-migrating relay that has published NIP-FI
verified profiles. `0076`/`0078` (#8523) hit any relay with artifact
data or channel members. Each of these tables is small enough on such
relays that a plain index build takes milliseconds.

The root cause is `docs/database.md` Rule 6. It told authors that an
index on a populated table MUST be built concurrently by an operator
ahead of time, so migrations were written to refuse populated tables.
With auto-migration that refusal is a crash loop, not a safe stop.

## Fix

The migrations are already applied on some databases, so they can't be
edited: sqlx would reject the changed checksums. Instead,
`run_migrations_locked` builds their missing indexes first. For each
entry in `INLINE_INDEX_BUILDS` (`0076`, `0078`, `0081`), it runs
migrations through the version before it, then builds any missing index.

Each build is a plain `CREATE INDEX` in its own transaction, under the
same 1s `lock_timeout` and 5s `statement_timeout` the migrations use.
That is the bounded inline build `0049` already does:

- **Small populated databases** upgrade the same way empty ones do.
- **Large databases** still fail closed. Only a lock or statement
timeout (`55P03` / `57014`) gets the "prebuild concurrently" hint,
pointing at that migration's own doc. Other errors pass through
unchanged. The blocked migration is not recorded.
- **Prebuilt indexes** are found with `to_regclass` and skipped before
`CREATE INDEX`, so deployments that prebuilt concurrently never request
the writer-conflicting lock.
- **Invalid prebuilt indexes** are left for the migration to reject, as
before.

`CONCURRENTLY` is not used in this path. The runner holds the migration
advisory lock, and another replica blocked on that lock holds a snapshot
that a concurrent build would wait on.

### Rules

- `docs/database.md` Rule 6: with `BUZZ_AUTO_MIGRATE` on, every
migration must apply its own change on a populated database and must
never refuse one because it has rows. A new index is built inline under
the migration's budgets. Only deployments that apply migrations out of
band (auto-migration off) prebuild concurrently.
- Root `AGENTS.md` (step 4) and `migrations/AGENTS.md` repeat the
never-refuse rule.
- `docs/artifact-sync-deployment.md` and `docs/profile-search.md`
describe the startup behavior first and the concurrent prebuild as the
out-of-band path.

## Tests

- `populated_database_auto_migrates_through_every_migration` (Postgres)
migrates to `0075`, seeds artifact rows, and checks four phases:
1. **Lock held:** while a writer lock is held, the `0076` build fails
closed.
2. **Index prebuilt:** with the `0076` index prebuilt, the build is
skipped.
3. **Partial `0078`:** a blocked `0078` build keeps the builds before it
committed.
4. **Verified profile seeded:** with a verified profile head inserted
before `0081`, a final startup succeeds. It asserts that the applied
versions equal every embedded migration and that all six indexes are
valid, ready and live.
- `frozen_prebuild_guards_have_inline_builds` (no database) scans
`migrations/`. It fails if any migration refuses a populated table
(`RAISE EXCEPTION 'prebuild …`) without an inline build, and if a new
guard is added.
- `migration_execution_cannot_bypass_schema_destruction_lock` allows
exactly one production `run_to`, inside `run_migrations_locked`.

## What got simpler

- Auto-migrating relays no longer need a manual, operator-only prebuild
step for small deployments. That restores the chart README's contract
that `helm upgrade` is the whole upgrade procedure.
- Rule 6 no longer steers authors toward refusing populated tables, and
a test enforces that.
- The change adds one table-driven step for three frozen migrations and
no new configuration.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Luke Tornquist <tornquist@squareup.com>
Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 13:52:06 -05:00

4.3 KiB

Profile name search

POST /profiles/search finds community members by profile name. It matches only the name and display_name string fields of each current global kind:0 profile and, where the community advertises NIP-FI verified profiles (federated_identity.public_profile.searchable in NIP-11), the display_name, username and full_name tags of each current kind:30382 verified profile signed by the relay key. It does not search picture (including data URLs), about, nip05, banner, or any other profile field. NIP-50 search over kind:0 matches the whole profile JSON, so it can return profiles whose avatar data or bio happens to contain the query text.

Apart from the fields it reads, the endpoint matches and pages like a NIP-50 kind:0 search with search_mode: "prefix". A client can switch from one to the other without a visible change in which people a query finds.

The endpoint requires NIP-98 authentication over the request body and relay membership, like other relay-owned member endpoints. A relay without the endpoint answers 404; clients fall back to NIP-50 search for that request.

Request:

{ "query": "wes", "limit": 20, "page": 1 }
  • query: 1 through 100 characters after trimming. Required.
  • limit: results per page, 1 through 100. Defaults to 20.
  • page: 1-based page of limit results, 1 through 1000. Defaults to 1.
  • Unknown fields are rejected with 400.

Response: a JSON array of signed events for one page of people, best match first. Each person contributes their current kind:0, when they have one, then their current relay-signed kind:30382, when verified profiles are advertised and they have one, whichever of the two matched. A page with fewer than limit people is the last page. Clients that read only kind:0 keep working; they also find people who match only by verified name.

Matching is the NIP-50 prefix rule, case-insensitive. The query is split on whitespace. Every word except the last must equal a word of name or display_name; the last word may be a word prefix. Words may appear in any order and may be split across the two fields, so "johnson wes" finds "Wes Johnson". A verified profile matches the same way over its three name tags, except that its last word must be at least three characters, as in NIP-50. The two sources match separately: a query whose words are split between a person's kind:0 and their verified profile does not match. Accents are not folded.

People are ordered by their best match across both sources:

  1. a name equal to the query;
  2. a name that starts with the query;
  3. a name with a later word that starts with the query;
  4. any other match.

For these tiers the query and the names are lower-cased, with whitespace runs collapsed to one space and trimmed, by the same database expression. Ties go by relevance of the query to the names alone (not the whole profile, as in NIP-50), then newest first, then pubkey, so pages do not overlap. Profile content that is not a JSON object, or a name field that is not a string, never matches. Content that PostgreSQL cannot read as jsonb (for example a \u0000 escape or very deep nesting) has no names, so it never matches and never makes the search fail (schema/functions/profile_name_fields.sql).

Deployment

Verified-name matching reads idx_nip_fi_profile_heads_names. Relay startup migration and buzz-admin migrate build it before migration 0081 under a one-second lock and five-second statement budget. Applied directly as SQL, 0081 builds it only while nip_fi_profile_heads is empty. A deployment that applies migrations out of band (auto-migration off) builds it first, outside a transaction:

CREATE INDEX CONCURRENTLY idx_nip_fi_profile_heads_names
    ON public.nip_fi_profile_heads USING GIN (
    to_tsvector('simple',
        COALESCE(jsonb_path_query_first(event, 'strict $.tags[*] ? (@[0] == "display_name")[1]', '{}', true) #>> '{}', '')
        || ' ' || COALESCE(jsonb_path_query_first(event, 'strict $.tags[*] ? (@[0] == "username")[1]', '{}', true) #>> '{}', '')
        || ' ' || COALESCE(jsonb_path_query_first(event, 'strict $.tags[*] ? (@[0] == "full_name")[1]', '{}', true) #>> '{}', ''))
);

If the build fails, drop the invalid index and build it again. The migration then checks that the index is valid and has this definition.