Files
buzz/docs/profile-search.md
Luke Tornquist 279a31daac buzz-search: read profile names without the profile_name_fields function (#9397)
## What

This is phase 1 of 2 for retiring the plpgsql function
`profile_name_fields(text)`. Its only caller, profile name search
(`search_profile_names`), now gets `name` and `display_name` from an
inline SQL expression (`PROFILE_NAME_FIELDS` in
`crates/buzz-search/src/query.rs`) instead.

The function's exception handler made unreadable kind:0 content have no
names rather than fail the search. The inline version keeps that
guarantee in two steps:

- **`pg_input_is_valid(content, 'jsonb')`** rejects the soft parse
errors: a `\^@` escape, a lone surrogate, a huge number, and malformed
JSON.
- **A bracket bound for deep nesting.** PostgreSQL's JSON parser is
recursive. Past about 10k levels at the default `max_stack_depth`, it
raises `stack depth limit exceeded`, and `pg_input_is_valid` does not
catch that. I confirmed this on PG 17: depth 10k is fine and 20k raises.
So content with more than 4096 `[` and `{` is never parsed and has no
names. The count bounds the depth. Ingest already rejects kind:0 content
nested deeper than 128 levels (serde_json), so this only matters for
content stored another way.

`OFFSET 0` keeps the parse from being copied into each reference to
`doc`, so each candidate row is parsed once, plus one validity pass. The
plpgsql function also parsed each row once.

**Behavior change (edge only):** a kind:0 with more than 4096 brackets,
even inside strings, is no longer findable by name. Real profiles don't
come close, and a `picture` data URL is base64 with no brackets.

`pg_input_is_valid` needs PostgreSQL 16 or later. CI runs 16, and the
dev compose runs 17.

**Phase 2 (not here):** a migration and desired-state change drops the
function, its `schema/functions/` file and its `schema.sql` entry once
this release has shipped.

## Tests

- `profile_names_survive_unparsable_content` already stores a `\^@`
profile, a lone-surrogate profile, a `1e1000000` profile and a 60k-deep
profile next to a plain one, and requires the search to return only the
plain one. It now also covers the bound itself: a profile with exactly
4096 brackets, all but one inside a string, is still found.
- Must-fail mutants, each run on Blox: bracket bound removed (the search
fails with a stack overflow), validity check removed, bound tightened by
one, and the string-type check loosened.

## What got simpler

Profile search no longer depends on a database function. Name extraction
lives next to the rest of the search SQL it feeds.

## Verification

Blox, at `be0db303ea90033cc2c37ac8f8f07c27f4b08e31` (clean tree):

| Gate | Result |
|---|---|
| `cargo fmt --all --check` | pass |
| `cargo clippy -p buzz-search --all-targets -- -D warnings` | pass |
| `cargo nextest run -p buzz-search --lib` | 4/4 |
| `scripts/postgres-test-run.sh -p buzz-search --tests -E
'test(/profile_names/)'` | 4/4 |
| Must-fail mutants (no bound, no validity check, bound tightened by
one, non-string names allowed) | 4/4 killed |

Probe on PostgreSQL 17.11: `pg_input_is_valid` returns false for `\^@`,
a lone surrogate, `1e1000000` and truncated JSON; nesting depth 10k is
valid; depth 20k raises `stack depth limit exceeded`. CI runs the full
Postgres lanes.

🤖 Generated with [Claude Code](https://claude.com/claude-code)


buzz-review-completed

---------

Signed-off-by: Luke Tornquist <tornquist@squareup.com>
Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 19:34:56 -05:00

4.3 KiB

Profile name search

POST /profiles/search finds community members by profile name. It matches only the name and display_name string fields of each current global kind:0 profile and, where the community advertises NIP-FI verified profiles (federated_identity.public_profile.searchable in NIP-11), the display_name, username and full_name tags of each current kind:30382 verified profile signed by the relay key. It does not search picture (including data URLs), about, nip05, banner, or any other profile field. NIP-50 search over kind:0 matches the whole profile JSON, so it can return profiles whose avatar data or bio happens to contain the query text.

Apart from the fields it reads, the endpoint matches and pages like a NIP-50 kind:0 search with search_mode: "prefix". A client can switch from one to the other without a visible change in which people a query finds.

The endpoint requires NIP-98 authentication over the request body and relay membership, like other relay-owned member endpoints. A relay without the endpoint answers 404; clients fall back to NIP-50 search for that request.

Request:

{ "query": "wes", "limit": 20, "page": 1 }
  • query: 1 through 100 characters after trimming. Required.
  • limit: results per page, 1 through 100. Defaults to 20.
  • page: 1-based page of limit results, 1 through 1000. Defaults to 1.
  • Unknown fields are rejected with 400.

Response: a JSON array of signed events for one page of people, best match first. Each person contributes their current kind:0, when they have one, then their current relay-signed kind:30382, when verified profiles are advertised and they have one, whichever of the two matched. A page with fewer than limit people is the last page. Clients that read only kind:0 keep working; they also find people who match only by verified name.

Matching is the NIP-50 prefix rule, case-insensitive. The query is split on whitespace. Every word except the last must equal a word of name or display_name; the last word may be a word prefix. Words may appear in any order and may be split across the two fields, so "johnson wes" finds "Wes Johnson". A verified profile matches the same way over its three name tags, except that its last word must be at least three characters, as in NIP-50. The two sources match separately: a query whose words are split between a person's kind:0 and their verified profile does not match. Accents are not folded.

People are ordered by their best match across both sources:

  1. a name equal to the query;
  2. a name that starts with the query;
  3. a name with a later word that starts with the query;
  4. any other match.

For these tiers the query and the names are lower-cased, with whitespace runs collapsed to one space and trimmed, by the same database expression. Ties go by relevance of the query to the names alone (not the whole profile, as in NIP-50), then newest first, then pubkey, so pages do not overlap. Profile content that is not a JSON object, or a name field that is not a string, never matches. Content that PostgreSQL cannot read as jsonb (for example a \u0000 escape), and content with more than 4096 [ and {, which bounds how deeply it can nest, has no names, so it never matches and never makes the search fail (PROFILE_NAME_FIELDS in crates/buzz-search/src/query.rs).

Deployment

Verified-name matching reads idx_nip_fi_profile_heads_names. Relay startup migration and buzz-admin migrate build it before migration 0081 under a one-second lock and five-second statement budget. Applied directly as SQL, 0081 builds it only while nip_fi_profile_heads is empty. A deployment that applies migrations out of band (auto-migration off) builds it first, outside a transaction:

CREATE INDEX CONCURRENTLY idx_nip_fi_profile_heads_names
    ON public.nip_fi_profile_heads USING GIN (
    to_tsvector('simple',
        COALESCE(jsonb_path_query_first(event, 'strict $.tags[*] ? (@[0] == "display_name")[1]', '{}', true) #>> '{}', '')
        || ' ' || COALESCE(jsonb_path_query_first(event, 'strict $.tags[*] ? (@[0] == "username")[1]', '{}', true) #>> '{}', '')
        || ' ' || COALESCE(jsonb_path_query_first(event, 'strict $.tags[*] ? (@[0] == "full_name")[1]', '{}', true) #>> '{}', ''))
);

If the build fails, drop the invalid index and build it again. The migration then checks that the index is valid and has this definition.