mirror of
https://github.com/block/buzz.git
synced 2026-11-13 21:01:53 +00:00
279a31daac
## What
This is phase 1 of 2 for retiring the plpgsql function
`profile_name_fields(text)`. Its only caller, profile name search
(`search_profile_names`), now gets `name` and `display_name` from an
inline SQL expression (`PROFILE_NAME_FIELDS` in
`crates/buzz-search/src/query.rs`) instead.
The function's exception handler made unreadable kind:0 content have no
names rather than fail the search. The inline version keeps that
guarantee in two steps:
- **`pg_input_is_valid(content, 'jsonb')`** rejects the soft parse
errors: a `\^@` escape, a lone surrogate, a huge number, and malformed
JSON.
- **A bracket bound for deep nesting.** PostgreSQL's JSON parser is
recursive. Past about 10k levels at the default `max_stack_depth`, it
raises `stack depth limit exceeded`, and `pg_input_is_valid` does not
catch that. I confirmed this on PG 17: depth 10k is fine and 20k raises.
So content with more than 4096 `[` and `{` is never parsed and has no
names. The count bounds the depth. Ingest already rejects kind:0 content
nested deeper than 128 levels (serde_json), so this only matters for
content stored another way.
`OFFSET 0` keeps the parse from being copied into each reference to
`doc`, so each candidate row is parsed once, plus one validity pass. The
plpgsql function also parsed each row once.
**Behavior change (edge only):** a kind:0 with more than 4096 brackets,
even inside strings, is no longer findable by name. Real profiles don't
come close, and a `picture` data URL is base64 with no brackets.
`pg_input_is_valid` needs PostgreSQL 16 or later. CI runs 16, and the
dev compose runs 17.
**Phase 2 (not here):** a migration and desired-state change drops the
function, its `schema/functions/` file and its `schema.sql` entry once
this release has shipped.
## Tests
- `profile_names_survive_unparsable_content` already stores a `\^@`
profile, a lone-surrogate profile, a `1e1000000` profile and a 60k-deep
profile next to a plain one, and requires the search to return only the
plain one. It now also covers the bound itself: a profile with exactly
4096 brackets, all but one inside a string, is still found.
- Must-fail mutants, each run on Blox: bracket bound removed (the search
fails with a stack overflow), validity check removed, bound tightened by
one, and the string-type check loosened.
## What got simpler
Profile search no longer depends on a database function. Name extraction
lives next to the rest of the search SQL it feeds.
## Verification
Blox, at `be0db303ea90033cc2c37ac8f8f07c27f4b08e31` (clean tree):
| Gate | Result |
|---|---|
| `cargo fmt --all --check` | pass |
| `cargo clippy -p buzz-search --all-targets -- -D warnings` | pass |
| `cargo nextest run -p buzz-search --lib` | 4/4 |
| `scripts/postgres-test-run.sh -p buzz-search --tests -E
'test(/profile_names/)'` | 4/4 |
| Must-fail mutants (no bound, no validity check, bound tightened by
one, non-string names allowed) | 4/4 killed |
Probe on PostgreSQL 17.11: `pg_input_is_valid` returns false for `\^@`,
a lone surrogate, `1e1000000` and truncated JSON; nesting depth 10k is
valid; depth 20k raises `stack depth limit exceeded`. CI runs the full
Postgres lanes.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
buzz-review-completed
---------
Signed-off-by: Luke Tornquist <tornquist@squareup.com>
Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
88 lines
4.3 KiB
Markdown
88 lines
4.3 KiB
Markdown
# Profile name search
|
|
|
|
`POST /profiles/search` finds community members by profile name. It matches
|
|
only the `name` and `display_name` string fields of each current global kind:0
|
|
profile and, where the community advertises NIP-FI verified profiles
|
|
(`federated_identity.public_profile.searchable` in NIP-11), the
|
|
`display_name`, `username` and `full_name` tags of each current kind:30382
|
|
verified profile signed by the relay key. It does not search `picture`
|
|
(including data URLs), `about`, `nip05`, `banner`, or any other profile field. NIP-50 search over kind:0 matches the
|
|
whole profile JSON, so it can return profiles whose avatar data or bio happens
|
|
to contain the query text.
|
|
|
|
Apart from the fields it reads, the endpoint matches and pages like a NIP-50
|
|
kind:0 search with `search_mode: "prefix"`. A client can switch from one to the
|
|
other without a visible change in which people a query finds.
|
|
|
|
The endpoint requires NIP-98 authentication over the request body and relay
|
|
membership, like other relay-owned member endpoints. A relay without the
|
|
endpoint answers `404`; clients fall back to NIP-50 search for that request.
|
|
|
|
Request:
|
|
|
|
```json
|
|
{ "query": "wes", "limit": 20, "page": 1 }
|
|
```
|
|
|
|
- `query`: 1 through 100 characters after trimming. Required.
|
|
- `limit`: results per page, 1 through 100. Defaults to 20.
|
|
- `page`: 1-based page of `limit` results, 1 through 1000. Defaults to 1.
|
|
- Unknown fields are rejected with `400`.
|
|
|
|
Response: a JSON array of signed events for one page of people, best match
|
|
first. Each person contributes their current kind:0, when they have one, then
|
|
their current relay-signed kind:30382, when verified profiles are advertised
|
|
and they have one, whichever of the two matched. A page with fewer than
|
|
`limit` people is the last page. Clients that read only kind:0 keep working;
|
|
they also find people who match only by verified name.
|
|
|
|
Matching is the NIP-50 prefix rule, case-insensitive. The query is split on
|
|
whitespace. Every word except the last must equal a word of `name` or
|
|
`display_name`; the last word may be a word prefix. Words may appear in any
|
|
order and may be split across the two fields, so "johnson wes" finds
|
|
"Wes Johnson". A verified profile matches the same way over its three name
|
|
tags, except that its last word must be at least three characters, as in
|
|
NIP-50. The two sources match separately: a query whose words are split
|
|
between a person's kind:0 and their verified profile does not match.
|
|
Accents are not folded.
|
|
|
|
People are ordered by their best match across both sources:
|
|
|
|
1. a name equal to the query;
|
|
2. a name that starts with the query;
|
|
3. a name with a later word that starts with the query;
|
|
4. any other match.
|
|
|
|
For these tiers the query and the names are lower-cased, with whitespace
|
|
runs collapsed to one space and trimmed, by the same database expression.
|
|
Ties go by relevance of the query to the names alone (not the whole
|
|
profile, as in NIP-50), then newest first, then pubkey, so pages do not
|
|
overlap. Profile content that is not a JSON object, or a name field that
|
|
is not a string, never matches. Content that PostgreSQL cannot read as `jsonb`
|
|
(for example a `\u0000` escape), and content with more than 4096 `[` and `{`,
|
|
which bounds how deeply it can nest, has no names, so it never matches and
|
|
never makes the search fail (`PROFILE_NAME_FIELDS` in
|
|
`crates/buzz-search/src/query.rs`).
|
|
|
|
## Deployment
|
|
|
|
Verified-name matching reads `idx_nip_fi_profile_heads_names`. Relay startup
|
|
migration and `buzz-admin migrate` build it before migration `0081` under a
|
|
one-second lock and five-second statement budget. Applied directly as SQL,
|
|
`0081` builds it only while `nip_fi_profile_heads` is empty. A deployment that
|
|
applies migrations out of band (auto-migration off) builds it first, outside a
|
|
transaction:
|
|
|
|
```sql
|
|
CREATE INDEX CONCURRENTLY idx_nip_fi_profile_heads_names
|
|
ON public.nip_fi_profile_heads USING GIN (
|
|
to_tsvector('simple',
|
|
COALESCE(jsonb_path_query_first(event, 'strict $.tags[*] ? (@[0] == "display_name")[1]', '{}', true) #>> '{}', '')
|
|
|| ' ' || COALESCE(jsonb_path_query_first(event, 'strict $.tags[*] ? (@[0] == "username")[1]', '{}', true) #>> '{}', '')
|
|
|| ' ' || COALESCE(jsonb_path_query_first(event, 'strict $.tags[*] ? (@[0] == "full_name")[1]', '{}', true) #>> '{}', ''))
|
|
);
|
|
```
|
|
|
|
If the build fails, drop the invalid index and build it again. The migration
|
|
then checks that the index is valid and has this definition.
|