statement_timeout only counts time spent executing, so a session that
opens a transaction and then stops issuing statements holds its pooled
connection -- and every lock it already took -- with nothing for the
statement timeout to cancel. That is the same connection-starvation
failure the other two limits close, reached a different way.
Set idle_in_transaction_session_timeout alongside them, defaulting to
60s: the budget covers only the gaps between a transaction's statements,
so continuous work is never at risk.
The three limits move into a RuntimeTimeouts struct rather than growing
apply_runtime_connection_timeouts to three same-typed arguments, where a
swapped pair would be a silent misconfiguration. Migration exemption
assertions now iterate every GUC the applier sets, so a fourth limit
added without a matching exemption fails the test.
The armed-pool assertions compare milliseconds read from pg_settings
instead of SHOW text: Postgres re-spells settings on the way out (60s
reads back as 1min), which coupled the test to server formatting.
Signed-off-by: Eli Foster <efoster@squareup.com>
pg_timeout_or_default only checked shape, so a well-formed but
unstorable value ("999...9d") reached every pool's after_connect and
Postgres rejected it there — failing all database access, the exact
outcome the fallback exists to avoid. Parse the magnitude, convert it to
milliseconds by unit, and fall back on overflow or anything above the
int GUC ceiling.
The ceiling lives in buzz-db as PG_TIMEOUT_MAX_MILLIS next to the
timeouts it bounds, pinned to the live server by a Postgres-backed test
so it can drift in neither direction.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Eli Foster <efoster@squareup.com>
Schema migrations now run on a connection with both limits lifted. An
index build on a populated table, or an ACCESS EXCLUSIVE wait behind live
traffic, routinely outlasts the runtime caps, and startup treats a
migration failure as fatal — so inheriting them turned a slow migration
into a relay that cannot boot. sqlx also takes its migration advisory
lock as one waiting statement, so a second replica rolling out would be
canceled mid-wait instead of queueing. The connection is closed rather
than returned to the pool, since its session still carries no limits.
The two values move to DbConfig, wired to BUZZ_DB_STATEMENT_TIMEOUT and
BUZZ_DB_LOCK_TIMEOUT with the previous constants as defaults, so a
backfill or an incident does not need a code change. A malformed value
falls back to the default with a warning: handing it to Postgres would
fail every after_connect and take all database access with it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Eli Foster <efoster@squareup.com>
This change requires a valid signed Blossom authorization request and
current relay membership for every media GET and HEAD request. It
removes the unauthenticated compatibility path and updates desktop reads
to send the required authorization.
This blocks anonymous retrieval and access after relay-membership
revocation. It does not yet bind a blob to its originating channel, so
someone removed from a private channel can still read a known blob while
remaining a relay member. That channel-ACL follow-up remains required
before closing the full finding.
## Testing
- `git diff --check origin/main...codex/security-media-read-auth`
- Rebased onto `origin/main` at `5c98932`
- Full CI pending
Originating Buzz thread:
`buzz://message?channel=3928fe05-df61-4b5d-b9c7-d623b9b10ea1&id=3c6c02312f763fbe0d2bfc33a6c1a362f91d0354f3d18b039cf7a0558c1439d1`
---------
Signed-off-by: Jordan Mecom <jm@squareup.com>
Signed-off-by: Alex Rosenzweig <arosenzweig@squareup.com>
Signed-off-by: Eli Foster <efoster@squareup.com>
Co-authored-by: Eli Foster <efoster@squareup.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
## Summary
- add one strict `BUZZ_S3_ADDRESSING_STYLE=path|virtual` setting shared
by media and Git/CAS storage
- preserve path-style defaults for bundled Compose/Helm MinIO while
supporting Railway's virtual-hosted bucket contract
- fail startup on invalid or non-Unicode values before dependency
connection, and validate the Helm value with the same two choices
- document operator mappings and why endpoint and bucket remain separate
for routing and SigV4 signing
## Best-practice rationale
AWS documents both URL forms and favors virtual-hosted addressing for
S3, while compatibility endpoints such as the bundled MinIO deployment
can require path style. `rust-s3` defaults to virtual/subdomain
addressing and provides `with_path_style()` for the explicit
compatibility case.
Some providers buckets only support as virtual-hosted bucket styles.
This PR therefore uses one explicit, provider-neutral switch rather than
endpoint heuristics or fallback behavior, while retaining `path` as
Buzz's backward-compatible default.
Sources:
-
https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html
- https://docs.rs/rust-s3/0.37.0/s3/bucket/struct.Bucket.html
- https://docs.railway.com/storage-buckets#url-style
-
https://github.com/minio/minio/blob/master/docs/config/README.md#domain
## Validation
- `cargo fmt --all`
- `cargo check --workspace --all-targets`
- targeted `buzz-media` and `buzz-relay` parsing/client-construction
tests for defaults, strict errors, and both URL styles
- Helm unittest: 45/45 passed
- Compose config/render validation passed
- local MinIO path-mode relay startup passed the Git A3 conformance
probe and became ready
- unreachable object storage failed startup and readiness never opened
- push hooks completed the broader Rust and desktop suites successfully
---------
Signed-off-by: npub122y0pqkertljmedu303rl0aqrj3w8pvu43t6jxm6875lzg6f2pwqegc3xc <5288f082d91aff2de5bc8be23fbfa01ca2e3859cac57a91b7a3fa9f12349505c@buzz.block.builderlab.xyz>
Signed-off-by: Kalvin Chau <kalvin@block.xyz>
Co-authored-by: npub122y0pqkertljmedu303rl0aqrj3w8pvu43t6jxm6875lzg6f2pwqegc3xc <5288f082d91aff2de5bc8be23fbfa01ca2e3859cac57a91b7a3fa9f12349505c@buzz.block.builderlab.xyz>
## Summary
- Raise the relay's Postgres pool cap from the `buzz-db` default of 20
to 50 per pool, and expose `BUZZ_DB_POOL_SIZE` for per-deploy tuning
- Applies to the writer pool and, when `READ_DATABASE_URL` is set, the
reader pool; zero/unparsable values fall back to the default
- The `buzz-db` library default is unchanged — only the relay opts into
the larger cap
## Why
During the 2026-07-27 18:40–19:05Z traffic burst on bb-public, per-pod
PG pools pinned at 20 fleet-wide and ~380 requests failed on the 3s
acquire timeout — membership checks, channel access lookups, and
historical queries returning errors to users. The database was nowhere
near a limit: Aurora (db.r8g.8xlarge, ~5,000 max connections) sat at 19%
CPU, 201 connections (~4% of capacity), commit latency flat at 0.01ms.
The 20-connection default was sized for "four relay pods against PG
max_connections=100" (the comment in `buzz-db` says exactly that).
Production now runs 12–15 pods against Aurora — the per-pod cap is the
binding constraint, not the DB.
Budget at the new default: 15 pods × (50 writer + 50 reader + 5 audit) ≈
1,575 potential connections, ~30% of Aurora's ceiling — and actual usage
stays demand-driven (`min_connections` stays 2, connections only open
under load).
Same shape as #2521 (`BUZZ_REDIS_POOL_SIZE`), which fixed the identical
class of ceiling on the Redis side.
## Testing
- `cargo test -p buzz-relay`: 762 passed, 1 failed — the lone red is
`api::mesh_demo::tests::demo_join_forwarded_arm_round_trips_echo`, the
known pre-existing flake; it fails identically on clean `main` at the
same SHA (verified via `git stash` / rerun)
- New test `db_pool_size_env_override_and_invalid_fallback` covers
override, zero, and unparsable fallback
- `defaults_are_valid` extended to pin the new default
- `cargo clippy -p buzz-relay --all-targets -- -D warnings` and `cargo
fmt --check` clean
Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
Configure a dedicated datastore tracing target on the OTLP layer while
preserving explicit logging filters and avoiding span overhead when OTLP
is disabled.
This is in preparation for adding trace spans for datastores used in
Buzz
## Update — 2026-07-27
- Export HTTP requests as `INFO` server spans under `buzz_relay`,
preserving request parentage for datastore spans.
- Configure OTEL span filtering independently with `BUZZ_OTEL_FILTER`,
so `RUST_LOG` changes cannot break trace topology.
- Verify exported HTTP and datastore spans share a trace ID and have the
expected parent/child relationship.
Co-authored-by: Amp <amp@ampcode.com>
