jerry/buzz
1
0
Fork 0
mirror of https://github.com/block/buzz.git synced 2026-11-07 01:41:06 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
8331e4c4be8c413692abf3f14cfe4681db359713
buzz/Cargo.toml
Luke Tornquist abb2586f8c Pin sqlx to the upstream fix for a cancelled BEGIN (#9192) 
## Problem

sqlx 0.9.0 counts a transaction only after Postgres acknowledges its
`BEGIN`. If the future is dropped while `BEGIN` is still in flight (a
cancelled REQ history read, a client that disconnects, a timeout), sqlx
doesn't queue a `ROLLBACK`. The pool's release ping then reads the late
`BEGIN` reply, and the writer pool hands the connection to the next
borrower while it is still inside that transaction. The reader pool
already closes such connections in its release check
(`Db::read_pool_options`). The writer pool has no release check.

Load tests showed what happens next:

- `there is already a transaction in progress` (25001) on the next event
read's `BEGIN`.
- Sometimes a burst of `current transaction is aborted` (25P02). The
leftover from the event read is `READ ONLY`, so the next write fails and
aborts it, and every later statement on that connection fails until the
connection is closed. During a burst, ban checks failed closed with 503
and window reads returned 500.
- A read-write leftover would also leave autocommit writes uncommitted,
and the next read's `ROLLBACK` would discard them.

## Change

- Pin every sqlx crate to transact-rs/sqlx#4394 (merged, not yet
released). That commit takes the transaction count before the `BEGIN`
round trip, so a cancelled begin queues its `ROLLBACK`. The crates share
types, so they have to move together. The pin also picks up the other 23
commits between v0.9.0 and that fix, including the libsqlite3-sys fix
`sqlx-sqlite` was already pinned for.
- Add a regression test,
`writer_pool_postgres_tests::begin_cancelled_in_flight_leaves_no_writer_transaction_open`.
It holds `BEGIN` in flight behind a paused proxy and times it out, for
both the event read's `BEGIN ... READ ONLY` and a plain `begin()`. It
then asserts that the next borrower's autocommit write commits and that
its backend is `idle`, not inside a transaction.

Other upstream changes in the pin that reach the Postgres path:
`TCP_NODELAY` on sockets (transact-rs/sqlx#4336), reuse of the TLS
connector across connection attempts (transact-rs/sqlx#4242), a fix for
a memory leak when the statement cache is disabled
(transact-rs/sqlx#4337), a describe-time generic-plan change used only
by the macros (transact-rs/sqlx#4286), and the `whoami` default-user fix
(transact-rs/sqlx#4303). The rest are SQLite, MySQL, CLI, or docs
changes.

Remove the pin once sqlx 0.9.1 ships.

## What got simpler

The lockfile loses its second `sqlx-core` copy, which the
sqlx-sqlite-only pin used to pull in. No runtime code changes. A
pool-level release check would have fixed the same bug, but it costs a
round trip on every writer release.

Comments that described the 0.9.0 behavior as current are refreshed: the
reader pool's release check (`Db::read_pool_options`) and the lease
heartbeat (`EntityDeletionStore::heartbeat`). The reader test
`proof_timeout_mid_begin_leaves_no_reader_transaction_open` asserted
that the release check closed the connection left mid-`BEGIN`. With the
pin, sqlx rolls that transaction back itself, so the test now asserts
the check finds nothing to close. That also catches a sqlx regression
the check would otherwise hide.

Identified debt (not in this PR): the release check's open-transaction
arm, and the reader-session `default_transaction_isolation` pin it
depends on, are now only a backstop against a sqlx regression. They
could be removed in a follow-up.

## Testing

- Against sqlx 0.9.0 (main's `Cargo.toml`/`Cargo.lock` at 962b3fdd), the
new writer test fails for each `BEGIN` form on its own. The event read
fails with `cannot execute INSERT in a read-only transaction`, and after
a plain `begin()` the next borrower's backend is not `idle`. With the
pin it passes 5/5.
- Single-machine load-test A/B against main (962b3fdd), with three main
runs and two pin runs. Main logged 25001 and 25P02 in every run (41/80,
10/9, 12/18); the pin logged none. Client error rate, latency, and
throughput matched within run-to-run noise.
- Full suite on the pin: every `just ci` recipe, the PostgreSQL lane,
and `just test`. The only failure from the pin was the reader test
above. Reds that don't involve sqlx: the desktop Tauri recipes
(`alsa-sys` needs ALSA headers on the build host),
`mobile-check`/`mobile-test` (a Flutter SDK `ColorScheme` type clash),
and two `buzz-acp` `git_bootstrap` tests (no git credentials on the
host). Two bridge search-timeout tests failed only under the full
parallel lane; run alone they pass on both the pin and 0.9.0.
- Two later commits change only tests, comments, and `Cargo.lock` (the
lock now moves only the sqlx crates). At those heads: the PostgreSQL
lane, the unit lane, and workspace clippy pass. Three tests failed once
under a contended full lane, and each passed 3/3 alone on both the pin
and 0.9.0. The two transaction tests fail on 0.9.0 and pass on the pin.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Luke Tornquist <tornquist@squareup.com>
Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 13:07:04 -05:00

213 lines
8.2 KiB
TOML
Raw Blame History

[workspace]
members = [
"crates/buzz-relay",
"crates/buzz-core",
"crates/ifc-core",
"crates/buzz-conformance",
"crates/buzz-push-gateway",
"crates/buzz-db",
"crates/buzz-pubsub",
"crates/buzz-auth",
"crates/buzz-search",
"crates/buzz-audit",
"crates/buzz-acp",
"crates/buzz-agent",
"crates/sprig",
"crates/buzz-test-client",
"crates/buzz-ws-client",
"crates/buzz-admin",
"crates/buzz-deletion",
"crates/buzz-feature-flags",
"crates/buzz-workflow",
"crates/buzz-media",
"crates/buzz-cli",
"crates/buzz-pairing-cli",
"crates/buzz-sdk",
"crates/buzz-persona",
"crates/git-credential-nostr",
"crates/git-sign-nostr",
"crates/buzz-pair-relay",
"crates/buzz-relay-mesh",
"crates/buzz-mesh-smoke",
"crates/buzz-dev-mcp",
"crates/buzz-voice",
"crates/buzz-audio-client",
"crates/buzz-backend-kubernetes",
"crates/buzz-datastore-tracing",
"examples/countdown-bot",
]
exclude = ["desktop/src-tauri"]
resolver = "2"
[workspace.package]
version = "0.1.0"
edition = "2021"
rust-version = "1.88.0"
license = "Apache-2.0"
repository = "https://github.com/block/sprout"
[workspace.dependencies]
# Runtime
tokio = { version = "1", features = ["rt-multi-thread", "macros", "net", "time", "sync", "io-util", "signal", "process"] }
tokio-util = { version = "0.7", features = ["rt", "codec"] }
# HTTP + WebSocket
axum = { version = "0.8", features = ["ws", "macros"] }
tower = { version = "0.5", features = ["timeout", "util", "limit"] }
tower-http = { version = "0.6", features = ["trace", "cors", "compression-gzip", "limit", "timeout", "fs"] }
# Database
sqlx = { version = "0.9", features = [
"runtime-tokio", "tls-rustls", "postgres", "uuid", "chrono", "json"
] }
# Redis
redis = { version = "1.0", features = ["tokio-comp", "tokio-rustls-comp"] }
# Kubernetes (buzz-backend-kubernetes provider). No `ring` feature here: the
# process-level CryptoProvider is installed explicitly at startup, matching
# buzz-cli/buzz-acp/buzz-admin/buzz-relay/buzz-dev-mcp — see the comment on the
# crate's own rustls dependency.
kube = { version = "2.0", default-features = false, features = ["client", "rustls-tls"] }
k8s-openapi = { version = "0.26", features = ["v1_31"] }
# Nostr
nostr = { version = "0.44", features = ["nip44", "nip98"] }
# Serialization
serde = { version = "1", features = ["derive"] }
postcard = { version = "1", default-features = false, features = ["use-std"] }
# Inter-relay mesh transport (buzz-relay-mesh)
iroh = { version = "1.0.0-rc.0", default-features = false, features = ["tls-ring"] }
serde_json = { version = "1", features = ["raw_value"] }
serde_yaml = "0.9"
evalexpr = "11"
cron = "0.16"
# Observability
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter", "json"] }
tracing-opentelemetry = { version = "0.33" }
opentelemetry = { version = "0.32", features = ["trace"] }
opentelemetry_sdk = { version = "0.32", features = ["trace", "rt-tokio"] }
opentelemetry-otlp = { version = "0.32", default-features = false, features = ["trace", "grpc-tonic", "tls-ring"] }
metrics = "0.24"
metrics-exporter-prometheus = "0.18"
metrics-util = "0.20"
# Error handling
thiserror = "2"
anyhow = "1"
# Utilities
uuid = { version = "1", features = ["v4", "serde"] }
chrono = { version = "0.4", features = ["serde"] }
httpdate = "1"
# JWT / JWS verification (NIP-FI federated identity assertions)
jsonwebtoken = { version = "10.4.0", default-features = false, features = ["aws_lc_rs"] }
# HTTP client (webhook delivery)
reqwest = { version = "0.13", features = ["json", "rustls", "stream"], default-features = false }
# Cryptography
sha2 = "0.11"
hex = "0.4"
hmac = "0.13"
base64 = "0.22"
# Randomness
rand = "0.10"
subtle = "2.6"
zeroize = "1.8"
# Concurrent data structures
dashmap = "6"
moka = { version = "0.12", features = ["sync"] }
arc-swap = "1"
lru = "0.16"
# Async stream utilities
futures-util = "0.3"
# WebSocket client (test client)
tokio-tungstenite = { version = "0.29", features = ["rustls-tls-webpki-roots"] }
url = "2"
# Property-based testing (dev-only)
proptest = "1"
# MCP SDK (used by buzz-dev-mcp and buzz-agent)
rmcp = { version = "1.1.0", features = ["server", "transport-io", "macros"] }
schemars = { version = "1", default-features = false }
# Internal crates
buzz-core = { path = "crates/buzz-core" }
buzz-conformance = { path = "crates/buzz-conformance" }
buzz-db = { path = "crates/buzz-db" }
buzz-deletion = { path = "crates/buzz-deletion" }
buzz-feature-flags = { path = "crates/buzz-feature-flags" }
buzz-auth = { path = "crates/buzz-auth" }
buzz-pubsub = { path = "crates/buzz-pubsub" }
buzz-search = { path = "crates/buzz-search" }
buzz-audit = { path = "crates/buzz-audit" }
buzz-workflow = { path = "crates/buzz-workflow" }
buzz-media = { path = "crates/buzz-media" }
buzz-sdk = { path = "crates/buzz-sdk" }
buzz-ws-client = { path = "crates/buzz-ws-client" }
buzz-relay-mesh = { path = "crates/buzz-relay-mesh" }
buzz-datastore-tracing = { path = "crates/buzz-datastore-tracing" }
# CI profile — builds the relay for desktop e2e. Dependencies keep full
# release optimization (warm from main's cache; they carry the runtime hot
# path: tokio/sqlx/axum). Workspace crates build at opt-level 1 — enough for
# stable e2e timing (PR #307 flakiness was opt-0 + debug-assertions) at
# roughly half the codegen cost. Incremental stays disabled: rust-cache exports
# CARGO_INCREMENTAL=0, and the CI compiler cache requires non-incremental units.
[profile.ci]
inherits = "release"
lto = false
opt-level = 1
[profile.ci.package."*"]
opt-level = 3
# Sprig profile — optimized for deploy-anywhere Sprig release artifacts.
# Sprig is distributed over the network and installed on fresh hosts, so binary
# size matters more than compile speed here. Keep this separate from the normal
# `release` profile so desktop/dev release builds do not inherit the slower
# size-focused settings unless they opt in explicitly.
[profile.sprig]
inherits = "release"
opt-level = "z"
lto = "fat"
codegen-units = 1
panic = "abort"
strip = true
# Temporary fork pin: aws-creds 0.39.1 (via rust-s3) cannot read EKS Pod Identity
# credentials (AWS_CONTAINER_CREDENTIALS_FULL_URI + AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE),
# which the relay pod on bb-block requires for S3 media + git storage. This pins
# aws-creds to a fork that adopts the aws-creds portion of durch/rust-s3#449
# (FULL_URI + token-file + Authorization header, refresh-safe, with a loopback
# allowlist for the auth token). Revert to crates.io once #449 lands upstream.
[patch.crates-io]
aws-creds = { git = "https://github.com/tlongwell-block/rust-s3", rev = "c9fce3620dd434c1f810101d672cf384268dbb0f" }
# SQLx: pin every crate to one unreleased upstream commit until 0.9.1 ships.
# The crates share types, so they must move together. It carries two fixes:
# - https://github.com/transact-rs/sqlx/pull/4394: a `BEGIN` whose future is
# dropped before Postgres answers no longer returns its pooled connection
# still inside the transaction. 0.9.0 queues no `ROLLBACK` for it, so the
# next borrower runs inside the leftover transaction (`25001`, then `25P02`
# once a statement in it fails).
# - https://github.com/transact-rs/sqlx/pull/4402: MeshLLM 0.76 uses
# libsqlite3-sys 0.38, which 0.9.0's optional SQLite driver rejects; Cargo
# resolves it even though Buzz only enables PostgreSQL.
sqlx = { git = "https://github.com/launchbadge/sqlx", rev = "f1e94ec34dd5f11ad6488a3a8ac7988439a3f757" }
sqlx-core = { git = "https://github.com/launchbadge/sqlx", rev = "f1e94ec34dd5f11ad6488a3a8ac7988439a3f757" }
sqlx-macros = { git = "https://github.com/launchbadge/sqlx", rev = "f1e94ec34dd5f11ad6488a3a8ac7988439a3f757" }
sqlx-macros-core = { git = "https://github.com/launchbadge/sqlx", rev = "f1e94ec34dd5f11ad6488a3a8ac7988439a3f757" }
sqlx-mysql = { git = "https://github.com/launchbadge/sqlx", rev = "f1e94ec34dd5f11ad6488a3a8ac7988439a3f757" }
sqlx-postgres = { git = "https://github.com/launchbadge/sqlx", rev = "f1e94ec34dd5f11ad6488a3a8ac7988439a3f757" }
sqlx-sqlite = { git = "https://github.com/launchbadge/sqlx", rev = "f1e94ec34dd5f11ad6488a3a8ac7988439a3f757" }
Reference in New Issue View Git Blame Copy Permalink