jerry/buzz
1
0
Fork 0
mirror of https://github.com/block/buzz.git synced 2026-10-27 04:30:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

Use Silo for bundled object storage (#8023)

Browse Source
This commit is contained in:
Luke Tornquist
2026-10-07 18:02:27 -04:00
committed by GitHub
parent 032198ce43
commit d742fc5f30
14 changed files with 94 additions and 30 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
.github/ci/minio/README.md vendored
View File

@@ -1,8 +1,9 @@
# Buzz CI MinIO
`ghcr.io/block/buzz-minio:latest` contains MinIO and `mc` for the disposable
Linux AMD64 CI runners. `docker-compose.ci.yml` selects it for both services;
development and deployment defaults stay in `docker-compose.yml`.
Linux AMD64 runners. Only the publisher smoke test selects it through
`docker-compose.ci.yml`. Ordinary CI inherits the Silo pins from
`docker-compose.yml`; the Helm install test inherits the chart defaults.
The **MinIO image** workflow builds only when its inputs change, or on a manual
dispatch. Pull requests build and smoke-test without publishing. On `main`, a
@@ -13,9 +14,10 @@ Once consumers adopt the override, ordinary CI only pulls it; there is no build
fallback or dependency on the publisher. `latest` deliberately floats, and
Compose always pulls it. Docker's pull output records the resolved digest.
The Dockerfile uses the same upstream releases as the development services,
with checksummed official GitHub release binaries and a digest-pinned Alpine
base. MinIO and `mc` are AGPL-3.0; their corresponding source is available at
The Dockerfile pins upstream MinIO releases independently of the Silo images
used by the development and deployment examples. It uses checksummed official
GitHub release binaries and a digest-pinned Alpine base. MinIO and `mc` are
AGPL-3.0; their corresponding source is available at
the [MinIO release](https://github.com/minio/minio/tree/RELEASE.2025-09-07T16-13-09Z)
and [mc release](https://github.com/minio/mc/tree/RELEASE.2025-08-13T08-35-41Z).

4
.github/workflows/_ci-desktop.yml vendored
View File

@@ -122,7 +122,9 @@ jobs:
desktop-smoke-e2e:
name: Desktop Smoke E2E (${{ matrix.shard }})
runs-on: ubuntu-latest
timeout-minutes: 30
# Chromium dependency setup can take 10m; a serial smoke shard needs
# ~25m. Leave time for the full suite and its reports to finish.
timeout-minutes: 45
if: github.event_name == 'push' || inputs.desktop || inputs.desktop_rust || inputs.rust
strategy:
fail-fast: false

6
.github/workflows/_ci-relay.yml vendored
View File

@@ -209,8 +209,6 @@ jobs:
--archive-file target/ci/postgres-tests.tar.zst
desktop-e2e-integration-shard:
env:
COMPOSE_FILE: docker-compose.yml:docker-compose.ci.yml
name: Desktop E2E Integration (${{ matrix.shard }}/2)
runs-on: ubuntu-latest
timeout-minutes: 20
@@ -404,8 +402,6 @@ jobs:
echo "Desktop E2E Integration shards passed"
backend-integration:
env:
COMPOSE_FILE: docker-compose.yml:docker-compose.ci.yml
name: Backend Integration (relay e2e)
runs-on: ubuntu-latest
timeout-minutes: 20
@@ -576,8 +572,6 @@ jobs:
if-no-files-found: ignore
relay-e2e:
env:
COMPOSE_FILE: docker-compose.yml:docker-compose.ci.yml
name: Relay E2E
runs-on: ubuntu-latest
timeout-minutes: 20

2
.github/workflows/mesh-lifecycle.yml vendored
View File

@@ -49,8 +49,6 @@ env:
jobs:
lifecycle-smoke:
env:
COMPOSE_FILE: docker-compose.yml:docker-compose.ci.yml
name: Relay-Driven Mesh Lifecycle Smoke
runs-on: ubuntu-24.04
timeout-minutes: 45

2
ARCHITECTURE.md
View File

@@ -900,7 +900,7 @@ Docker Compose provides the full local development stack. All services include h
| Postgres | `postgres:17-alpine` | 5432 | Primary event store — events, channels, tokens, workflows, audit; full-text search (`search_tsv` GIN) |
| Redis | `redis:7-alpine` | 6379 | Pub/sub fan-out, presence (SET EX), typing (sorted sets) |
| Adminer | `adminer` | 8082 | DB web UI (dev only) |
| MinIO | `quay.io/minio/minio` | 9000 (API), 9001 (console) | S3-compatible object storage (media) |
| Silo (MinIO fork) | `pgsty/silo` | 9000 (API), 9001 (console) | S3-compatible object storage (media) |
| Prometheus | `prom/prometheus` | 9090 | Metrics collection |
### Postgres Schema (key tables)

14
deploy/charts/buzz/README.md
View File

@@ -38,6 +38,20 @@ image:
digest: sha256:<64-lowercase-hex-characters>
```
Bundled object storage uses [Silo](https://silo.pgsty.com/download/), a maintained
MinIO fork, with matching `pgsty/silo` and `pgsty/mc` release images pinned by
multi-platform digest. Both images allow anonymous pulls and support Linux
amd64 and arm64; the upstream Quay MinIO images no longer allow anonymous pulls.
The existing `minio` names and `MINIO_*` settings remain compatible. Keep the
Helm and Compose pins aligned when upgrading, and run Buzz's
[object-store conformance probe](../../../docs/git-on-object-storage.md#conformance-admitting-a-backend-for-a3)
against the new release before deploying it.
For an existing MinIO data volume, follow Silo's
[migration guide](https://silo.pgsty.com/compatibility/migration/) and take a
backup before upgrading. Replacing the container image upgrades the storage
server as well as changing its registry.
## Production (GitOps)
The chart is designed for ArgoCD and Flux. Both render charts with `helm template`, in which mode Helm's `lookup` function returns empty — any chart-side `randAlphaNum` call would regenerate secrets on every sync. The chart-managed Secret path is **only** safe for `helm install` / `helm upgrade`.

2
deploy/charts/buzz/ci/quickstart-values.yaml
View File

@@ -9,8 +9,6 @@ redis:
enabled: true
minio:
enabled: true
image: ghcr.io/block/buzz-minio:latest
mcImage: ghcr.io/block/buzz-minio:latest
relayUrl: wss://buzz.test.local
ownerPubkey: "0000000000000000000000000000000000000000000000000000000000000001"
relay:

10
deploy/charts/buzz/values.yaml
View File

@@ -416,12 +416,14 @@ operatorJobs:
cpu: "1"
memory: 1Gi
# In-cluster MinIO for the quickstart profile only. Production deploys leave
# In-cluster Silo (MinIO fork) for the quickstart profile only. Production deploys leave
# this disabled and use s3.* (or secrets.existingSecret) against managed S3.
minio:
enabled: false # quickstart: set true for bundled in-cluster MinIO
image: quay.io/minio/minio:RELEASE.2025-09-07T16-13-09Z@sha256:14cea493d9a34af32f524e538b8346cf79f3321eff8e708c1e2960462bd8936e
mcImage: quay.io/minio/mc:RELEASE.2025-08-13T08-35-41Z@sha256:a7fe349ef4bd8521fb8497f55c6042871b2ae640607cf99d9bede5e9bdf11727
enabled: false # quickstart: set true for bundled in-cluster Silo
# Silo is a maintained MinIO fork with public amd64/arm64 images.
# Keep the matching server/client release and digest pins aligned with Compose.
image: pgsty/silo:RELEASE.2026-09-16T00-00-00Z@sha256:635197cb9f36d01bee221d34d1c7d7960f6a95c48b0b6c01d99cd13bdae51a46
mcImage: pgsty/mc:RELEASE.2026-09-16T00-00-00Z@sha256:cfc83108c3abb371f8fb84d99c1fdc88f8c237e022409b0081fb7c0a3be634dd
persistence:
enabled: true
size: 10Gi

14
deploy/compose/README.md
View File

@@ -50,6 +50,20 @@ keypair.
Run `./run.sh backup-hint` for the backup checklist.
Bundled object storage uses [Silo](https://silo.pgsty.com/download/), a maintained
MinIO fork, with matching `pgsty/silo` and `pgsty/mc` release images pinned by
multi-platform digest. Both images allow anonymous pulls and support Linux
amd64 and arm64; the upstream Quay MinIO images no longer allow anonymous pulls.
The existing `minio` names and `MINIO_*` settings remain compatible. Keep the
Helm and Compose pins aligned when upgrading, and run Buzz's
[object-store conformance probe](../../docs/git-on-object-storage.md#conformance-admitting-a-backend-for-a3)
against the new release before deploying it.
For an existing MinIO data volume, follow Silo's
[migration guide](https://silo.pgsty.com/compatibility/migration/) and take a
backup before upgrading. Replacing the container image upgrades the storage
server as well as changing its registry.
## Validation
Before sharing an install link publicly, verify a fresh install with:

5
deploy/compose/compose.yml
View File

@@ -85,7 +85,8 @@ services:
- buzz-net
minio:
image: quay.io/minio/minio:RELEASE.2025-09-07T16-13-09Z@sha256:14cea493d9a34af32f524e538b8346cf79f3321eff8e708c1e2960462bd8936e
# Silo preserves the MinIO configuration and health endpoints.
image: pgsty/silo:RELEASE.2026-09-16T00-00-00Z@sha256:635197cb9f36d01bee221d34d1c7d7960f6a95c48b0b6c01d99cd13bdae51a46
command: server /data --console-address ":9001"
environment:
MINIO_ROOT_USER: ${BUZZ_S3_ACCESS_KEY:?set BUZZ_S3_ACCESS_KEY}
@@ -103,7 +104,7 @@ services:
- buzz-net
minio-init:
image: quay.io/minio/mc:RELEASE.2025-08-13T08-35-41Z@sha256:a7fe349ef4bd8521fb8497f55c6042871b2ae640607cf99d9bede5e9bdf11727
image: pgsty/mc:RELEASE.2026-09-16T00-00-00Z@sha256:cfc83108c3abb371f8fb84d99c1fdc88f8c237e022409b0081fb7c0a3be634dd
depends_on:
minio:
condition: service_healthy

5
desktop/tests/e2e/community-rail.spec.ts
View File

@@ -1682,8 +1682,13 @@ test.describe("community rail", () => {
}),
);
}, `community-rail-button-${COMMUNITY_B.id}`);
// Wait for the sensor to activate and publish the move before dropping.
await expect(buttonB).toHaveAttribute("aria-pressed", "true");
// ArrowUp moves the active item one slot up.
await page.keyboard.press("ArrowUp");
await expect(
page.getByRole("status").filter({ hasText: COMMUNITY_B.id }),
).toContainText(`was moved over droppable area ${COMMUNITY_A.id}`);
// Space drops the item — same synthetic dispatch for consistency.
await page.evaluate((testId) => {
const el = document.querySelector(`[data-testid="${testId}"]`);

36
desktop/tests/e2e/virtualization.spec.ts
View File

@@ -2,6 +2,7 @@ import { expect, test } from "@playwright/test";
import type { Locator, Page } from "@playwright/test";
import { installMockBridge } from "../helpers/bridge";
import { waitForMockChannelHeadReady } from "../helpers/channelHeadReady";
const WATERCOOLER_CHANNEL_ID = "a27e1ee9-76a6-5bdf-a5d5-1d85610dad11";
const FORUM_THREAD_ID = "mock-forum-release-thread";
@@ -866,16 +867,47 @@ test("live tail arrivals stay buffered while reading and release on jump", async
);
await page.getByTestId("channel-deep-history").click();
await waitForMockChannelHeadReady(
page,
"deep-history",
"feedf00d-0000-4000-8000-000000000007",
);
await page.evaluate(() => document.fonts.ready.then(() => undefined));
const timeline = page.getByTestId("message-timeline");
await expect(timeline.locator("[data-message-id]").first()).toBeVisible();
await timeline.evaluate((element) => {
// Reader input retires the virtualizer's bottom-follow intent before
// moving into history; a scrollTop write alone leaves it armed.
element.dispatchEvent(new WheelEvent("wheel", { deltaY: -1 }));
element.scrollTop = Math.max(500, element.scrollHeight / 2);
element.dispatchEvent(new Event("scroll", { bubbles: true }));
});
await expect(page.getByTestId("message-scroll-to-latest")).toBeVisible();
const frozenHeight = await timeline.evaluate(
(element) => element.scrollHeight,
const frozenHeight = await timeline.evaluate(async (element) => {
// Entering history renders and measures new virtual rows. Capture the
// baseline only after that layout has stopped changing across frames.
let previousHeight = element.scrollHeight;
let previousOffset = element.scrollTop;
let stableFrames = 0;
for (let frame = 0; frame < 120; frame += 1) {
await new Promise<void>((resolve) =>
requestAnimationFrame(() => resolve()),
);
const height = element.scrollHeight;
const offset = element.scrollTop;
stableFrames =
height === previousHeight && offset === previousOffset
? stableFrames + 1
: 0;
if (stableFrames >= 3) return height;
previousHeight = height;
previousOffset = offset;
}
throw new Error(
"virtual timeline layout did not settle before live arrival",
);
});
await page.evaluate(() => {
window.__BUZZ_E2E_EMIT_MOCK_MESSAGE__?.({

5
docker-compose.harness.yml
View File

@@ -42,7 +42,8 @@ services:
start_period: 5s
minio:
image: quay.io/minio/minio:RELEASE.2025-09-07T16-13-09Z@sha256:14cea493d9a34af32f524e538b8346cf79f3321eff8e708c1e2960462bd8936e
# Silo preserves the MinIO configuration and health endpoints.
image: pgsty/silo:RELEASE.2026-09-16T00-00-00Z@sha256:635197cb9f36d01bee221d34d1c7d7960f6a95c48b0b6c01d99cd13bdae51a46
command: server /data --console-address ":9001"
environment:
MINIO_ROOT_USER: buzz_dev
@@ -60,7 +61,7 @@ services:
start_period: 10s
minio-init:
image: quay.io/minio/mc:RELEASE.2025-08-13T08-35-41Z@sha256:a7fe349ef4bd8521fb8497f55c6042871b2ae640607cf99d9bede5e9bdf11727
image: pgsty/mc:RELEASE.2026-09-16T00-00-00Z@sha256:cfc83108c3abb371f8fb84d99c1fdc88f8c237e022409b0081fb7c0a3be634dd
depends_on:
minio:
condition: service_healthy

5
docker-compose.yml
View File

@@ -101,7 +101,8 @@ services:
restart: unless-stopped
minio:
image: quay.io/minio/minio:RELEASE.2025-09-07T16-13-09Z@sha256:14cea493d9a34af32f524e538b8346cf79f3321eff8e708c1e2960462bd8936e
# Silo preserves the MinIO configuration and health endpoints.
image: pgsty/silo:RELEASE.2026-09-16T00-00-00Z@sha256:635197cb9f36d01bee221d34d1c7d7960f6a95c48b0b6c01d99cd13bdae51a46
container_name: buzz-minio
command: server /data --console-address ":9001"
environment:
@@ -130,7 +131,7 @@ services:
restart: unless-stopped
minio-init:
image: quay.io/minio/mc:RELEASE.2025-08-13T08-35-41Z@sha256:a7fe349ef4bd8521fb8497f55c6042871b2ae640607cf99d9bede5e9bdf11727
image: pgsty/mc:RELEASE.2026-09-16T00-00-00Z@sha256:cfc83108c3abb371f8fb84d99c1fdc88f8c237e022409b0081fb7c0a3be634dd
container_name: buzz-minio-init
depends_on:
minio: