jerry/buzz
1
0
Fork 0
mirror of https://github.com/block/buzz.git synced 2026-11-10 11:21:05 +00:00
Code Issues Packages Projects Releases Wiki Activity

Add manual-only relay image publishing (#9303)

Browse Source 
## Summary

Add **Manual relay image**, a manual-only GitHub Actions workflow that
builds and uploads a Buzz relay image from a chosen repository branch,
tag or full commit SHA. It does not wait for normal CI checks or require
the chosen source to be merged to `main`.

This PR changes only `.github/workflows/docker-manual.yml`. It adds no
scripts or runbook. The normal Docker release workflow and existing
staging-only workflow are unchanged.

### What the workflow does

1. Resolves `source_ref` to one exact source commit.
2. Builds the existing Dockerfile's `runtime` target on native AMD64 and
ARM64 runners.
3. Uploads each architecture to `ghcr.io/block/buzz` using the existing
`GITHUB_TOKEN`.
4. Publishes a unique `manual-<source-sha12>-<run-id>-<attempt>` tag
when both builds succeed.
5. Verifies that the published image contains exactly one runtime for
each architecture and that both runtime source labels match the selected
commit.
6. Reports the source SHA, registry tag and combined digest in the run
summary and a downloadable JSON artifact.

There are no push, pull-request or scheduled triggers. The workflow does
not run the test suite, wait for `ci.yml`, or require a successful CI
qualification. The source must still build successfully, and image
verification must succeed before the workflow reports success.

### Run from GitHub UI

1. Open [Buzz Actions](https://github.com/block/buzz/actions).
2. Select **Manual relay image**.
3. Click **Run workflow** and select `main` or the repository branch
containing the workflow definition you want to run.
4. Enter your block/buzz branch, tag or full SHA in **source_ref**.
Example: `fix-db-timeout`. The commit must be reachable from a
repository branch; PR refs and PR-only commits are rejected.
5. Click **Run workflow** and wait for both builds and **Publish and
verify the combined image** to succeed.
6. Open the run summary and copy its registry tag and digest.

**`source_ref` is the code you want to build.** The workflow branch
selects the build instructions and may be non-main; CLI `--ref` selects
that branch. Your source branch does not need to merge. This new
workflow appears in the UI once its definition is on the default branch.

### Run from CLI

You need GitHub CLI authenticated as a collaborator who can run
workflows. No repository checkout, laptop Docker setup, AWS login or
personal package-publishing token is required.

```bash
# First-time GitHub CLI login only; skip if already authenticated.
gh auth login --hostname github.com --web

# Replace this with your Buzz branch, tag or full commit SHA.
export BUZZ_SOURCE_REF='fix-db-timeout'
gh workflow run docker-manual.yml \
  --repo block/buzz --ref main \
  -f source_ref="$BUZZ_SOURCE_REF"

# Find YOUR new run; its title includes the selected source_ref.
gh run list --repo block/buzz --workflow docker-manual.yml \
  --event workflow_dispatch --limit 5

export BUZZ_RUN_ID='REPLACE_WITH_YOUR_RUN_ID'
gh run watch "$BUZZ_RUN_ID" --repo block/buzz --exit-status
gh run view "$BUZZ_RUN_ID" --repo block/buzz --web
```

### Source and workflow selection

Dispatch may use `main` or another branch in `block/buzz`. CLI `--ref`
selects the workflow definition; `-f source_ref=...` separately selects
the code to build. For example, `--ref feature/build-tooling -f
source_ref=fix-db-timeout` runs the feature branch's workflow against
the fix branch's code. The dispatcher is responsible for selecting
trusted workflow instructions.

The resolver has no package-write permission and runs no selected-source
scripts. It validates the input, fetches canonical branch histories, and
proves the selected commit is reachable from a repository branch before
either publishing job can start. A full SHA cannot bypass the PR-ref
restriction. Tags that point only to unreachable or PR-only commits are
rejected too.

Before this new workflow is registered on the default branch, its exact
definition can be tested through the existing workflow path on the
isolated test branch:

```bash
gh workflow run docker.yml --repo block/buzz \
  --ref codex/manual-relay-image-test -f source_ref=fix-db-timeout
```

This runs a fresh build of the selected repository source. It does not
modify the default branch's normal workflow.

### Image output and deployment

The summary reports an immutable reference such as
`ghcr.io/block/buzz@sha256:...`. The `manual-image-<run-id>-<attempt>`
artifact contains the same digest, source identity and per-architecture
digests. Keep the actual `manual-*` tag as deployment/version metadata
alongside the digest. No synthetic `sha-*` deployment tag or registry
alias is emitted.

The image registry is shared by consumers; the workflow does not select
staging or production. Downstream deployment configuration selects the
image digest and environment. This workflow does not edit deployment
manifests, sync Argo CD, restart workloads, overwrite normal release
tags, or upload directly to AWS ECR. Any downstream cache import and
deployment are separate operations.

Manual images do not receive the normal CI deployment-eligibility
attestation. They are not automatically eligible for canonical
promotion. The selected code and any skipped checks must be evaluated
before an explicit deployment.

### Testing

The earlier implementation completed a real two-architecture build and
publish in [run
42704887052](https://github.com/block/buzz/actions/runs/42704887052).
Its image was also verified in the downstream ECR cache.

The build-and-publish path passed [run
42713322329](https://github.com/block/buzz/actions/runs/42713322329) for
unmerged source `beda6eeaec8ca4a00231a05a079eb168d2a6aec2`. Both native
builds and combined-image verification succeeded. That run predates the
source restrictions added during review.

The current resolver and workflow-origin guard were executed directly
from this YAML against local Git fixtures: 19 source cases and 3 origin
cases passed. These include accepting internal branches/tags/ancestor
SHAs and rejecting PR refs, the raw SHA of a PR-only commit, unreachable
tags, malformed inputs and a foreign repository. Main and non-main
workflow refs are both accepted in block/buzz. Live resolution of a
block/buzz feature branch also passed. The exact image verifier passed
valid-image cases and rejected missing/duplicate platforms, wrong
architecture and wrong source. No deployment was started.

Full local `just ci` stopped at the existing mobile formatting check
(exit 65), with an unresolved `flutter_lints/flutter.yaml` include. The
repository-wide check is not green; no mobile source was changed by this
PR.

### Related issue

No matching issue found. The existing staging-dev workflow publishes to
a distinct staging-only package and accepts branches/tags; this workflow
adds general manual publication and raw commit selection.


<img width="1465" height="821" alt="Screenshot 2026-11-09 at 5 55 42 PM"
src="https://github.com/user-attachments/assets/6a78b31f-9646-4f19-b44a-116c16818331"
/>

---------

Signed-off-by: Ravneet Arora <rarora@squareup.com>
This commit is contained in:
ravarora2
2026-11-09 19:10:25 -05:00
committed by GitHub
parent 402bd1897f
commit 3e9d2e90ce
1 changed files with 248 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

248
.github/workflows/docker-manual.yml vendored Normal file
View File

@@ -0,0 +1,248 @@
name: Manual relay image
run-name: Manual relay image · ${{ inputs.source_ref }}
# Operator-only publishing. This deliberately does not qualify the source via CI.
# Publish manual-* tags only; normal release publishing remains in docker.yml.
on:
workflow_dispatch:
inputs:
source_ref:
description: block/buzz branch, tag, or full SHA reachable from a repository branch
required: true
type: string
permissions: {}
env:
IMAGE_NAME: ${{ vars.GHCR_IMAGE != '' && vars.GHCR_IMAGE || 'ghcr.io/block/buzz' }}
jobs:
resolve:
name: Resolve the selected source
runs-on: ubuntu-24.04
timeout-minutes: 5
permissions:
contents: read
outputs:
sha: ${{ steps.source.outputs.sha }}
tag: ${{ steps.source.outputs.tag }}
steps:
- name: Require block/buzz repository
env:
REPOSITORY: ${{ github.repository }}
shell: bash
run: |
set -euo pipefail
[[ "$REPOSITORY" == block/buzz ]] || {
echo '::error::Run this workflow in block/buzz; use source_ref to select repository code.'
exit 1
}
- name: Resolve a source reachable from a repository branch
id: source
env:
SOURCE_REF: ${{ inputs.source_ref }}
shell: bash
run: |
set -euo pipefail
if [[ -z "$SOURCE_REF" || ${#SOURCE_REF} -gt 255 || ! "$SOURCE_REF" =~ ^[A-Za-z0-9._/@-]+$ ]]; then
echo '::error::Supply a repository branch, tag, or full commit SHA.'
exit 1
fi
case "$SOURCE_REF" in
-*|*..*|*.lock|refs/pull/*|pull/*|*/pull/*)
echo '::error::PR refs and malformed refs are not allowed.'
exit 1
;;
esac
workdir=$(mktemp -d)
trap 'rm -rf "$workdir"' EXIT
git -C "$workdir" init --quiet
git -C "$workdir" remote add origin https://github.com/block/buzz.git
# Fetch only canonical branch histories, never refs/pull or arbitrary SHAs.
# Full history is needed to prove ancestry; blobs are fetched only by build jobs.
git -C "$workdir" fetch --filter=blob:none --no-tags origin \
'+refs/heads/*:refs/remotes/origin/*'
if [[ "$SOURCE_REF" =~ ^[0-9a-fA-F]{40}$ ]]; then
sha=$(git -C "$workdir" rev-parse --verify "${SOURCE_REF}^{commit}")
elif [[ "$SOURCE_REF" == refs/heads/* ]]; then
sha=$(git -C "$workdir" rev-parse --verify "refs/remotes/origin/${SOURCE_REF#refs/heads/}^{commit}")
else
tag_ref="$SOURCE_REF"
if [[ "$SOURCE_REF" != refs/* ]] && \
sha=$(git -C "$workdir" rev-parse --verify "refs/remotes/origin/${SOURCE_REF}^{commit}" 2>/dev/null); then
:
else
if [[ "$SOURCE_REF" != refs/* ]]; then
tag_ref="refs/tags/$SOURCE_REF"
fi
[[ "$tag_ref" == refs/tags/* ]] || {
echo '::error::Only repository branches and tags are allowed.'
exit 1
}
git -C "$workdir" fetch --no-tags origin "$tag_ref:refs/tags/manual-source"
sha=$(git -C "$workdir" rev-parse --verify 'refs/tags/manual-source^{commit}')
fi
fi
[[ "$sha" =~ ^[0-9a-f]{40}$ ]]
branches=$(git -C "$workdir" for-each-ref --format='%(refname)' --contains="$sha" refs/remotes/origin/)
[[ -n "$branches" ]] || {
echo '::error::Selected commit must be reachable from a block/buzz branch; PR-only commits are not allowed.'
exit 1
}
echo "sha=$sha" >> "$GITHUB_OUTPUT"
echo "tag=manual-${sha:0:12}-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}" >> "$GITHUB_OUTPUT"
echo "Selected source: $sha. CI qualification is skipped."
build:
name: Build (${{ matrix.platform }})
needs: resolve
runs-on: ${{ matrix.runner }}
timeout-minutes: 90
permissions:
contents: read
packages: write
strategy:
fail-fast: false
matrix:
include:
- platform: linux/amd64
runner: ubuntu-24.04
arch: amd64
- platform: linux/arm64
runner: ubuntu-24.04-arm
arch: arm64
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
ref: ${{ needs.resolve.outputs.sha }}
persist-credentials: false
- uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0
with:
buildkitd-config-inline: |
[worker.oci]
max-parallelism = 2
- uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push the runtime by digest
id: build
uses: docker/build-push-action@f9f3042f7e2789586610d6e8b85c8f03e5195baf # v7.2.0
with:
context: .
file: ./Dockerfile
target: runtime
platforms: ${{ matrix.platform }}
build-args: |
BUZZ_SOURCE_SHA=${{ needs.resolve.outputs.sha }}
BUZZ_BUILD_ID=manual-actions:${{ github.run_id }}:${{ github.run_attempt }}
BUZZ_BUILD_URL=https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}/attempts/${{ github.run_attempt }}
labels: |
org.opencontainers.image.source=https://github.com/${{ github.repository }}
org.opencontainers.image.revision=${{ needs.resolve.outputs.sha }}
xyz.buzz.manual-build=true
outputs: type=image,name=${{ env.IMAGE_NAME }},push-by-digest=true,name-canonical=true,push=true
cache-from: |
type=registry,ref=${{ env.IMAGE_NAME }}-buildcache:${{ matrix.arch }}
type=registry,ref=${{ env.IMAGE_NAME }}-buildcache:manual-${{ matrix.arch }}
cache-to: type=registry,ref=${{ env.IMAGE_NAME }}-buildcache:manual-${{ matrix.arch }},mode=max,compression=zstd
- name: Record this architecture's published digest
env:
IMAGE_DIGEST: ${{ steps.build.outputs.digest }}
shell: bash
run: |
set -euo pipefail
[[ "$IMAGE_DIGEST" =~ ^sha256:[0-9a-f]{64}$ ]]
mkdir -p "$RUNNER_TEMP/manual-digest"
printf '%s\n' "$IMAGE_DIGEST" > "$RUNNER_TEMP/manual-digest/digest"
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: manual-digest-${{ matrix.arch }}
path: ${{ runner.temp }}/manual-digest/digest
if-no-files-found: error
overwrite: true
retention-days: 7
publish:
name: Publish and verify the combined image
needs: [resolve, build]
runs-on: ubuntu-24.04
timeout-minutes: 10
permissions:
contents: read
packages: write
steps:
- uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0
- uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
pattern: manual-digest-*
path: ${{ runner.temp }}/manual-digests
- name: Assemble and verify the published image
env:
SOURCE_SHA: ${{ needs.resolve.outputs.sha }}
MANUAL_TAG: ${{ needs.resolve.outputs.tag }}
shell: bash
run: |
set -euo pipefail
digests=()
for arch in amd64 arm64; do
digest=$(cat "$RUNNER_TEMP/manual-digests/manual-digest-$arch/digest")
[[ "$digest" =~ ^sha256:[0-9a-f]{64}$ ]]
digests+=("$IMAGE_NAME@$digest")
done
image="$IMAGE_NAME:$MANUAL_TAG"
docker buildx imagetools create --tag "$image" \
--annotation "index:org.opencontainers.image.revision=$SOURCE_SHA" \
--annotation "index:org.opencontainers.image.source=https://github.com/$GITHUB_REPOSITORY" \
--annotation "index:xyz.buzz.manual-build=true" "${digests[@]}"
# Verify the published bytes using this workflow, not scripts from the selected source.
merged_digest=$(docker buildx imagetools inspect "$image" --format '{{json .Manifest}}' |
jq -er '.digest | select(test("^sha256:[0-9a-f]{64}$"))')
pinned="$IMAGE_NAME@$merged_digest"
source_url="https://github.com/$GITHUB_REPOSITORY"
docker buildx imagetools inspect "$pinned" --raw > "$RUNNER_TEMP/index.json"
jq -e '.manifests |
map(select((.platform.os == "unknown" and .platform.architecture == "unknown") | not)) |
map(.platform.os + "/" + .platform.architecture) | sort == ["linux/amd64", "linux/arm64"]' \
"$RUNNER_TEMP/index.json" >/dev/null
runtime_digests='{}'
for arch in amd64 arm64; do
child=$(jq -er --arg arch "$arch" '.manifests[] |
select(.platform.os == "linux" and .platform.architecture == $arch) |
.digest | select(test("^sha256:[0-9a-f]{64}$"))' "$RUNNER_TEMP/index.json")
docker buildx imagetools inspect "$IMAGE_NAME@$child" --format '{{json .Image}}' |
jq -e --arg arch "$arch" --arg sha "$SOURCE_SHA" --arg source "$source_url" '
.os == "linux" and .architecture == $arch and
.config.Labels["org.opencontainers.image.revision"] == $sha and
.config.Labels["org.opencontainers.image.source"] == $source' >/dev/null
runtime_digests=$(jq -cn --argjson current "$runtime_digests" \
--arg platform "linux/$arch" --arg digest "$child" '$current + {($platform): $digest}')
done
jq -n --arg image "$pinned" --arg tag "$image" --arg digest "$merged_digest" \
--arg sha "$SOURCE_SHA" --arg source "$source_url" --argjson platforms "$runtime_digests" \
'{image:$image, tag:$tag, digest:$digest, source_sha:$sha, source_url:$source,
platforms:$platforms,
ci_qualification:"skipped", status:"registry-verified"}' \
> "$RUNNER_TEMP/manual-image.json"
{
echo '## Manual relay image published to GHCR'
echo
echo "Source commit: $SOURCE_SHA"
echo "Registry tag: $image"
echo "Digest: $(jq -r .digest "$RUNNER_TEMP/manual-image.json")"
echo 'Platforms: linux/amd64, linux/arm64'
echo 'CI qualification was skipped. No deployment was started.'
echo 'Any downstream registry import must be verified separately.'
} >> "$GITHUB_STEP_SUMMARY"
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: manual-image-${{ github.run_id }}-${{ github.run_attempt }}
path: ${{ runner.temp }}/manual-image.json
if-no-files-found: error
retention-days: 30