mirror of
https://github.com/block/buzz.git
synced 2026-10-27 12:40:30 +00:00
Buzz push currently selects an application profile, such as `dogfood`, during client registration. We want each gateway to serve one Apple app configured by the operator instead. This removes profile selection from the client, relay, and gateway, while allowing internal and public gateways to use their own Apple identities. This PR implements that model in the Buzz source and upstream deployment chart. Discovery, enrollment, grants, and leases no longer carry an application-profile selector. Each gateway has one configured Apple application and APNs transport. The protocol changes are incompatible with the old registration format, so relays advertise `buzz-push-v1` instead of `nip-pl`. Old clients do not discover the new push capability. Updated clients use separate Keychain records and register again. The [push notes](https://github.com/block/buzz/blob/push-capability-v1/docs/buzz-push.md) describe the experimental contract. Landing this PR is the first step. Publish the matching gateway image and upstream chart, then land the [shared chart update](https://github.com/squareup/cash-helm-charts/pull/5740) and [gateway deployment update](https://github.com/squareup/builderbot-platform-core-infrastructure/pull/1085). Before cutover, stop old gateway instances and check legacy registration state. Migrations refuse to proceed if old gateway installations or active relay leases remain; they do not delete that state. Physical-device verification remains necessary before production activation. ### Validation Gateway and focused Flutter push tests pass, as do the Swift package, simulator build, and native notification/enrichment tests. A local relay advertises `buzz-push-v1` without `nip-pl` or `app_profiles`. The PostgreSQL suite passed 868 tests; its two migration-test failures were corrected and pass a focused rerun together with the gateway cutover guard. Chart rendering and release-contract checks pass. Desktop JavaScript tests, desktop and web builds, native desktop compilation, mobile analysis/tests, and desktop native tests pass. The full Rust unit lane and prescribed `just test` command pass, including workspace integration tests against task-isolated local services. The production overlay now requires an explicitly supplied APNs topic, with omission and rendered-value regression coverage. The corrected Swift checksum assertion passes the complete package suite: 78 XCTest tests and 42 Swift Testing tests. No physical-device enrollment or production cutover is claimed. Local merge validation: the embedded migration test and formatting pass after reserving profile migration 0060 behind rollback migration 0059. The local pre-push Rust lane failed ACP timing/steering tests (`idle_resets_on_stdout_activity`, `keepalive_resets_idle_past_deadline`, and the failed/missing-outcome steering tests). ACP and its dependency manifests are unchanged by this branch; all four pass in an isolated single-thread rerun. The documented local-check exception skips only the Rust pre-push lane for publication; required CI remains mandatory. Mobile and desktop pre-push lanes passed. Combined migration validation: after rollback 0059 landed, the main merge preserves profile migration 0060 and both embedded migration assertions. The focused migration test and formatting pass. The next local push again failed only the unchanged ACP idle/keepalive timing tests (1053 passed, 2 failed, 3 ignored); other lanes passed. Publication uses the same documented local Rust-lane exception; required CI remains mandatory. Follow-up review fix: removed the legacy profile column from the copied-lease fixture. All four event-follow-up contract tests pass on isolated PostgreSQL databases, covering desired and migrated schemas in dual and app-only modes. Formatting passes. Publication retains the prior passing local-lane evidence and uses the authorized pre-push exception for the known unrelated ACP timing failures; required CI remains mandatory. --------- Signed-off-by: Tom Brow <tomb@block.xyz>
273 lines
13 KiB
Bash
Executable File
273 lines
13 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
set -euo pipefail
|
|
bash deploy/charts/buzz-push-gateway/tests/runtime-options.sh
|
|
bash deploy/charts/buzz-push-gateway/tests/grant-lifetime.sh
|
|
bash deploy/charts/buzz-push-gateway/tests/migration-annotations.sh
|
|
out=$(mktemp); production_out=$(mktemp); route_out=$(mktemp); datadog_out=$(mktemp)
|
|
trap 'rm -f "$out" "$production_out" "$route_out" "$datadog_out" "${monitoring_out:-}"' EXIT
|
|
gateway_origin_arg=(--set 'gatewayOrigin=https://push.example')
|
|
|
|
# Generic values work without an application origin.
|
|
helm lint deploy/charts/buzz-push-gateway >/dev/null
|
|
helm template push deploy/charts/buzz-push-gateway >"$out"
|
|
# Production values support a platform-owned ingress without rendering an
|
|
# HTTPRoute. The environment-owned inputs remain mandatory.
|
|
production_args=(
|
|
-f deploy/charts/buzz-push-gateway/values-production.yaml
|
|
--set 'image.digest=sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa'
|
|
--set 'application.appAttestAppId=REALTEAM.com.example.production'
|
|
--set 'networkPolicy.postgresEgressCidrs[0]=10.42.0.0/16'
|
|
)
|
|
if helm template push deploy/charts/buzz-push-gateway "${production_args[@]}" >/dev/null 2>&1; then
|
|
echo 'expected production values without an APNs topic to fail' >&2
|
|
exit 1
|
|
fi
|
|
production_args+=(--set 'application.apnsTopic=com.example.production')
|
|
helm lint deploy/charts/buzz-push-gateway "${production_args[@]}" >/dev/null
|
|
helm template push deploy/charts/buzz-push-gateway "${production_args[@]}" >"$production_out"
|
|
|
|
# Gateway API remains an explicit supported ingress mode when an operator opts
|
|
# in and supplies the environment-owned parent.
|
|
helm template push deploy/charts/buzz-push-gateway \
|
|
"${gateway_origin_arg[@]}" \
|
|
--set httpRoute.enabled=true \
|
|
--set 'httpRoute.parentRefs[0].name=production-gateway' \
|
|
--set 'httpRoute.parentRefs[0].namespace=gateway-system' \
|
|
>"$route_out"
|
|
|
|
env -u GEM_HOME -u GEM_PATH -u RUBYLIB -u RUBYOPT ruby -ryaml -rset \
|
|
- "$out" "$production_out" "$route_out" <<'RUBY'
|
|
def assert!(condition, detail = "assertion failed")
|
|
raise detail unless condition
|
|
end
|
|
|
|
# The application serves plaintext HTTP behind ingress TLS termination. Check
|
|
# every ingress mode so service discovery never advertises application TLS.
|
|
ARGV.each do |path|
|
|
resources = YAML.load_stream(File.read(path)).compact
|
|
service = resources.find { |x| x["kind"] == "Service" }
|
|
ports = service.dig("spec", "ports")
|
|
assert!(ports == [{ "name" => "http", "port" => 8080, "targetPort" => "public" }], ports.inspect)
|
|
deployment = resources.find { |x| x["kind"] == "Deployment" }
|
|
container_ports = deployment.dig("spec", "template", "spec", "containers", 0, "ports")
|
|
assert!(container_ports.any? { |port| port["name"] == "public" && port["containerPort"] == 8080 })
|
|
end
|
|
|
|
xs = YAML.load_stream(File.read(ARGV[0])).compact
|
|
svc = xs.find { |x| x["kind"] == "Service" }
|
|
assert!(svc.dig("spec", "ports").map { |port| port["targetPort"] } == ["public"])
|
|
d = xs.find { |x| x["kind"] == "Deployment" }
|
|
j = xs.find { |x| x["kind"] == "Job" }
|
|
runtime = {
|
|
"app.kubernetes.io/name" => "buzz-push-gateway",
|
|
"app.kubernetes.io/instance" => "push",
|
|
"app.kubernetes.io/component" => "runtime",
|
|
}
|
|
migration = runtime.merge("app.kubernetes.io/component" => "migration")
|
|
assert!(svc.dig("spec", "selector") == runtime)
|
|
assert!(d.dig("spec", "selector", "matchLabels") == runtime)
|
|
assert!(d.dig("spec", "template", "metadata", "labels") == runtime)
|
|
assert!(d.dig("spec", "template", "metadata", "annotations").nil?)
|
|
assert!(j.dig("spec", "template", "metadata", "labels") == migration)
|
|
assert!(svc.dig("spec", "selector") != j.dig("spec", "template", "metadata", "labels"))
|
|
jenv = j.dig("spec", "template", "spec", "containers", 0, "env").to_h { |entry| [entry["name"], entry] }
|
|
assert!(jenv.dig("BUZZ_PUSH_RUNTIME_DATABASE_ROLE", "value") == "buzz_push_gateway_runtime")
|
|
assert!(jenv.fetch("DATABASE_URL").key?("valueFrom"))
|
|
assert!(j.dig("spec", "template", "spec", "containers", 0, "args") == ["--migrate-only"])
|
|
assert!(j.dig("metadata", "annotations") == {
|
|
"helm.sh/hook" => "pre-install,pre-upgrade",
|
|
"helm.sh/hook-weight" => "-5",
|
|
"helm.sh/hook-delete-policy" => "before-hook-creation,hook-succeeded",
|
|
})
|
|
env_names = d.dig("spec", "template", "spec", "containers", 0, "env")
|
|
.map { |entry| entry["name"] }.to_set
|
|
required = Set.new(%w[
|
|
DATABASE_URL BUZZ_PUSH_APNS_CERT_PATH
|
|
BUZZ_PUSH_APNS_TOPIC BUZZ_PUSH_APP_ATTEST_APP_ID
|
|
BUZZ_PUSH_GRANT_KEYS BUZZ_PUSH_TOKEN_KEYS BUZZ_PUSH_MAX_GRANT_LIFETIME_SECONDS
|
|
])
|
|
assert!(required.subset?(env_names))
|
|
assert!(!env_names.include?("BUZZ_PUSH_GATEWAY_ORIGIN"))
|
|
assert!(!env_names.any? { |name| name.include?("APP_STORE") })
|
|
apns_volume = d.dig("spec", "template", "spec", "volumes").find { |volume| volume["name"] == "apns" }
|
|
assert!(apns_volume.dig("secret", "defaultMode") == 0o400, apns_volume.inspect)
|
|
assert!(d.dig("spec", "replicas") >= 2)
|
|
assert!(!xs.any? { |x| x["kind"] == "HTTPRoute" })
|
|
# Observability is opt-in: default render exposes no scrape CRDs and 8081 stays
|
|
# free of pod ingress (only 8080 is reachable).
|
|
assert!(!xs.any? { |x| %w[PodMonitor PrometheusRule].include?(x["kind"]) })
|
|
nps = xs.select { |x| x["kind"] == "NetworkPolicy" }
|
|
np = nps.find { |x| x.dig("metadata", "name") == "push-buzz-push-gateway" }
|
|
migration_np = nps.find { |x| x.dig("metadata", "name") == "push-buzz-push-gateway-migration" }
|
|
assert!(np.dig("spec", "podSelector", "matchLabels") == runtime)
|
|
assert!(migration_np.dig("spec", "podSelector", "matchLabels") == migration)
|
|
assert!(migration_np.dig("metadata", "annotations") == {
|
|
"helm.sh/hook" => "pre-install,pre-upgrade",
|
|
"helm.sh/hook-weight" => "-10",
|
|
"helm.sh/hook-delete-policy" => "before-hook-creation",
|
|
})
|
|
assert!(migration_np.dig("metadata", "annotations", "helm.sh/hook-weight").to_i < j.dig("metadata", "annotations", "helm.sh/hook-weight").to_i)
|
|
assert!(migration_np.dig("spec", "ingress") == [])
|
|
assert!(migration_np.dig("spec", "policyTypes") == %w[Ingress Egress])
|
|
migration_ports = migration_np.dig("spec", "egress")
|
|
.flat_map { |rule| rule.fetch("ports", []) }.map { |port| port["port"] }.to_set
|
|
assert!(migration_ports == Set[53, 5432], migration_ports.inspect)
|
|
assert!(!migration_np.dig("spec", "egress").flat_map { |rule| rule.fetch("ports", []) }.any? { |port| port["port"] == 443 })
|
|
ingress_ports = np.dig("spec", "ingress")
|
|
.flat_map { |rule| rule.fetch("ports", []) }.map { |port| port["port"] }.to_set
|
|
assert!(ingress_ports == Set[8080], ingress_ports.inspect)
|
|
production = YAML.load_stream(File.read(ARGV[1])).compact
|
|
assert!(!production.any? { |x| x["kind"] == "HTTPRoute" })
|
|
production_deployment = production.find { |x| x["kind"] == "Deployment" }
|
|
production_image = production_deployment.dig("spec", "template", "spec", "containers", 0, "image")
|
|
assert!(production_image == "ghcr.io/block/buzz-push-gateway@sha256:#{"a" * 64}", production_image.inspect)
|
|
production_env = production_deployment.dig("spec", "template", "spec", "containers", 0, "env")
|
|
assert!(production_env.find { |env| env["name"] == "BUZZ_PUSH_APNS_TOPIC" }["value"] == "com.example.production")
|
|
route = YAML.load_stream(File.read(ARGV[2])).compact.find { |x| x["kind"] == "HTTPRoute" }
|
|
assert!(!route.dig("spec", "parentRefs").empty?)
|
|
assert!(route.dig("spec", "hostnames") == ["push.example"])
|
|
RUBY
|
|
|
|
# An origin is needed only to produce the optional routing hostname.
|
|
if helm template push deploy/charts/buzz-push-gateway \
|
|
--set httpRoute.enabled=true --set 'httpRoute.parentRefs[0].name=gateway' >/dev/null 2>&1; then
|
|
echo 'expected enabled HTTPRoute without gatewayOrigin to fail' >&2
|
|
exit 1
|
|
fi
|
|
for invalid_origin in 'http://push.example' 'https://push.example:8443' 'https://push.example/base'; do
|
|
if helm template push deploy/charts/buzz-push-gateway \
|
|
--set "gatewayOrigin=$invalid_origin" >/dev/null 2>&1; then
|
|
echo "expected malformed gatewayOrigin $invalid_origin to fail" >&2
|
|
exit 1
|
|
fi
|
|
done
|
|
|
|
# Legacy token-auth values must fail rather than silently selecting the default
|
|
# certificate Secret.
|
|
if helm template push deploy/charts/buzz-push-gateway \
|
|
"${gateway_origin_arg[@]}" \
|
|
--set apnsKey.secretName=legacy-apns-secret \
|
|
--set apnsKey.secretKey=legacy-provider.p8 >/dev/null 2>&1; then
|
|
echo 'expected legacy apnsKey values to fail schema validation' >&2
|
|
exit 1
|
|
fi
|
|
|
|
# Enabling a route without a Gateway attachment must fail schema validation.
|
|
if helm template push deploy/charts/buzz-push-gateway "${gateway_origin_arg[@]}" --set httpRoute.enabled=true >/dev/null 2>&1; then
|
|
echo 'expected httpRoute.enabled=true without parentRefs to fail' >&2
|
|
exit 1
|
|
fi
|
|
|
|
# The checked-in production contract is intentionally undeployable until CI or
|
|
# the release system supplies its environment-owned values.
|
|
if helm template push deploy/charts/buzz-push-gateway -f deploy/charts/buzz-push-gateway/values-production.yaml >/dev/null 2>&1; then
|
|
echo 'expected uninjected production values to fail' >&2
|
|
exit 1
|
|
fi
|
|
|
|
# Enabling observability renders the scrape CRDs and adds a scoped 8081 ingress
|
|
# keyed to the named monitoring source — never a blanket 8081 rule.
|
|
monitoring_out=$(mktemp)
|
|
helm template push deploy/charts/buzz-push-gateway \
|
|
"${gateway_origin_arg[@]}" \
|
|
--set podMonitor.enabled=true \
|
|
--set prometheusRule.enabled=true \
|
|
--set networkPolicy.monitoring.enabled=true \
|
|
--set 'networkPolicy.monitoring.namespaceSelector.kubernetes\.io/metadata\.name=monitoring' \
|
|
--set 'networkPolicy.monitoring.podSelector.app\.kubernetes\.io/name=prometheus' \
|
|
>"$monitoring_out"
|
|
|
|
env -u GEM_HOME -u GEM_PATH -u RUBYLIB -u RUBYOPT ruby -ryaml -rset \
|
|
- "$monitoring_out" <<'RUBY'
|
|
def assert!(condition, detail = "assertion failed")
|
|
raise detail unless condition
|
|
end
|
|
|
|
xs = YAML.load_stream(File.read(ARGV[0])).compact
|
|
pm = xs.find { |x| x["kind"] == "PodMonitor" }
|
|
endpoint = pm.dig("spec", "podMetricsEndpoints", 0)
|
|
assert!(endpoint["port"] == "health" && endpoint["path"] == "/metrics", endpoint.inspect)
|
|
assert!(!xs.find { |x| x["kind"] == "PrometheusRule" }.dig("spec", "groups").empty?)
|
|
np = xs.find do |x|
|
|
x["kind"] == "NetworkPolicy" && x.dig("metadata", "name") == "push-buzz-push-gateway"
|
|
end
|
|
monitoring = np.dig("spec", "ingress").select do |rule|
|
|
rule.fetch("ports", []).map { |port| port["port"] }.to_set == Set[8081]
|
|
end
|
|
assert!(monitoring.length == 1, "exactly one scoped 8081 ingress rule")
|
|
from = monitoring[0].fetch("from")[0]
|
|
# 8081 ingress must be scoped by both selectors, never empty/blanket.
|
|
assert!(!from.dig("namespaceSelector", "matchLabels").empty? && !from.dig("podSelector", "matchLabels").empty?, from.inspect)
|
|
RUBY
|
|
|
|
# Datadog discovers the same private endpoint from pod annotations and needs no
|
|
# prometheus-operator CRDs. Its agent ingress remains selector-scoped.
|
|
helm lint deploy/charts/buzz-push-gateway \
|
|
-f deploy/charts/buzz-push-gateway/tests/datadog-values.yaml \
|
|
"${gateway_origin_arg[@]}" >/dev/null
|
|
helm template push deploy/charts/buzz-push-gateway \
|
|
-f deploy/charts/buzz-push-gateway/tests/datadog-values.yaml \
|
|
"${gateway_origin_arg[@]}" \
|
|
>"$datadog_out"
|
|
|
|
env -u GEM_HOME -u GEM_PATH -u RUBYLIB -u RUBYOPT ruby -rjson -ryaml -rset \
|
|
- "$datadog_out" <<'RUBY'
|
|
def assert!(condition, detail = "assertion failed")
|
|
raise detail unless condition
|
|
end
|
|
|
|
xs = YAML.load_stream(File.read(ARGV[0])).compact
|
|
assert!(!xs.any? { |x| %w[PodMonitor PrometheusRule].include?(x["kind"]) })
|
|
deployment = xs.find { |x| x["kind"] == "Deployment" }
|
|
raw_check = deployment.dig(
|
|
"spec", "template", "metadata", "annotations",
|
|
"ad.datadoghq.com/gateway.checks",
|
|
)
|
|
check = JSON.parse(raw_check)
|
|
instance = check.dig("openmetrics", "instances", 0)
|
|
assert!(instance["openmetrics_endpoint"] == "http://%%host%%:8081/metrics", instance.inspect)
|
|
assert!(instance["metrics"] == ["push_gateway_.*"], instance.inspect)
|
|
|
|
np = xs.find do |x|
|
|
x["kind"] == "NetworkPolicy" && x.dig("metadata", "name") == "push-buzz-push-gateway"
|
|
end
|
|
monitoring = np.dig("spec", "ingress").select do |rule|
|
|
rule.fetch("ports", []).map { |port| port["port"] }.to_set == Set[8081]
|
|
end
|
|
assert!(monitoring.length == 1, "exactly one scoped Datadog 8081 ingress rule")
|
|
from = monitoring[0].fetch("from")[0]
|
|
assert!(!from.dig("namespaceSelector", "matchLabels").empty?, from.inspect)
|
|
assert!(!from.dig("podSelector", "matchLabels").empty?, from.inspect)
|
|
RUBY
|
|
|
|
# Negative: monitoring enabled with default empty selectors must fail (would
|
|
# otherwise render a blanket 8081 rule matching all namespaces/pods).
|
|
if helm template push deploy/charts/buzz-push-gateway \
|
|
"${gateway_origin_arg[@]}" \
|
|
--set podMonitor.enabled=true \
|
|
--set networkPolicy.monitoring.enabled=true >/dev/null 2>&1; then
|
|
echo 'expected monitoring.enabled with empty selectors to fail' >&2
|
|
exit 1
|
|
fi
|
|
|
|
# Negative: PodMonitor without ingress is an unreachable scraper and must fail.
|
|
# Scoped ingress without PodMonitor is valid for annotation-discovered agents.
|
|
if helm template push deploy/charts/buzz-push-gateway \
|
|
"${gateway_origin_arg[@]}" \
|
|
--set podMonitor.enabled=true \
|
|
--set 'networkPolicy.monitoring.namespaceSelector.kubernetes\.io/metadata\.name=monitoring' \
|
|
--set 'networkPolicy.monitoring.podSelector.app\.kubernetes\.io/name=prometheus' \
|
|
>/dev/null 2>&1; then
|
|
echo 'expected podMonitor.enabled without monitoring ingress to fail' >&2
|
|
exit 1
|
|
fi
|
|
|
|
# Negative: retry-ratio threshold is a fraction; a value > 1 must fail schema.
|
|
if helm template push deploy/charts/buzz-push-gateway \
|
|
"${gateway_origin_arg[@]}" \
|
|
--set prometheusRule.enabled=true \
|
|
--set prometheusRule.apnsRetryRatioThreshold=2 >/dev/null 2>&1; then
|
|
echo 'expected apnsRetryRatioThreshold=2 to fail' >&2
|
|
exit 1
|
|
fi
|