jerry/buzz
1
0
Fork 0
mirror of https://github.com/block/buzz.git synced 2026-10-27 04:30:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

test: fix recurring CI flakes in desktop and Postgres lanes (#8343)

Browse Source 
Fixes the CI flakes that failed most often across `main` and unrelated
branches from Oct 5 to Oct 14. The counts come from 150 failed `CI`
runs. Every change except the nextest config is test-only.

| Flake | Runs hit | Root cause | Fix |
|---|---|---|---|
| `probe_codex_acp_version_…descendant_holds_pipe_open` | 5 | The test
writes its script in-process and then execs it. If a sibling test spawns
a process in that window, the child inherits the open write fd and Linux
rejects the exec with `ETXTBSY`. The probe swallows that error as
`None`. | Write probe scripts from a short-lived `/bin/sh`
(`write_executable_script`) |
| `empty-edit-delete.spec.ts` (`:51`, `:74`, `:95`) | 11 | The seeded
`mock-general-welcome` id isn't hex, so `formatTimelineMessages` can't
overlay its edit. The post-subscribe window refetch then reverts the
row. | Emit a per-test message with a real id after the exact channel
subscription is live |
| `huddle-transcription.spec.ts:798` | 9 | The composer's delayed
initial autofocus (TipTap `focus()` runs in rAF) lands after the voice
popover opens and dismisses it. | Wait for `message-input` focus before
opening the popover |
| `voice-note.spec.ts:906` | 6 | The fixture `ping.mp3` is about 1s
long, so playback can end before `paused === false` is observed. | Loop
the fixture audio |
| `workflows.spec.ts:291` | 5 | The `Message text` fill runs before the
inspector swaps to the new step, so it fills the trigger's condition
instead. | Scope the fill to the step inspector after its `Action` menu
appears |
| `cluster_global_probe_commits_tokens_and_sessions_prove_coverage` | 3
| The `max-threads = 1` group only serializes these tests with each
other. Other tests' sessions still mask cluster-wide `pg_stat_activity`
(`MaskedActivity`). | `threads-required = "num-test-threads"` |
| `test_usage_metrics_lock_has_single_owner_and_releases_on_drop` | 1
(this PR) | The test retries the lock right after dropping the leader.
Postgres releases a session advisory lock only after the backend notices
the disconnect, so the retry can run first. | Poll for the successor,
bounded at 5s |

### Validation
- **ETXTBSY:** a Linux experiment with Rust 1.95 using the same
write→chmod→spawn pattern under concurrent spawns hit `ETXTBSY` 4–7% of
the time. Using the `sh` writer, it hit 0 of 6,000. I did not catch it
inside the full suite locally, so this fix is based on the mechanism,
not an in-suite repro. All 11 probe tests pass on Linux and macOS.
- **empty-edit-delete:** the baseline failed 16 of 45 runs
(`--repeat-each`, 6 workers, no retries). The spec fix alone brings that
to 4/45. The remaining failures come from a separate product race,
described below.
- **huddle:** the baseline failed 5 of 10 runs. With the fix, 60 of 60
passed without retries.
- **nextest:** checked with a throwaway crate on 0.9.136. A grouped test
with `num-test-threads` ran alone, even with `max-threads = 1`.
- I have not stress-run the voice-note and workflows fixes; they rely on
CI.

### Not in this PR
- **Product bug in `useChannelLinks`:** the debounced `#channel`
autocomplete close lets Enter accept a stale suggestion for about 120ms.
That accounts for the remaining `empty-edit-delete` failures, and it
affects users too. With the spec fix plus a synchronous close, the spec
passed 120 of 120 runs. This needs a product decision, so it's held for
a separate PR.
- **Composer autofocus race:** the delayed autofocus can close an
overlay the user just opened. The real fix is to revalidate when the
delayed focus runs. Overlaps #6530.
- **`thread-head-stale-edit.spec.ts:55`:** a forced click on a
pointer-events-none action bar is suspected, but I couldn't reproduce
it.
- **Remaining ETXTBSY exposure:** about 40 other Rust tests write and
then exec scripts, and 17 specs still edit the non-hex seed message.

Root-cause lanes by Princess Donut, Mongo and Mordecai. Carl integrated
and verified them.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Wes <wesbillman@users.noreply.github.com>
Co-authored-by: Carl <32a2e2c9d428ee08902cab75d956da2c1d235a22d4766b0dd4138bf6e2e5db1d@buzz.block.builderlab.xyz>
Co-authored-by: Princess Donut <5d97ac8c272fa949af56c71e586146b1706e5c4c3daa125db5b4625e12e0686c@buzz.block.builderlab.xyz>
Co-authored-by: Mongo <81cabd2ca1792494372ed410283c00770c0cf35afca052e940fa522883212b45@buzz.block.builderlab.xyz>
Co-authored-by: Mordecai <f314b6033b6e13df94c8e9e1187d3666b6b9ce54e66ee986e31cdaa909e8dcbc@buzz.block.builderlab.xyz>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Wes
2026-10-14 19:32:00 -07:00
committed by GitHub
parent 8222e99915
commit 7b11c6147d
8 changed files with 113 additions and 60 deletions
Download Patch File Download Diff File Expand all files Collapse all files

68
desktop/src-tauri/src/managed_agents/discovery/tests.rs
View File

@@ -612,21 +612,36 @@ fn update_time_override_preserves_pin_for_persona_less_agent() {
mod effort_clear;
mod forced_discovery;
mod managed_path_resolution;
/// Write an executable script without opening it for writing in this process.
///
/// Linux refuses to exec a file while any process holds it open for writing
/// (`ETXTBSY`). A sibling test that spawns between our open and close copies
/// that descriptor into its child until the child execs, so an in-process
/// `fs::write` can make the probe's spawn fail intermittently. A short-lived
/// shell writes the file instead, and its descriptor is gone once it exits.
#[cfg(unix)]
fn write_executable_script(path: &std::path::Path, contents: impl AsRef<str>) {
let status = std::process::Command::new("/bin/sh")
.args(["-c", r#"printf '%s' "$2" > "$1" && chmod 755 "$1""#, "sh"])
.arg(path)
.arg(contents.as_ref())
.status()
.expect("spawn script writer");
assert!(status.success(), "write script {}", path.display());
}
#[cfg(unix)]
#[test]
fn probe_codex_acp_version_parses_full_semver_output() {
use std::os::unix::fs::PermissionsExt;
// Simulate a current `@agentclientprotocol/codex-acp` output.
let dir = std::env::temp_dir().join(format!("buzz-probe-1x-{}", uuid::Uuid::new_v4()));
std::fs::create_dir_all(&dir).expect("create temp dir");
let bin = dir.join("codex-acp");
std::fs::write(
write_executable_script(
&bin,
"#!/bin/sh\necho '@agentclientprotocol/codex-acp 1.1.7'\nexit 0\n",
)
.expect("write script");
std::fs::set_permissions(&bin, std::fs::Permissions::from_mode(0o755)).expect("chmod script");
);
let version = probe_codex_acp_version(&bin);
let _ = std::fs::remove_dir_all(dir);
@@ -643,14 +658,11 @@ mod codex_version;
#[cfg(unix)]
#[test]
fn probe_codex_acp_version_returns_none_for_nonzero_exit() {
use std::os::unix::fs::PermissionsExt;
// Simulate old 0.16.x adapter: `--version` is unrecognised, exits non-zero
let dir = std::env::temp_dir().join(format!("buzz-probe-0x-{}", uuid::Uuid::new_v4()));
std::fs::create_dir_all(&dir).expect("create temp dir");
let bin = dir.join("codex-acp");
std::fs::write(&bin, "#!/bin/sh\nexit 1\n").expect("write script");
std::fs::set_permissions(&bin, std::fs::Permissions::from_mode(0o755)).expect("chmod script");
write_executable_script(&bin, "#!/bin/sh\nexit 1\n");
let version = probe_codex_acp_version(&bin);
let _ = std::fs::remove_dir_all(dir);
@@ -677,17 +689,13 @@ fn probe_codex_acp_version_returns_none_for_missing_binary() {
#[cfg(unix)]
#[test]
fn codex_adapter_availability_available_for_minimum_supported_binary() {
use std::os::unix::fs::PermissionsExt;
let dir = std::env::temp_dir().join(format!("buzz-avail-1x-{}", uuid::Uuid::new_v4()));
std::fs::create_dir_all(&dir).expect("create temp dir");
let bin = dir.join("codex-acp");
std::fs::write(
write_executable_script(
&bin,
"#!/bin/sh\necho '@agentclientprotocol/codex-acp 1.10.0'\nexit 0\n",
)
.expect("write script");
std::fs::set_permissions(&bin, std::fs::Permissions::from_mode(0o755)).expect("chmod script");
);
let status = codex_adapter_availability(&bin);
let _ = std::fs::remove_dir_all(dir);
@@ -702,14 +710,11 @@ fn codex_adapter_availability_available_for_minimum_supported_binary() {
#[cfg(unix)]
#[test]
fn codex_adapter_availability_outdated_for_0x_binary() {
use std::os::unix::fs::PermissionsExt;
// Simulate old 0.16.x: `--version` exits non-zero (unrecognised flag)
let dir = std::env::temp_dir().join(format!("buzz-avail-0x-{}", uuid::Uuid::new_v4()));
std::fs::create_dir_all(&dir).expect("create temp dir");
let bin = dir.join("codex-acp");
std::fs::write(&bin, "#!/bin/sh\nexit 1\n").expect("write script");
std::fs::set_permissions(&bin, std::fs::Permissions::from_mode(0o755)).expect("chmod script");
write_executable_script(&bin, "#!/bin/sh\nexit 1\n");
let status = codex_adapter_availability(&bin);
let _ = std::fs::remove_dir_all(dir);
@@ -726,18 +731,13 @@ fn codex_adapter_availability_outdated_for_0x_binary() {
#[cfg(unix)]
#[test]
fn codex_adapter_availability_outdated_for_uncomparable_version() {
use std::os::unix::fs::PermissionsExt;
for version in ["1.2", "1.2.0-rc1"] {
let dir = tempfile::tempdir().expect("temp dir");
let bin = dir.path().join("codex-acp");
std::fs::write(
write_executable_script(
&bin,
format!("#!/bin/sh\necho '@agentclientprotocol/codex-acp {version}'\nexit 0\n"),
)
.expect("write script");
std::fs::set_permissions(&bin, std::fs::Permissions::from_mode(0o755))
.expect("chmod script");
);
assert_eq!(
codex_adapter_availability(&bin),
@@ -766,7 +766,6 @@ fn codex_adapter_availability_outdated_for_missing_binary() {
#[cfg(unix)]
#[test]
fn probe_codex_acp_version_returns_none_for_hung_direct_child() {
use std::os::unix::fs::PermissionsExt;
use std::time::Instant;
// Simulate a process that writes version to stdout then blocks forever.
@@ -775,12 +774,10 @@ fn probe_codex_acp_version_returns_none_for_hung_direct_child() {
let dir = std::env::temp_dir().join(format!("buzz-probe-hung-{}", uuid::Uuid::new_v4()));
std::fs::create_dir_all(&dir).expect("create temp dir");
let bin = dir.join("codex-acp");
std::fs::write(
write_executable_script(
&bin,
"#!/bin/sh\nprintf '@agentclientprotocol/codex-acp 1.1.2\\n'\nexec sleep 300\n",
)
.expect("write script");
std::fs::set_permissions(&bin, std::fs::Permissions::from_mode(0o755)).expect("chmod script");
);
let start = Instant::now();
let version = probe_codex_acp_version(&bin);
@@ -801,7 +798,6 @@ fn probe_codex_acp_version_returns_none_for_hung_direct_child() {
#[cfg(unix)]
#[test]
fn probe_codex_acp_version_returns_version_when_descendant_holds_pipe_open() {
use std::os::unix::fs::PermissionsExt;
use std::time::Instant;
// Simulate a process that forks a background child which inherits stdout
@@ -817,12 +813,10 @@ fn probe_codex_acp_version_returns_version_when_descendant_holds_pipe_open() {
let dir = std::env::temp_dir().join(format!("buzz-probe-descendant-{}", uuid::Uuid::new_v4()));
std::fs::create_dir_all(&dir).expect("create temp dir");
let bin = dir.join("codex-acp");
std::fs::write(
write_executable_script(
&bin,
"#!/bin/sh\necho '@agentclientprotocol/codex-acp 1.1.2'\nsleep 60 &\nexit 0\n",
)
.expect("write script");
std::fs::set_permissions(&bin, std::fs::Permissions::from_mode(0o755)).expect("chmod script");
);
let start = Instant::now();
let version = probe_codex_acp_version(&bin);

21
desktop/src-tauri/src/managed_agents/discovery/tests/codex_version.rs
View File

@@ -1,10 +1,11 @@
use super::super::probe_codex_acp_version_with_path;
#[cfg(unix)]
use super::write_executable_script;
#[cfg(unix)]
#[test]
fn probe_codex_acp_version_uses_augmented_path_for_env_shebang_interpreter() {
use std::fs;
use std::os::unix::fs::PermissionsExt;
let temp = tempfile::tempdir().expect("temp dir");
let script_dir = temp.path().join("script-bin");
let interpreter_dir = temp.path().join("interpreter-bin");
@@ -14,17 +15,13 @@ fn probe_codex_acp_version_uses_augmented_path_for_env_shebang_interpreter() {
fs::create_dir_all(&empty_path_dir).expect("empty path dir");
let interpreter_path = interpreter_dir.join("node");
fs::write(
write_executable_script(
&interpreter_path,
"#!/bin/sh\necho '@agentclientprotocol/codex-acp 1.1.2'\n",
)
.expect("write interpreter");
fs::set_permissions(&interpreter_path, fs::Permissions::from_mode(0o755))
.expect("chmod interpreter");
);
let shim_path = script_dir.join("codex-acp");
fs::write(&shim_path, "#!/usr/bin/env node\n").expect("write shim");
fs::set_permissions(&shim_path, fs::Permissions::from_mode(0o755)).expect("chmod shim");
write_executable_script(&shim_path, "#!/usr/bin/env node\n");
let scrubbed_path = std::env::join_paths([empty_path_dir.as_path()])
.expect("join scrubbed PATH")
@@ -52,18 +49,14 @@ fn probe_codex_acp_version_uses_augmented_path_for_env_shebang_interpreter() {
fn codex_adapter_availability_outdated_for_older_1x_binary() {
use super::super::{codex_adapter_availability, codex_adapter_is_outdated};
use crate::managed_agents::AcpAvailabilityStatus;
use std::os::unix::fs::PermissionsExt;
for version in ["1.1.5", "1.1.7", "1.6.2", "1.9.0"] {
let dir = tempfile::tempdir().expect("temp dir");
let bin = dir.path().join("codex-acp");
std::fs::write(
write_executable_script(
&bin,
format!("#!/bin/sh\necho '@agentclientprotocol/codex-acp {version}'\nexit 0\n"),
)
.expect("write script");
std::fs::set_permissions(&bin, std::fs::Permissions::from_mode(0o755))
.expect("chmod script");
);
assert_eq!(
codex_adapter_availability(&bin),

45
desktop/tests/e2e/empty-edit-delete.spec.ts
View File

@@ -2,11 +2,14 @@ import { expect, test } from "@playwright/test";
import { installMockBridge } from "../helpers/bridge";
// The mock identity's own pre-seeded message in #general (authored by
// DEFAULT_MOCK_IDENTITY.pubkey in e2eBridge.ts). Editing/deleting one's own
// A message the mock identity authors in #general. Editing/deleting one's own
// message is exactly Sam's workflow: "delete a message by clearing its edit."
const OWN_MESSAGE_ID = "mock-general-welcome";
const RENDERED_ORIGINAL_CONTENT = "Welcome to general";
// It is emitted per test rather than reusing the seeded `mock-general-welcome`:
// edits and deletions only overlay a target with a real 64-hex event id, so
// that seed keeps its edit only until the post-subscribe window refetch.
const ORIGINAL_CONTENT = "Ship notes for #general";
const RENDERED_ORIGINAL_CONTENT = "Ship notes for general";
let OWN_MESSAGE_ID: string;
// Open the more-actions menu for a message row and wait for the menu to mount.
async function openMoreActionsMenu(
@@ -46,6 +49,40 @@ test.beforeEach(async ({ page }) => {
await page.goto("/");
await page.getByTestId("channel-general").click();
await expect(page.getByTestId("chat-title")).toHaveText("general");
await expect
.poll(() =>
page.evaluate(
() =>
(
window as Window & {
__BUZZ_E2E_HAS_MOCK_LIVE_SUBSCRIPTION__?: (input: {
channelName: string;
kind: number;
exactChannel: boolean;
}) => boolean;
}
).__BUZZ_E2E_HAS_MOCK_LIVE_SUBSCRIPTION__?.({
channelName: "general",
kind: 9,
exactChannel: true,
}) ?? false,
),
)
.toBe(true);
const emitted = await page.evaluate(
(content) =>
(
window as Window & {
__BUZZ_E2E_EMIT_MOCK_MESSAGE__?: (input: {
channelName: string;
content: string;
}) => { id: string } | undefined;
}
).__BUZZ_E2E_EMIT_MOCK_MESSAGE__?.({ channelName: "general", content }),
ORIGINAL_CONTENT,
);
if (!emitted?.id) throw new Error("mock message emit did not return an id");
OWN_MESSAGE_ID = emitted.id;
});
test("clearing an edit to empty prompts to delete, then deletes on confirm", async ({

4
desktop/tests/e2e/huddle-transcription.spec.ts
View File

@@ -850,6 +850,10 @@ test("assigns distinct agent voices and exposes compact per-agent controls", asy
});
expect(new Set(assignedVoices).size).toBe(2);
// Let the huddle chat finish mounting and autofocus before opening an
// overlay; a late composer focus would dismiss the popover.
await expect(page.getByTestId("message-input")).toBeFocused();
await page.getByRole("button", { name: "Voice settings for alice" }).click();
await waitForAnimations(page);
const voiceMenu = page.locator(

5
desktop/tests/e2e/voice-note.spec.ts
View File

@@ -943,6 +943,11 @@ test("starting a duplicate voice-note player pauses the other instance", async (
const cards = page.getByTestId("audio-message-attachment");
await expect(cards).toHaveCount(2);
// The ping fixture lasts about one second, so it can finish before the
// playing state is observed. Loop it so only exclusivity pauses a player.
await cards.locator("audio").evaluateAll((audios) => {
for (const audio of audios) (audio as HTMLAudioElement).loop = true;
});
const firstAudio = cards.nth(0).locator("audio");
const secondAudio = cards.nth(1).locator("audio");

8
desktop/tests/e2e/workflows.spec.ts
View File

@@ -96,7 +96,13 @@ async function createWorkflow(
await dialog.getByRole("button", { name: "Add step", exact: true }).click();
await page.getByRole("menuitem", { name: "Send Message" }).click();
await dialog.getByLabel("Message text").fill("Workflow notification");
// The inspector swaps from the trigger to the new step after the click; an
// unscoped fill can land in the trigger's message-text condition instead.
const stepInspector = dialog.getByTestId("workflow-node-inspector");
await expect(
stepInspector.getByRole("button", { name: "Action", exact: true }),
).toBeVisible();
await stepInspector.getByLabel("Message text").fill("Workflow notification");
if (options?.stepName) {
await dialog.getByRole("button", { name: "Step details" }).click();
await dialog.getByLabel("Name (optional)").fill(options.stepName);