Files
buzz/desktop/tests/e2e/harness-catalog-screenshots.spec.ts
Taylor Ho d98da7389e feat(desktop): redesign agent runtime settings (#3093)
**Category:** improvement
**User Impact:** Users can understand, install, authenticate, and manage
agent runtimes from one progressively disclosed Agents settings
experience.

**Problem:** Runtime health and custom harness management were split
across overlapping settings surfaces, exposing low-level configuration
too early while leaving setup and authentication states hard to
understand. **Solution:** Consolidate those operations into one stable
runtime list and an Add runtimes catalog, with task-oriented state
labels, direct setup actions, and custom configuration contained in a
dedicated form.

<details>
<summary>File changes</summary>

**desktop/playwright.config.ts**
Registers the visual coverage needed for the redesigned runtime catalog.

**desktop/public/harness-logos/CREDITS.md**
Documents bundled runtime-mark provenance and the decision not to ship
the withdrawn OpenAI mark.

**desktop/public/runtime-icons/codex.png**
Removes the obsolete Codex bitmap in favor of the neutral fallback.

**desktop/public/runtime-icons/goose.svg**
Removes the old Goose asset now replaced by the theme-adaptive mark.

**desktop/src-tauri/src/managed_agents/discovery.rs**
Aligns runtime discovery guidance with the new task-oriented setup
language.

**desktop/src-tauri/src/managed_agents/discovery/runtime_metadata.rs**
Updates runtime metadata used by the redesigned settings states.

**desktop/src/features/agents/ui/runtimeAvailabilityWarning.test.mjs**
Updates availability-warning expectations for the revised runtime
guidance.

**desktop/src/features/onboarding/assets/harness-logos/chatgpt.png**
Removes the redundant bitmap from the unified runtime icon pipeline.

**desktop/src/features/onboarding/assets/harness-logos/goose.png**
Removes the redundant Goose bitmap.

**desktop/src/features/onboarding/ui/HarnessMarks.tsx**
Adds theme-adaptive bundled runtime marks with safe fallbacks.

**desktop/src/features/onboarding/ui/RuntimeIcon.tsx**
Centralizes runtime logo rendering so settings and catalog rows cannot
drift.

**desktop/src/features/onboarding/ui/SetupStep.tsx**
Aligns onboarding runtime setup copy with the settings terminology.

**desktop/src/features/onboarding/ui/presetLogos.test.mjs**
Guards bundled-logo behavior and prevents the withdrawn Codex mark from
returning.

**desktop/src/features/settings/ui/CustomHarnessForm.tsx**
Reworks custom runtime creation and editing into a clear, dedicated
catalog form.

**desktop/src/features/settings/ui/HarnessCatalogDialog.tsx**
Introduces the Add runtimes master-detail catalog, grouped setup states,
loading treatment, and pinned actions.

**desktop/src/features/settings/ui/HarnessManagementCard.tsx**
Removes the superseded standalone custom-harness management surface.

**desktop/src/features/settings/ui/HarnessRow.tsx**
Provides stable operational runtime rows with install, update,
authentication, edit, and delete behavior.

**desktop/src/features/settings/ui/HarnessesSettingsPanel.tsx**
Consolidates runtime health and custom management into one Agents
settings panel.

**desktop/src/features/settings/ui/SettingsPanels.tsx**
Wires the consolidated panel into Agents settings.

**desktop/src/features/settings/ui/harnessCatalogCopy.ts**
Adds restrained, source-annotated runtime descriptions and setup
guidance.

**desktop/src/features/settings/ui/harnessCatalogLogic.test.mjs**
Covers grouping, state labels, stable row order, actions, and adapter
warnings.

**desktop/src/features/settings/ui/harnessCatalogLogic.ts**
Centralizes catalog grouping, actions, status labels, and runtime-safe
warning copy.

**desktop/src/features/settings/ui/harnessGalleryLogic.test.mjs**
Removes obsolete gallery-only tests after consolidation.

**desktop/src/features/settings/ui/harnessGalleryLogic.ts**
Retains only the shared custom-runtime safety logic needed by the new
surface.

**desktop/src/shared/ui/config-nudge-attachment.tsx**
Points configuration nudges to Agent runtimes with matching terminology.

**desktop/src/testing/e2eBridge.ts**
Adds deterministic runtime states for authentication and catalog E2E
coverage.

**desktop/tests/e2e/doctor-states.spec.ts**
Verifies ready, setup-required, node-gated, authentication, loading, and
error contracts.

**desktop/tests/e2e/harness-catalog-screenshots.spec.ts**
Captures whole-pane visual states for the runtime catalog experience.

**desktop/tests/e2e/harness-management.spec.ts**
Exercises catalog actions and custom runtime create, edit,
authentication, and deletion flows.

**desktop/tests/e2e/onboarding-agent-defaults.spec.ts**
Updates default-runtime expectations for the consolidated experience.

**desktop/tests/e2e/profile.spec.ts**
Aligns profile navigation assertions with the new settings surface.

</details>

## Reproduction steps

1. Open Settings → Agents and inspect Agent runtimes; ready, signed-out,
installable, and setup-required runtimes should have stable rows and
explicit actions.
2. Open Add runtimes and browse the Setup and Installed groups; select
entries to see sourced guidance and a pinned Install or Setup guide
action.
3. Select Custom harness, create a runtime, then edit and delete it;
verify required-field gating and the blast-radius confirmation.
4. Exercise a signed-out runtime and connect it; the row should move
from Sign-in needed to Ready without reordering.
5. Resize the window and switch themes to verify responsive layout and
adaptive bundled marks.

## Screenshots

| Mixed runtime states | Goose not installed | Sign-in needed |
|---|---|---|
| <img width="980" height="1000" alt="image"
src="https://github.com/user-attachments/assets/28705b39-1d91-440a-a954-fb2d7d8ee759"
/> | <img width="980" height="1000" alt="image"
src="https://github.com/user-attachments/assets/e65ce8e1-8c85-4272-afb5-e6001b94d560"
/> | <img width="980" height="1000" alt="image"
src="https://github.com/user-attachments/assets/3987f355-960d-4744-8c26-b3746944a33c"
/> |

| Add runtimes catalog | Custom runtime | Setup guide |
|---|---|---|
| <img width="896" height="672" alt="image"
src="https://github.com/user-attachments/assets/a2f32c8a-4dff-4929-af96-b07346933335"
/> | <img width="896" height="672" alt="image"
src="https://github.com/user-attachments/assets/b2e85a0c-3940-4875-a138-4035423c9de7"
/> | <img width="896" height="672" alt="image"
src="https://github.com/user-attachments/assets/5a690c1c-368e-4ee3-98b9-ffefdc3cf804"
/> |

Full 13-state screenshot matrix and review evidence:
https://buzz.block.builderlab.xyz/channels/c5af9e3e-4317-4853-b3e3-ed9c15bc511d?event=ec88f6e472f2b08d6318ae76fd754fd4f218385f67cfe59b73034c4bc34c9252

---------

Signed-off-by: Taylor Ho <taylorkmho@gmail.com>
Signed-off-by: npub1223z34hd7vtwc6qj4s7flsxkj644nlre2nthu7lrrmkumhu3xddsrx9r6w <52a228d6edf316ec6812ac3c9fc0d696ab59fc7954d77e7be31eedcddf91335b@buzz.block.builderlab.xyz>
Co-authored-by: npub1223z34hd7vtwc6qj4s7flsxkj644nlre2nthu7lrrmkumhu3xddsrx9r6w <52a228d6edf316ec6812ac3c9fc0d696ab59fc7954d77e7be31eedcddf91335b@buzz.block.builderlab.xyz>
2026-07-27 11:01:42 -07:00

180 lines
5.5 KiB
TypeScript

import { test, expect } from "@playwright/test";
import { installMockBridge } from "../helpers/bridge";
import { openSettings } from "../helpers/settings";
const SHOTS = "test-results/byoh-after";
/**
* AFTER screenshots for the consolidated Harnesses surface — same catalog
* fixture as byoh-before-screenshots.spec.ts so the before/after pair is a
* true apples-to-apples comparison.
*/
const CATALOG = [
{
id: "buzz-agent",
label: "Buzz Agent",
avatar_url: "",
availability: "available",
command: "buzz-agent",
binary_path: "/usr/local/bin/buzz-agent",
default_args: [],
mcp_command: "buzz-dev-mcp",
install_hint: "",
install_instructions_url: "https://github.com/block/buzz",
can_auto_install: false,
underlying_cli_path: null,
node_required: false,
auth_status: { status: "not_applicable" },
},
{
id: "claude",
label: "Claude Code",
avatar_url: "",
availability: "available",
command: "claude-agent-acp",
binary_path: "/usr/local/bin/claude-agent-acp",
default_args: [],
mcp_command: null,
install_hint: "",
install_instructions_url:
"https://github.com/agentclientprotocol/claude-agent-acp",
can_auto_install: true,
underlying_cli_path: "/usr/local/bin/claude",
node_required: false,
auth_status: { status: "logged_in" },
},
{
id: "codex",
label: "Codex",
avatar_url: "",
availability: "adapter_missing",
command: null,
binary_path: null,
default_args: [],
mcp_command: null,
install_hint: "Install the Codex ACP adapter via npm.",
install_instructions_url:
"https://github.com/agentclientprotocol/codex-acp",
can_auto_install: true,
underlying_cli_path: "/usr/local/bin/codex",
node_required: false,
auth_status: { status: "unknown" },
},
{
id: "cursor",
label: "Cursor",
avatar_url: "",
availability: "available",
command: "cursor-agent",
binary_path: "/usr/local/bin/cursor-agent",
default_args: ["acp"],
mcp_command: null,
install_hint:
"Buzz talks to Cursor through the cursor-agent CLI's ACP mode.",
install_instructions_url: "https://cursor.com/downloads",
can_auto_install: false,
underlying_cli_path: null,
node_required: false,
auth_status: { status: "not_applicable" },
source: "preset",
},
{
id: "omp",
label: "Oh My Pi",
avatar_url: "",
availability: "not_installed",
command: null,
binary_path: null,
default_args: ["acp"],
mcp_command: null,
install_hint:
"Buzz talks to Oh My Pi through its CLI's ACP mode (omp acp).",
install_instructions_url: "https://github.com/can1357/oh-my-pi",
can_auto_install: false,
underlying_cli_path: null,
node_required: false,
auth_status: { status: "not_applicable" },
source: "preset",
},
{
id: "opencode",
label: "OpenCode",
avatar_url: "",
availability: "not_installed",
command: null,
binary_path: null,
default_args: ["acp"],
mcp_command: null,
install_hint:
"Buzz talks to OpenCode through its CLI's ACP mode (opencode acp).",
install_instructions_url: "https://opencode.ai/docs",
can_auto_install: false,
underlying_cli_path: null,
node_required: false,
auth_status: { status: "not_applicable" },
source: "preset",
},
{
id: "my-custom",
label: "My Custom Harness",
avatar_url: "",
availability: "available",
command: "my-acp",
binary_path: "/usr/local/bin/my-acp",
default_args: [],
mcp_command: null,
install_hint: "",
install_instructions_url: "",
can_auto_install: false,
underlying_cli_path: null,
node_required: false,
auth_status: { status: "not_applicable" },
source: "custom",
},
];
test("after: consolidated harnesses panel + catalog dialog", async ({
page,
}) => {
test.setTimeout(60_000);
await page.setViewportSize({ width: 1280, height: 2400 });
await installMockBridge(page, { acpRuntimesCatalog: CATALOG });
await page.goto("/", { waitUntil: "domcontentloaded" });
await openSettings(page, "agents");
await expect(page.getByTestId("settings-harnesses")).toBeVisible({
timeout: 10_000,
});
await page.waitForTimeout(700);
// 1. The consolidated panel — ready rows only, no inert controls.
await page.getByTestId("settings-harnesses").screenshot({
path: `${SHOTS}/after-harnesses-panel.png`,
});
// 2. Add-harnesses catalog — needs-setup entry auto-selected in detail.
await page.getByTestId("harness-add-button").click();
await expect(page.getByTestId("harness-catalog-dialog")).toBeVisible();
await page.waitForTimeout(500);
await page.getByTestId("harness-catalog-dialog").screenshot({
path: `${SHOTS}/after-catalog-needs-setup.png`,
});
// 3. Ready entry detail (Ready state, no install action). Ready entries
// sit in the "Installed" accordion, collapsed by default — expand it.
await page.getByTestId("harness-catalog-section-installed").click();
await page.getByTestId("harness-catalog-list-item-claude").click();
await page.waitForTimeout(300);
await page.getByTestId("harness-catalog-dialog").screenshot({
path: `${SHOTS}/after-catalog-ready-detail.png`,
});
// 4. Custom harness form — all fields inline.
await page.getByTestId("harness-catalog-list-item-custom").click();
await expect(page.getByTestId("custom-harness-form")).toBeVisible();
await page.waitForTimeout(300);
await page.getByTestId("harness-catalog-dialog").screenshot({
path: `${SHOTS}/after-catalog-custom-form.png`,
});
});