Files
buzz/benchmarks/buzz-dataset/cross-thread-requests/task.toml
Salman Mohammed 79df468255 Add Buzz benchmark evaluation layers (#6823)
## Summary

- classify every Buzz-native benchmark task as `regression` or
`workflow`
- add metadata-backed `--layer` selection with k=1 regression and k=3
workflow defaults, while preserving explicit `--attempts`/`-k`
- split an unqualified mixed Buzz dataset run into two Harbor jobs so
each layer gets the correct default
- document the layer contract and cover metadata, selection, defaults,
overrides, invalid input, and command construction

---------

Signed-off-by: Salman Mohammed <smohammed@squareup.com>
2026-08-26 17:41:56 -04:00

26 lines
573 B
TOML

schema_version = "1.3"
[task]
name = "buzz-native/cross-thread-requests"
description = "Keep two concurrent top-level requests in one channel isolated and reply to each exact thread."
authors = [{ name = "Buzz" }]
keywords = ["buzz-native", "threading", "batching", "concurrency"]
[metadata]
evaluation_layer = "workflow"
difficulty = "hard"
category = "collaboration"
tags = ["threading", "batching", "concurrency", "routing"]
[agent]
timeout_sec = 300.0
[verifier]
timeout_sec = 30.0
[environment]
network_mode = "public"
cpus = 1
memory_mb = 1024
storage_mb = 1024