Benchmarks
Reusable test suites for your agents. Change the instructions, tools, skills, model or runtime, run the benchmark, and see what improved and what regressed before it ships.
v1v2v3Draft
Versioned cases
Fixed cases saved as immutable versions, so a result today means the same as one next month.
- Output
- Behavior
- Files
- Commands
- Model judge
Checks for every outcome
Grade the output, the agent's behaviour, the files it changed and the commands it ran, or have a model judge.
Claude Opus 5.592%
GPT-6 Sol86%
Kimi K378%
Qwen3.671%
Compare models side by side
Run up to ten model configurations on the same cases and see pass rate and cost for each.
BuildBenchmarkDeploy
Gate your deploys
Everything is in the API, so a Benchmark can stand between a change and production.