AutomationBench
1.0.6 / Zapier held-out private set ↗- GPT-6 Astra:
- Claude Fable 5:
Compare models
Find the right model, side by side.
Updated: Data as of:
First model
Second model
Model specifications
| Specifications |  GPT-6 AstraOpenAI |  Claude Fable 5Anthropic |
|---|---|---|
| Provider | ||
| Context sizetokens | 1,000,000 | 1,000,000 |
| Cutoff date | 2026-04-30 | Not reported |
| Input costUSD / 1M tokens | $10.00 | $10.00 |
| Output costUSD / 1M tokens | $50.00 | $50.00 |
| Max outputtokens | 128,000 | 128,000 |
| LatencyTTFT · seconds | 365.127 | 117.332 |
| Speedtokens / sec | 70.564 | 68.168 |
Benchmark performance
10 shared benchmark comparisons. Versions and setups matched.
GPT-6 AstraClaude Fable 5
AutomationBench-AA
1.0.6 / AA private 657 ↗Tools enabled- GPT-6 Astra:
- Claude Fable 5:
- GPT-6 Astra:
- Claude Fable 5:
- GPT-6 Astra:
- Claude Fable 5:
DeepSWE
v1.1 ↗- GPT-6 Astra:
- Claude Fable 5:
GDPval-AA
2.1 ↗- GPT-6 Astra:
- Claude Fable 5:
GPQA Diamond
Diamond / Independent evaluation ↗- GPT-6 Astra:
- Claude Fable 5:
Humanity’s Last Exam
Stable HLE — full set ↗Tools enabled- GPT-6 Astra:
- Claude Fable 5:
Terminal-Bench
4.0 / mini-swe-agent ↗mini-swe-agent- GPT-6 Astra:
- Claude Fable 5:
Terminal-Bench
4.0 ↗- GPT-6 Astra:
- Claude Fable 5:
