Work Automations
Private held-out set; API mode; 50 steps; strict pass/fail
- 01
- 02
- 03
- 04
- 05
Gemini 4 Carbon
Across the task benchmarks we track, Gemini 4 Carbon’s strongest relative results are in work automations and knowledge work. On AutomationBench 1.0.6, it scores 53.88%, ranking #1 among 17 models with reported results in our dataset.
Updated: Data as of:
🥇
Strongest results
| Benchmark | Measures | Score |
|---|---|---|
| AutomationBench-AA ↗1.0.6 / AA private 657 | Work automations | 77.8%#1 of 55 |
| AutomationBench ↗1.0.6 / Zapier held-out private set | Work automations | 53.88%#1 of 17 |
| HLE-Diamond ↗1,000 questions / no tools | Advanced knowledge and reasoning | 69.1%#1 of 35 |
| Terminal-Bench ↗4.0 / mini-swe-agent | Working in a terminal | 65.15%#2 of 56 |
Model details
| Model | Gemini 4 Carbon |
|---|---|
| Agent Index v1.2 | Insufficient coverage (4/7) · Preview |
| Creator | Google (opens in a new tab) |
| Availability | Proprietary |
| Context size | 1,000,000 tokens (opens in a new tab) |
| Max output | Not reported (opens in a new tab) |
| Knowledge cutoff | Not reported |
| License | Not reported |
Pricing and performance
Per provider and reported configuration. Prices are USD per 1M tokens.
| Provider / configuration | Contexttokens | InputUSD / 1M tokens | OutputUSD / 1M tokens | LatencyTTFT · seconds | Speedtokens / sec |
|---|---|---|---|---|---|
| Gemini 4 Carbon (high) | 1,000,000 | $4.00 (opens in a new tab) | $20.00 (opens in a new tab) | 156.309 (opens in a new tab) | 94.625 (opens in a new tab) |
Benchmark performance
Gemini 4 Carbon in color, alongside leading models. Latest stored reports, best effort per report date.
AutomationBench 1.0.6 ↗
Knowledge Work
220 tasks; Stirrup agent; pairwise Elo
- 01
- 02
- 03
- 04
- 05
- 06
Reported benchmarks
All published results in our dataset, with versions, evaluation settings, and sources.
| Benchmark | Score | Evaluation setup | Source |
|---|---|---|---|
| AA-Briefcase1.1 / combined EloCombined Elo · Higher is better | 1,610 Elo | selection: publisher default displayed configuration · Gemini 4 Carbon (high) | Artificial Analysis (opens in a new tab)independent |
| AutomationBench1.0.6 / Zapier held-out private setStrict task completion · Higher is better | 31.51% | Gemini 4 Carbon (Low); private held-out 657 tasks; API mode; 50-step cap; strict completion | Zapier ↗ (opens in a new tab)benchmark publisher |
| AutomationBench1.0.6 / Zapier held-out private setStrict task completion · Higher is better | 52.97% | Gemini 4 Carbon (Medium); private held-out 657 tasks; API mode; 50-step cap; strict completion | Zapier ↗ (opens in a new tab)benchmark publisher |
| AutomationBench1.0.6 / Zapier held-out private setStrict task completion · Higher is better | 53.88% | Gemini 4 Carbon (High); private held-out 657 tasks; API mode; 50-step cap; strict completion | Zapier ↗ (opens in a new tab)benchmark publisher |
| AutomationBench-AA1.0.6 / AA private 657Objectives completed without guardrail violations · Higher is better | 77.8% | high | Artificial Analysis (opens in a new tab)independent |
| GDP.pdf100 tasks / All-pass / five attemptsAccuracy · Higher is better | 29.6% | selection: best reported configuration per release · Gemini 4 Carbon (medium) · tools: benchmark-defined | Artificial Analysis (opens in a new tab)independent |
| GDP.pdf100 tasks / All-pass / five attemptsAccuracy · Higher is better | 28.4% | selection: publisher default displayed configuration · Gemini 4 Carbon (high) | Artificial Analysis (opens in a new tab)independent |
| GDPval-AA2.1Elo · Higher is better | 1,732.68 Elo | Gemini 4 Carbon (High); 220 tasks; Stirrup agent; pairwise Elo | Artificial Analysis (opens in a new tab)independent |
| HLE-Diamond1,000 questions / no toolsAccuracy · Higher is better | 69.1% | selection: best reported configuration per release · Gemini 4 Carbon (High) · No tools | Artificial Analysis (opens in a new tab)independent |
| Terminal-Bench4.0 / mini-swe-agentTask success · Higher is better | 65.152% | harness: mini-swe-agent · repeats: 3 · selection: best reported configuration per release · Gemini 4 Carbon (High) | Artificial Analysis (opens in a new tab)independent |