Reasoning
198-question Diamond subset; independent AA evaluation
- 01
- 02
- 03
- 03
- 03
Gemini 3.8 Flash
Across the task benchmarks we track, Gemini 3.8 Flash’s strongest relative results are in coding, reasoning, and work automations. On DeepSWE v1.1, it scores 74%, ranking joint #1 among 11 models with reported results in our dataset.
Updated: Data as of:
🥇
Strongest results
| Benchmark | Measures | Score |
|---|---|---|
| Agent Index ↗v1.2 Preview | Overall agent capability | 12.5 / 100#22 of 26 |
| DeepSWE ↗v1.1 | Coding agents | 74%Joint #1 of 11 |
| GPQA ↗Diamond / Artificial Analysis | Reasoning | 95.25%#2 of 30 |
| Humanity's Last Exam ↗Stable HLE — text-only subset | Expert reasoning | 47.8%#13 of 68 |
Model details
| Model | Gemini 3.8 Flash |
|---|---|
| Agent Index v1.2 | 12.5 / 100 · Rank 22 · Preview |
| Creator | Google (opens in a new tab) |
| Availability | Proprietary |
| Context size | 1,000,000 tokens (opens in a new tab) |
| Max output | 65,536 tokens (opens in a new tab) |
| Knowledge cutoff | Not reported |
| License | Not reported |
Pricing and performance
Per provider and reported configuration. Prices are USD per 1M tokens.
| Provider / configuration | Contexttokens | InputUSD / 1M tokens | OutputUSD / 1M tokens | LatencyTTFT · seconds | Speedtokens / sec |
|---|---|---|---|---|---|
| Gemini 3.8 Flash (high) AI Studio | 1,000,000 | $0.75 (opens in a new tab) | $3.75 (opens in a new tab) | 27.1 (opens in a new tab) | 136.644 (opens in a new tab) |
Benchmark performance
Gemini 3.8 Flash in color, alongside leading models. Latest stored reports, best effort per report date.
GPQA Diamond ↗
Coding
113 tasks; mini-swe-agent
- 01
- 01
- 01
- 04
- 05
Terminal Use
Official 4.0 board; native agent harnesses
- 01
- 02
- 03
- 03
- 05
- 12
Work Automations
Private held-out set; API mode; 50 steps; strict pass/fail
- 01
- 02
- 03
- 04
- 05
- 06
Knowledge Work
220 tasks; Stirrup agent; pairwise Elo
- 01
- 02
- 03
- 04
- 05
- 24
Reported benchmarks
All published results in our dataset, with versions, evaluation settings, and sources.
| Benchmark | Score | Evaluation setup | Source |
|---|---|---|---|
| AA-Briefcase1.1 / combined EloCombined Elo · Higher is better | 1,203 Elo | selection: publisher default displayed configuration · Gemini 3.8 Flash (high) | Artificial Analysis (opens in a new tab)independent |
| AutomationBench1.0.6 / Zapier held-out private setStrict task completion · Higher is better | 29.68% | Gemini 3.8 Flash (High); private held-out 657 tasks; API mode; 50-step cap; strict completion | Zapier ↗ (opens in a new tab)benchmark publisher |
| AutomationBench1.0.6 / Zapier held-out private setStrict task completion · Higher is better | 29.68% | Gemini 3.8 Flash (Medium); private held-out 657 tasks; API mode; 50-step cap; strict completion | Zapier ↗ (opens in a new tab)benchmark publisher |
| AutomationBench-AA1.0.6 / AA private 657Objectives completed without guardrail violations · Higher is better | 60.9% | reported precision decimals: 1 · result quality: reported · Gemini 3.8 Flash (medium); private held-out 657 tasks; 50 turns; objective completion with zero credit for any guardrail violation. · source model href: /models/gemini-3-8-flash-medium · task count: 657 · Tools enabled · turn cap: 50 | Artificial Analysis (opens in a new tab)independent |
| DeepSWEv1.1Pass@1 · Higher is better | 74% | mini-swe-agent; effort=high; ±1 percentage points | Datacurve ↗ (opens in a new tab)benchmark publisher |
| DeepSWEv1.1Pass@1 · Higher is better | 74% | high; ±1 percentage point | Datacurve ↗ (opens in a new tab)benchmark publisher |
| DeepSWEv1.1 / OpenAI Astra launch comparisonPass@1 · Higher is better | 73.8% | OpenAI GPT-6 Astra launch comparison; best reported reasoning setting; see source evaluation footnotes | OpenAI ↗ (opens in a new tab)provider reported |
| GDP.pdf100 tasks / All-pass / five attemptsAccuracy · Higher is better | 22.8% | selection: best reported configuration per release · Gemini 3.8 Flash (medium) · tools: benchmark-defined | Artificial Analysis (opens in a new tab)independent |
| GDP.pdf100 tasks / All-pass / five attemptsAccuracy · Higher is better | 21% | selection: publisher default displayed configuration · Gemini 3.8 Flash (high) | Artificial Analysis (opens in a new tab)independent |
| GDPval-AA2.1Elo · Higher is better | 1,316.07 Elo | Gemini 3.8 Flash (Low); 220 tasks; Stirrup agent; pairwise Elo | Artificial Analysis (opens in a new tab)independent |
| GDPval-AA2.1Elo · Higher is better | 1,445.43 Elo | Gemini 3.8 Flash (High); 220 tasks; Stirrup agent; pairwise Elo | Artificial Analysis (opens in a new tab)independent |
| GDPval-AA2.1Elo · Higher is better | 1,431.32 Elo | Gemini 3.8 Flash (Medium); 220 tasks; Stirrup agent; pairwise Elo | Artificial Analysis (opens in a new tab)independent |
| GPQADiamond / Artificial AnalysisAccuracy · Higher is better | 92.02% | Gemini 3.8 Flash (Low); 198-question Diamond subset; independent AA evaluation | Artificial Analysis (opens in a new tab)independent |
| GPQADiamond / Artificial AnalysisAccuracy · Higher is better | 95.253% | Gemini 3.8 Flash (High); 198-question Diamond subset; independent AA evaluation | Artificial Analysis (opens in a new tab)independent |
| GPQADiamond / Artificial AnalysisAccuracy · Higher is better | 93.535% | Gemini 3.8 Flash (Medium); 198-question Diamond subset; independent AA evaluation | Artificial Analysis (opens in a new tab)independent |
| HLE-Diamond1,000 questions / no toolsAccuracy · Higher is better | 33.1% | selection: best reported configuration per release · Gemini 3.8 Flash (High) · No tools | Artificial Analysis (opens in a new tab)independent |
| Humanity's Last ExamStable HLE — text-only subsetAccuracy · Higher is better | 47.8% | question count: 2158 · reported precision decimals: 1 · Gemini 3.8 Flash (high); original HLE; 2,158 text-only questions; no tools; pass@1; chart labels rounded to one decimal. · source model href: /models/gemini-3-8-flash · subset: text · No tools | Artificial Analysis (opens in a new tab)independent |
| LiveBench: Agentic2.0 / 2026-10-25Accuracy · Higher is better | 16.5% | release: 2026-10-25 · Highest supported effort unless publisher reports an exception; single run · suite: agentic | LiveBench ↗ (opens in a new tab)benchmark publisher · 2026-10-25 |
| LiveBench: Browser Use2.0 / 2026-10-25Accuracy · Higher is better | 45.1% | release: 2026-10-25 · Highest supported effort unless publisher reports an exception; single run · suite: browser-use | LiveBench ↗ (opens in a new tab)benchmark publisher · 2026-10-25 |
| LiveBench: Coding2.0 / 2026-10-25Accuracy · Higher is better | 82.2% | release: 2026-10-25 · Highest supported effort unless publisher reports an exception; single run · suite: coding | LiveBench ↗ (opens in a new tab)benchmark publisher · 2026-10-25 |
| LiveBench: Data Analysis2.0 / 2026-10-25Accuracy · Higher is better | 54% | release: 2026-10-25 · Highest supported effort unless publisher reports an exception; single run · suite: data-analysis | LiveBench ↗ (opens in a new tab)benchmark publisher · 2026-10-25 |
| LiveBench: Design2.0 / 2026-10-25Accuracy · Higher is better | 72.7% | release: 2026-10-25 · Highest supported effort unless publisher reports an exception; single run · suite: design | LiveBench ↗ (opens in a new tab)benchmark publisher · 2026-10-25 |
| LiveBench: Reasoning2.0 / 2026-10-25Accuracy · Higher is better | 89.3% | release: 2026-10-25 · Highest supported effort unless publisher reports an exception; single run · suite: reasoning | LiveBench ↗ (opens in a new tab)benchmark publisher · 2026-10-25 |
| LiveBench: Research2.0 / 2026-10-25Accuracy · Higher is better | 79.1% | release: 2026-10-25 · Highest supported effort unless publisher reports an exception; single run · suite: research | LiveBench ↗ (opens in a new tab)benchmark publisher · 2026-10-25 |
| Terminal-Bench4.0Task success · Higher is better | 19.1% | mini-SWE-agent; effort=high; 19.1% ± 3.4% | Terminal-Bench ↗ (opens in a new tab)benchmark publisher |
| Terminal-Bench4.0 / mini-swe-agentTask success · Higher is better | 19.697% | harness: mini-swe-agent · repeats: 3 · selection: best reported configuration per release · Gemini 3.8 Flash (Medium) | Artificial Analysis (opens in a new tab)independent |