Reasoning
198-question Diamond subset; independent AA evaluation
- 01
- 02
- 03
- 03
- 03
Gemini 3.1 Pro
Across the task benchmarks we track, Gemini 3.1 Pro’s strongest relative results are in reasoning, knowledge work, and work automations. On GPQA Diamond, it scores 94.1%, ranking joint #3 among 30 models with reported results in our dataset.
Updated: Data as of:
Strongest results
| Benchmark | Measures | Score |
|---|---|---|
| GPQA ↗Diamond / Artificial Analysis | Reasoning | 94.14%Joint #3 of 30 |
| Humanity's Last Exam ↗Stable HLE — text-only subset | Expert reasoning | 47%#14 of 68 |
| IFBench ↗58 constraint types | Precise instruction following | 77.1%#5 of 17 |
| HLE-Diamond ↗1,000 questions / no tools | Advanced knowledge and reasoning | 31.4%#13 of 35 |
Model details
| Model | Gemini 3.1 Pro |
|---|---|
| Agent Index v1.2 | Insufficient coverage (4/7) · Preview |
| Creator | Google (opens in a new tab) |
| Availability | Proprietary |
| Context size | 1,000,000 tokens (opens in a new tab) |
| Max output | 65,536 tokens (opens in a new tab) |
| Knowledge cutoff | Not reported |
| License | Not reported |
Pricing and performance
Per provider and reported configuration. Prices are USD per 1M tokens.
| Provider / configuration | Contexttokens | InputUSD / 1M tokens | OutputUSD / 1M tokens | LatencyTTFT · seconds | Speedtokens / sec |
|---|---|---|---|---|---|
| Gemini 3.1 Pro Preview (AI Studio) | 1,000,000 | $2.00 (opens in a new tab) | $12.00 (opens in a new tab) | 22.733 (opens in a new tab) | 143.159 (opens in a new tab) |
| Gemini 3.1 Pro Preview (Vertex) | 1,000,000 | $2.00 (opens in a new tab) | $12.00 (opens in a new tab) | 27.154 (opens in a new tab) | 94.583 (opens in a new tab) |
Benchmark performance
Gemini 3.1 Pro in color, alongside leading models. Latest stored reports, best effort per report date.
GPQA Diamond ↗
Work Automations
Private held-out set; API mode; 50 steps; strict pass/fail
- 01
- 02
- 03
- 04
- 05
- 17
Knowledge Work
220 tasks; Stirrup agent; pairwise Elo
- 01
- 02
- 03
- 04
- 05
- 35
Reported benchmarks
All published results in our dataset, with versions, evaluation settings, and sources.
| Benchmark | Score | Evaluation setup | Source |
|---|---|---|---|
| AA-Briefcase1.1 / combined EloCombined Elo · Higher is better | 457 Elo | selection: best reported configuration per release · Gemini 3.1 Pro Preview · tools: benchmark-defined | Artificial Analysis (opens in a new tab)independent |
| AutomationBench1.0.6 / Zapier held-out private setStrict task completion · Higher is better | 4.57% | Gemini 3.1 Pro (preview) (Low); private held-out 657 tasks; API mode; 50-step cap; strict completion | Zapier ↗ (opens in a new tab)benchmark publisher |
| AutomationBench1.0.6 / Zapier held-out private setStrict task completion · Higher is better | 7% | Gemini 3.1 Pro (preview) (Medium); private held-out 657 tasks; API mode; 50-step cap; strict completion | Zapier ↗ (opens in a new tab)benchmark publisher |
| AutomationBench1.0.6 / Zapier held-out private setStrict task completion · Higher is better | 8.68% | Gemini 3.1 Pro (preview) (High); private held-out 657 tasks; API mode; 50-step cap; strict completion | Zapier ↗ (opens in a new tab)benchmark publisher |
| AutomationBench-AA1.0.6 / AA private 657Objectives completed without guardrail violations · Higher is better | 35.4% | reported precision decimals: 1 · result quality: reported · Gemini 3.1 Pro Preview; private held-out 657 tasks; 50 turns; objective completion with zero credit for any guardrail violation. · source model href: /models/gemini-3-1-pro-preview · task count: 657 · Tools enabled · turn cap: 50 | Artificial Analysis (opens in a new tab)independent |
| Gaia2CLI · OpenClaw 2026.4.1Pass@1 · Higher is better | 52% | api host: google · capabilities: search, execution, adaptability, ambiguity, time · edition: Gaia2-CLI · harness: OpenClaw 2026.4.1 · reasoning: high · reasoning context: Not specified | Meta Agents Research Environments ↗ (opens in a new tab)benchmark publisher · 2026-04-14 |
| GDP.pdf100 tasks / All-pass / five attemptsAccuracy · Higher is better | 17.8% | selection: best reported configuration per release · Gemini 3.1 Pro Preview · tools: benchmark-defined | Artificial Analysis (opens in a new tab)independent |
| GDPval-AA2.1Elo · Higher is better | 798.88 Elo | Gemini 3.1 Pro Preview; 220 tasks; Stirrup agent; pairwise Elo | Artificial Analysis (opens in a new tab)independent |
| GPQADiamond / Artificial AnalysisAccuracy · Higher is better | 94.141% | Gemini 3.1 Pro Preview; 198-question Diamond subset; independent AA evaluation | Artificial Analysis (opens in a new tab)independent |
| HLE-Diamond1,000 questions / no toolsAccuracy · Higher is better | 31.4% | selection: best reported configuration per release · Gemini 3.1 Pro Preview · No tools | Artificial Analysis (opens in a new tab)independent |
| Humanity's Last ExamStable HLE — full setAccuracy · Higher is better | 51.4% | Launch comparison; GPT models evaluated at xhigh effort; research environment. Competitor settings follow source footnotes. · subset: full · Tools enabled | OpenAI ↗ (opens in a new tab)provider reported · 2026-04-23 |
| Humanity's Last ExamStable HLE — full setAccuracy · Higher is better | 44.4% | Launch comparison; GPT models evaluated at xhigh effort; research environment. Competitor settings follow source footnotes. · subset: full · No tools | OpenAI ↗ (opens in a new tab)provider reported · 2026-04-23 |
| Humanity's Last ExamStable HLE — text-only subsetAccuracy · Higher is better | 47% | question count: 2158 · reported precision decimals: 1 · Gemini 3.1 Pro Preview; original HLE; 2,158 text-only questions; no tools; pass@1; chart labels rounded to one decimal. · source model href: /models/gemini-3-1-pro-preview · subset: text · No tools | Artificial Analysis (opens in a new tab)independent |
| IFBench58 constraint typesAccuracy · Higher is better | 77.1% | selection: publisher default displayed configuration · Gemini 3.1 Pro Preview | Artificial Analysis (opens in a new tab)independent |
| MCP-AtlasApril 2026 / public 500Task success · Higher is better | 80.6% | high; public 500; 100 tool calls | Scale AI ↗ (opens in a new tab)benchmark publisher |
| Terminal-Bench4.0 / mini-swe-agentTask success · Higher is better | 4.04% | harness: mini-swe-agent · repeats: 3 · selection: best reported configuration per release · Gemini 3.1 Pro Preview | Artificial Analysis (opens in a new tab)independent |
| π-BenchOverall · PROCProactivity (PROC) · Higher is better | 57.1% | completeness score: 60 · completeness uncertainty: 0.8 · metric: Avg PROC · personas: 5 · proc uncertainty: 0.9 · selection: Overall rows; not per-persona averages. COMP is a separate metric and does not determine this ranking. · tasks: 100 | π-Bench research team ↗ (opens in a new tab)benchmark publisher |