Work Automations
Private 657 tasks; 50 turns; objective completion
- 01
- 02
- 03
- 04
- 05
Gemini 4 Hydrogen
Across the task benchmarks we track, Gemini 4 Hydrogen’s strongest relative results are in work automations. On AutomationBench-AA, it scores 76.14%, ranking #4 among 57 models with reported results in our dataset.
Updated: Data as of:
🥇
Strongest results
| Benchmark | Measures | Score |
|---|---|---|
| Vending-Bench 2 ↗2 / Google Hydrogen launch comparison | Knowledge work | 14,549 USD#1 of 3 |
| Agents’ Last Exam ↗1.1 / October 17 task set / Google Hydrogen launch | Knowledge work | 71.6%#1 of 3 |
| Chartography ↗Google Hydrogen launch / no tools | Knowledge work | 66.6%#1 of 3 |
| MKA ↗Climb WSR / Google Hydrogen launch | Knowledge work | 53.4%#1 of 3 |
Model details
| Model | Gemini 4 Hydrogen |
|---|---|
| Agent Index v1.3 | Insufficient coverage (4/7) · Preview |
| Creator | Google (opens in a new tab) |
| Availability | Proprietary |
| Context size | 1,048,576 tokens (opens in a new tab) |
| Max output | 65,536 tokens (opens in a new tab) |
| Knowledge cutoff | 2026-03 |
| License | Not reported |
Pricing and performance
Per provider and reported configuration. Prices are USD per 1M tokens.
| Provider / configuration | Contexttokens | InputUSD / 1M tokens | OutputUSD / 1M tokens | LatencyTTFT · seconds | Speedtokens / sec |
|---|---|---|---|---|---|
| Standard Gemini API; independent speed measured at xhigh | 1,048,576 | $1.50 (opens in a new tab) | $7.50 (opens in a new tab) | Not reported | 198.156 (opens in a new tab) |
Blended token cost
USD per 1M tokens · 50% input + 50% output · lower is better
Gemini 4 Hydrogen highlighted10 of 46 priced models selected
Pricing details and sources
Blended cost = (input price + output price) ÷ 2: 500,000 input and 500,000 output tokens. Each model uses its lowest sourced current provider rate, excluding batch and Flex tiers. Cache discounts, tool fees and workload-specific token usage are excluded. 52 models without complete sourced current prices are unavailable in this comparison. The default selection is this model and the nine closest in blended cost; if this model has no price, we show the ten cheapest. The scale stays fixed across pages of your selection.
| Model | Input / 1M | Output / 1M | Blended / 1M | Provider / source |
|---|---|---|---|---|
| Muse Spark 1.1 | $1.25 | $4.25 | $2.75 | MetaMuse Spark 1.1 (xhigh) · Checked 2026-11-02 |
| Muse Spark 1.2 | $1.25 | $4.25 | $2.75 | MetaMuse Spark 1.2 (xhigh) · Checked 2026-11-02 |
| GLM-5.2 | $1.40 | $4.40 | $2.90 | Z.aiGLM-5.2 (Max) · Checked 2026-11-02 |
| GPT-6.1 Terra | $1.00 | $5.00 | $3.00 | OpenAI ↗Standard API, prompts up to 272K input tokens · Checked 2026-11-02 |
| Grok 4.7 | $2.00 | $6.00 | $4.00 | SpaceXAIGrok 4.7 (xhigh) · Checked 2026-11-02 |
| Gemini 4 Hydrogen | $1.50 | $7.50 | $4.50 | Google ↗Standard Gemini API; independent speed measured at xhigh · Checked 2026-11-12 |
| Claude Sonnet 5 | $2.00 | $10.00 | $6.00 | AmazonClaude Sonnet 5 (max) · Checked 2026-11-02 |
| Claude Sonnet 5.5 | $2.00 | $10.00 | $6.00 | AmazonClaude Sonnet 5.5 (max with fallback) · Checked 2026-11-02 |
| GPT-6.1 Sol | $2.00 | $10.00 | $6.00 | AzureGPT-6.1 Sol (max) · Checked 2026-11-02 |
| Kimi K3.1 Pro | $2.00 | $10.00 | $6.00 | KimiKimi K3.1 Pro · Checked 2026-11-02 |
Output speed
Output tokens per second · higher is better
Gemini 4 Hydrogen highlighted10 of 34 models with reports selected
Measurement details and sources
Output speed measures generated tokens per second. Each model uses its fastest sourced current provider report. Batch and Flex tiers are excluded. Provider, configuration, prompt and reporting methods can differ, so these are reported observations rather than controlled head-to-head tests. 64 models have no qualifying sourced speed report. Defaults include this model and the nine closest reported values, or the ten leading models if this model has no report. The scale stays fixed across selected pages.
| Model | Output speed | Provider / configuration | Measurement | Source |
|---|---|---|---|---|
| Muse Spark 1.2 | 277.574 t/s | MetaMuse Spark 1.2 (xhigh) | longmedianArtificial Analysis: Muse Spark 1.2 (xhigh); host Meta; standard uncached input/output pricing. TTFT is the first streamed token, which may be reasoning. Retrieval time is not measurement time; region/concurrency/quantization not specified in this record. | SourceChecked 2026-11-02 |
| Gemini 4 Hydrogen | 198.156 t/s | GoogleStandard Gemini API; independent speed measured at xhigh | Prompt not specifiedStatistic not specifiedOfficial standard API price; output includes thinking. Explicit caching unsupported. Independent first-party Google output-speed headline at xhigh. 41.11s is time to first answer, so it is not entered as TTFT. No region/concurrency/measurement date was reported in this summary. | SourceChecked 2026-11-12 |
| Muse Spark 1.3 | 195.207 t/s | MetaMuse Spark 1.3 (max) | longmedianArtificial Analysis: Muse Spark 1.3 (max); host Meta; standard uncached input/output pricing. TTFT is the first streamed token, which may be reasoning. Retrieval time is not measurement time; region/concurrency/quantization not specified in this record. | SourceChecked 2026-11-02 |
| Claude Sonnet 5.5 | 144.019 t/s | GoogleClaude Sonnet 5.5 (max with fallback) | longmedianArtificial Analysis: Claude Sonnet 5.5 (max with fallback); host Google; standard uncached input/output pricing. TTFT is the first streamed token, which may be reasoning. Retrieval time is not measurement time; region/concurrency/quantization not specified in this record. | SourceChecked 2026-11-02 |
| Gemini 3.1 Pro | 143.159 t/s | Google (AI Studio)Gemini 3.1 Pro Preview (AI Studio) | longmedianArtificial Analysis: Gemini 3.1 Pro Preview (AI Studio); host Google (AI Studio); standard uncached input/output pricing. TTFT is the first streamed token, which may be reasoning. Retrieval time is not measurement time; region/concurrency/quantization not specified in this record. | SourceChecked 2026-11-02 |
| GPT-5.5 | 142.511 t/s | AmazonGPT-5.5 (xhigh) | longmedianArtificial Analysis: GPT-5.5 (xhigh); host Amazon; standard uncached input/output pricing. TTFT is the first streamed token, which may be reasoning. Retrieval time is not measurement time; region/concurrency/quantization not specified in this record. | SourceChecked 2026-11-02 |
| Gemini 3.8 Flash | 136.644 t/s | Google AI StudioGemini 3.8 Flash (high) AI Studio | longmedianArtificial Analysis: Gemini 3.8 Flash (high) AI Studio; host Google AI Studio; standard uncached input/output pricing. TTFT is the first streamed token, which may be reasoning. Retrieval time is not measurement time; region/concurrency/quantization not specified in this record. | SourceChecked 2026-11-02 |
| GPT-5.6 Sol | 126.629 t/s | OpenAIGPT-5.6 Sol (max) | longmedianArtificial Analysis: GPT-5.6 Sol (max); host OpenAI; standard uncached input/output pricing. TTFT is the first streamed token, which may be reasoning. Retrieval time is not measurement time; region/concurrency/quantization not specified in this record. | SourceChecked 2026-11-02 |
| MiniMax M3 | 119.672 t/s | MiniMaxMiniMax-M3 | source aggregatemedianExplicit first-party AA measurement; reasoning configuration retained. Cross-provider medians are excluded. Retrieval time is not measurement time. | SourceChecked 2026-11-02 |
| Claude Fable 5.2 | 103.417 t/s | AnthropicStandard uncached API | Prompt not specifiedStatistic not specifiedUSD per million uncached input/generated output tokens. Fast-mode speed multipliers are not absolute measured throughput. No TTFT inferred. AA max-effort/default-fallback first-party output-speed summary. Its first-answer latency is not TTFT and remains unpopulated. | SourceChecked 2026-11-09 |
Latency
Time to first token in seconds · lower is better
No sourced time-to-first-token latency for Gemini 4 Hydrogen.10 of 32 models with reports selected
Measurement details and sources
Latency measures time to first token (TTFT), converted to seconds. Each model uses its lowest sourced current TTFT report; total response time is excluded. Batch and Flex tiers are excluded. Provider, configuration, prompt and reporting methods can differ, so these are reported observations rather than controlled head-to-head tests. 66 models have no qualifying sourced latency report. Defaults include this model and the nine closest reported values, or the ten leading models if this model has no report. The scale stays fixed across selected pages.
| Model | Latency | Provider / configuration | Measurement | Source |
|---|---|---|---|---|
| DeepSeek V4.1 Flash 1017 | 0.182 s | Baseten (FAST)DeepSeek V4.1 Flash 1017 (max) (FAST) | longmedianArtificial Analysis: DeepSeek V4.1 Flash 1017 (max) (FAST); host Baseten (FAST); standard uncached input/output pricing. TTFT is the first streamed token, which may be reasoning. Retrieval time is not measurement time; region/concurrency/quantization not specified in this record. | SourceChecked 2026-11-02 |
| DeepSeek V4.1 Flash | 0.311 s | Together AIDeepSeek V4.1 Flash (max) | longmedianArtificial Analysis: DeepSeek V4.1 Flash (max); host Together AI; standard uncached input/output pricing. TTFT is the first streamed token, which may be reasoning. Retrieval time is not measurement time; region/concurrency/quantization not specified in this record. | SourceChecked 2026-11-02 |
| MiniMax M3.1 Flash | 0.584 s | Together AIMiniMax-M3.1-Flash | longmedianArtificial Analysis: MiniMax-M3.1-Flash; host Together AI; standard uncached input/output pricing. TTFT is the first streamed token, which may be reasoning. Retrieval time is not measurement time; region/concurrency/quantization not specified in this record. | SourceChecked 2026-11-02 |
| GLM-5.3 | 0.597 s | Together AIGLM-5.3 (max) | longmedianArtificial Analysis: GLM-5.3 (max); host Together AI; standard uncached input/output pricing. TTFT is the first streamed token, which may be reasoning. Retrieval time is not measurement time; region/concurrency/quantization not specified in this record. | SourceChecked 2026-11-02 |
| MiniMax M3 | 0.699 s | MiniMaxMiniMax-M3 | source aggregatemedianExplicit first-party AA measurement; reasoning configuration retained. Cross-provider medians are excluded. Retrieval time is not measurement time. | SourceChecked 2026-11-02 |
| DeepSeek V4 Pro 0813 | 0.832 s | DigitalOceanDeepSeek V4 Pro 0813 (max) | longmedianArtificial Analysis: DeepSeek V4 Pro 0813 (max); host DigitalOcean; standard uncached input/output pricing. TTFT is the first streamed token, which may be reasoning. Retrieval time is not measurement time; region/concurrency/quantization not specified in this record. | SourceChecked 2026-11-02 |
| Kimi K3 | 0.87 s | DatabricksKimi K3 (max) | longmedianArtificial Analysis: Kimi K3 (max); host Databricks; standard uncached input/output pricing. TTFT is the first streamed token, which may be reasoning. Retrieval time is not measurement time; region/concurrency/quantization not specified in this record. | SourceChecked 2026-11-02 |
| Claude Opus 4.6 | 1.35 s | AmazonClaude Opus 4.6 (non-reasoning, high) | longmedianArtificial Analysis: Claude Opus 4.6 (non-reasoning, high); host Amazon; standard uncached input/output pricing. TTFT is the first streamed token, which may be reasoning. Retrieval time is not measurement time; region/concurrency/quantization not specified in this record. | SourceChecked 2026-11-02 |
| GLM-5.3-Flash | 1.546 s | Z.aiGLM-5.3-Flash | source aggregatemedianExplicit first-party AA measurement; reasoning configuration retained. Cross-provider medians are excluded. Retrieval time is not measurement time. | SourceChecked 2026-11-02 |
| GLM-5.2 | 1.777 s | Z.aiGLM-5.2 (Max) | source aggregatemedianExplicit first-party AA measurement; reasoning configuration retained. Cross-provider medians are excluded. Retrieval time is not measurement time. | SourceChecked 2026-11-02 |
Benchmark performance
Gemini 4 Hydrogen in color, alongside leading models. Latest stored reports, best effort per report date.
AutomationBench-AA ↗
Reported benchmarks
All published results in our dataset, with versions, evaluation settings, and sources.
| Benchmark | Score | Evaluation setup | Source |
|---|---|---|---|
| AA-Briefcase1.1 / combined EloCombined Elo · Higher is better | 1,566.66 Elo | effort: xhigh · Exact independent xhigh record; Elo and rubric percentage remain separate. · Tools enabled | Artificial Analysis (opens in a new tab)independent |
| AA-Briefcase1.1 / rubric pass rateRubric pass rate · Higher is better | 55.96% | effort: xhigh · Exact independent xhigh record; Elo and rubric percentage remain separate. · Tools enabled | Artificial Analysis (opens in a new tab)independent |
| Agents’ Last Exam1.1 / October 17 task set / Google Hydrogen launchAverage rubric score · Higher is better | 71.6% | effort: highest available or best reported · Hydrogen: Antigravity CLI 1.4.8, 5h, safety filters. Sonnet: ALE-Claw under the reported setup. Terra: public board. Harnesses differ. · Tools enabled | Google DeepMind ↗ (opens in a new tab)provider reported · 2026-11-12 |
| AutomationBench-AA1.0.6 / AA private 657Objectives completed without guardrail violations · Higher is better | 76.144% | effort: xhigh · Exact independent xhigh record; methodology per published evaluation. · Tools enabled | Artificial Analysis (opens in a new tab)independent |
| ChartographyGoogle Hydrogen launch / no toolsAccuracy · Higher is better | 66.6% | effort: highest available or best reported · Hydrogen/Terra self-computed without tools following the Surge public evaluation. Sonnet quoted from system card. · No tools | Google DeepMind ↗ (opens in a new tab)provider reported · 2026-11-12 |
| FrontierCode2.0 / independent mini-swe-agentAccuracy · Higher is better | 38.959% | effort: xhigh · Exact independent xhigh record; methodology per published evaluation. · Tools enabled | Artificial Analysis (opens in a new tab)independent |
| GDP.pdf100 tasks / All-pass / five attemptsAccuracy · Higher is better | 27.6% | effort: xhigh · Exact independent xhigh record; methodology per published evaluation. · No tools | Artificial Analysis (opens in a new tab)independent |
| HLE-Diamond1,000 questions / no toolsAccuracy · Higher is better | 58.8% | effort: xhigh · Exact independent xhigh record; methodology per published evaluation. · No tools | Artificial Analysis (opens in a new tab)independent |
| MKAClimb WSR / Google Hydrogen launchClimb WSR accuracy · Higher is better | 53.4% | effort: highest available or best reported · Hydrogen self-computed by Google; peer figures quoted from independent evaluation. Not the separate xhigh independent Hydrogen record. · No tools | Google DeepMind ↗ (opens in a new tab)provider reported · 2026-11-12 |
| MKAClimb WSR / independent xhighClimb WSR accuracy · Higher is better | 49.52% | effort: xhigh · Exact independent xhigh record; methodology per published evaluation. | Artificial Analysis (opens in a new tab)independent |
| OSWorld2.1 / offline 82 / Google Hydrogen launchOffline partial score · Higher is better | 77.6% | effort: highest available or best reported · Hydrogen/Sonnet self-computed, best of three single-attempt runs. Official evaluator, 1080p, 500 steps. Hydrogen uses screenshot-only Gemini CUA with parallel tool calls and compaction. Terra from its system card. · Tools enabled | Google DeepMind ↗ (opens in a new tab)provider reported · 2026-11-12 |
| SWE-sweepGoogle Hydrogen launch / Antigravity CLI 1.4.8Subtask-weighted reward · Higher is better | 13.5% | effort: highest available or best reported · Hydrogen uses Antigravity CLI 1.4.8 with safety filters; peer figures are quoted from the public leaderboard. · Tools enabled | Google DeepMind ↗ (opens in a new tab)provider reported · 2026-11-12 |
| Terminal-Bench4.0 / mini-swe-agentTask success · Higher is better | 53.535% | effort: xhigh · Exact independent xhigh record; methodology per published evaluation. · Tools enabled | Artificial Analysis (opens in a new tab)independent |
| Terminal-Bench4.1 / Google Hydrogen launch / vision harnessCheating-adjusted accuracy · Higher is better | 57% | effort: high · Hydrogen: mini-swe-agent vision, high effort, five attempts per task averaged. Sonnet: official board. Not the Agent Index 4.0 three-repeat protocol. · Tools enabled | Google DeepMind ↗ (opens in a new tab)provider reported · 2026-11-12 |
| Terminal-Bench-Science0.1 / Google Hydrogen launch / vision harnessAverage reward · Higher is better | 51.4% | effort: highest available or best reported · Hydrogen uses mini-swe-agent vision, five attempts per task averaged. Peer figures quoted from public leaderboard. · Tools enabled | Google DeepMind ↗ (opens in a new tab)provider reported · 2026-11-12 |
| Terminal-Bench-Science0.1 / independent mini-swe-agentAccuracy · Higher is better | 49.524% | effort: xhigh · Exact independent xhigh record; methodology per published evaluation. · Tools enabled | Artificial Analysis (opens in a new tab)independent |
| Vending-Bench 22 / Google Hydrogen launch comparisonFinal bank balance · Higher is better | 14,549 USD | effort: highest available or best reported · Google quotes Andon Labs public leaderboard final bank balance; values are dollars, not percentages. · Tools enabled | Google DeepMind ↗ (opens in a new tab)provider reported · 2026-11-12 |