Work Automations
Private held-out set; API mode; 50 steps; strict pass/fail
- 01
- 02
- 03
- 04
- 05
Claude Fable 5.2
Across the task benchmarks we track, Claude Fable 5.2’s strongest relative results are in work automations and knowledge work. On AutomationBench 1.0.6, it scores 51.4%, ranking #2 among 18 models with reported results in our dataset.
Updated: Data as of:
🥇
Strongest results
| Benchmark | Measures | Score |
|---|---|---|
| Agent Index ↗v1.3 Preview | Overall agent capability | 89.6 / 100#1 of 27 |
| Remote Labor Index ↗240 freelance projects | Knowledge work | 41.3%#1 of 4 |
| CursorBench ↗4.0 / Cursor agent | Coding agents | 60%#1 of 17 |
| BenchCAD ↗2.1 / Core / with tools | Knowledge work | 83.5%#1 of 3 |
Model details
| Model | Claude Fable 5.2 |
|---|---|
| Agent Index v1.3 | 89.6 / 100 · Rank 1 · Preview |
| Creator | Anthropic (opens in a new tab) |
| Availability | Proprietary |
| Context size | 1,000,000 tokens (opens in a new tab) |
| Max output | 128,000 tokens (opens in a new tab) |
| Knowledge cutoff | 2026-06 |
| License | Not reported |
Pricing and performance
Per provider and reported configuration. Prices are USD per 1M tokens.
| Provider / configuration | Contexttokens | InputUSD / 1M tokens | OutputUSD / 1M tokens | LatencyTTFT · seconds | Speedtokens / sec |
|---|---|---|---|---|---|
| Fast mode API | 1,000,000 | $16.00 (opens in a new tab) | $80.00 (opens in a new tab) | Not reported | Not reported |
| Standard uncached API | 1,000,000 | $8.00 (opens in a new tab) | $40.00 (opens in a new tab) | Not reported | 103.417 (opens in a new tab) |
Blended token cost
USD per 1M tokens · 50% input + 50% output · lower is better
Claude Fable 5.2 highlighted10 of 45 priced models selected
Pricing details and sources
Blended cost = (input price + output price) ÷ 2: 500,000 input and 500,000 output tokens. Each model uses its lowest sourced current provider rate, excluding batch and Flex tiers. Cache discounts, tool fees and workload-specific token usage are excluded. 52 models without complete sourced current prices are unavailable in this comparison. The default selection is this model and the nine closest in blended cost; if this model has no price, we show the ten cheapest. The scale stays fixed across pages of your selection.
| Model | Input / 1M | Output / 1M | Blended / 1M | Provider / source |
|---|---|---|---|---|
| Claude Opus 4.6 | $5.00 | $25.00 | $15.00 | AmazonClaude Opus 4.6 (non-reasoning, high) · Checked 2026-11-02 |
| Claude Opus 4.7 | $5.00 | $25.00 | $15.00 | AmazonClaude Opus 4.7 (max) · Checked 2026-11-02 |
| Claude Opus 4.8 | $5.00 | $25.00 | $15.00 | AmazonClaude Opus 4.8 (max) · Checked 2026-11-02 |
| GPT-5.5 | $5.00 | $30.00 | $17.50 | OpenAIGPT-5.5 (xhigh) · Checked 2026-11-02 |
| Claude Fable 5.2 | $8.00 | $40.00 | $24.00 | Anthropic ↗Standard uncached API · Checked 2026-11-09 |
| Claude Mythos 5.2 | $8.00 | $40.00 | $24.00 | Anthropic ↗Standard uncached API · Checked 2026-11-09 |
| Claude Fable 5 | $10.00 | $50.00 | $30.00 | AnthropicClaude Fable 5 (with fallback) · Checked 2026-11-02 |
| Claude Fable 5.1 | $10.00 | $50.00 | $30.00 | AnthropicClaude Fable 5.1 (max with fallback) · Checked 2026-11-02 |
| Claude Mythos 5 | $10.00 | $50.00 | $30.00 | Anthropic ↗Official standard API pricing · Checked 2026-11-02 |
| GPT-6 Astra | $10.00 | $50.00 | $30.00 | AmazonGPT-6 Astra (max) · Checked 2026-11-02 |
Output speed
Output tokens per second · higher is better
Claude Fable 5.2 highlighted10 of 33 models with reports selected
Measurement details and sources
Output speed measures generated tokens per second. Each model uses its fastest sourced current provider report. Batch and Flex tiers are excluded. Provider, configuration, prompt and reporting methods can differ, so these are reported observations rather than controlled head-to-head tests. 64 models have no qualifying sourced speed report. Defaults include this model and the nine closest reported values, or the ten leading models if this model has no report. The scale stays fixed across selected pages.
| Model | Output speed | Provider / configuration | Measurement | Source |
|---|---|---|---|---|
| GPT-5.6 Sol | 126.629 t/s | OpenAIGPT-5.6 Sol (max) | longmedianArtificial Analysis: GPT-5.6 Sol (max); host OpenAI; standard uncached input/output pricing. TTFT is the first streamed token, which may be reasoning. Retrieval time is not measurement time; region/concurrency/quantization not specified in this record. | SourceChecked 2026-11-02 |
| MiniMax M3 | 119.672 t/s | MiniMaxMiniMax-M3 | source aggregatemedianExplicit first-party AA measurement; reasoning configuration retained. Cross-provider medians are excluded. Retrieval time is not measurement time. | SourceChecked 2026-11-02 |
| Claude Fable 5.2 | 103.417 t/s | AnthropicStandard uncached API | Prompt not specifiedStatistic not specifiedUSD per million uncached input/generated output tokens. Fast-mode speed multipliers are not absolute measured throughput. No TTFT inferred. AA max-effort/default-fallback first-party output-speed summary. Its first-answer latency is not TTFT and remains unpopulated. | SourceChecked 2026-11-09 |
| Claude Opus 5.5 | 97.19 t/s | AmazonClaude Opus 5.5 (max with fallback) | longmedianArtificial Analysis: Claude Opus 5.5 (max with fallback); host Amazon; standard uncached input/output pricing. TTFT is the first streamed token, which may be reasoning. Retrieval time is not measurement time; region/concurrency/quantization not specified in this record. | SourceChecked 2026-11-02 |
| Gemini 4 Carbon | 94.625 t/s | GoogleGemini 4 Carbon (high) | longmedianArtificial Analysis: Gemini 4 Carbon (high); host Google; standard uncached input/output pricing. TTFT is the first streamed token, which may be reasoning. Retrieval time is not measurement time; region/concurrency/quantization not specified in this record. | SourceChecked 2026-11-02 |
| GLM-5.2 | 94.073 t/s | Z.aiGLM-5.2 (Max) | source aggregatemedianExplicit first-party AA measurement; reasoning configuration retained. Cross-provider medians are excluded. Retrieval time is not measurement time. | SourceChecked 2026-11-02 |
| GPT-6 Astra | 89.577 t/s | AmazonGPT-6 Astra (max) | longmedianArtificial Analysis: GPT-6 Astra (max); host Amazon; standard uncached input/output pricing. TTFT is the first streamed token, which may be reasoning. Retrieval time is not measurement time; region/concurrency/quantization not specified in this record. | SourceChecked 2026-11-02 |
| Gemini 4 Argon | 87.964 t/s | Google AI StudioGemini 4 Argon (high) AI Studio | longmedianArtificial Analysis: Gemini 4 Argon (high) AI Studio; host Google AI Studio; standard uncached input/output pricing. TTFT is the first streamed token, which may be reasoning. Retrieval time is not measurement time; region/concurrency/quantization not specified in this record. | SourceChecked 2026-11-02 |
| Grok 4.7 | 87.436 t/s | SpaceXAIGrok 4.7 (xhigh) | longmedianArtificial Analysis: Grok 4.7 (xhigh); host SpaceXAI; standard uncached input/output pricing. TTFT is the first streamed token, which may be reasoning. Retrieval time is not measurement time; region/concurrency/quantization not specified in this record. | SourceChecked 2026-11-02 |
| Claude Sonnet 5 | 83.274 t/s | AzureClaude Sonnet 5 (max) | longmedianArtificial Analysis: Claude Sonnet 5 (max); host Azure; standard uncached input/output pricing. TTFT is the first streamed token, which may be reasoning. Retrieval time is not measurement time; region/concurrency/quantization not specified in this record. | SourceChecked 2026-11-02 |
Latency
Time to first token in seconds · lower is better
No sourced time-to-first-token latency for Claude Fable 5.2.10 of 32 models with reports selected
Measurement details and sources
Latency measures time to first token (TTFT), converted to seconds. Each model uses its lowest sourced current TTFT report; total response time is excluded. Batch and Flex tiers are excluded. Provider, configuration, prompt and reporting methods can differ, so these are reported observations rather than controlled head-to-head tests. 65 models have no qualifying sourced latency report. Defaults include this model and the nine closest reported values, or the ten leading models if this model has no report. The scale stays fixed across selected pages.
| Model | Latency | Provider / configuration | Measurement | Source |
|---|---|---|---|---|
| DeepSeek V4.1 Flash 1017 | 0.182 s | Baseten (FAST)DeepSeek V4.1 Flash 1017 (max) (FAST) | longmedianArtificial Analysis: DeepSeek V4.1 Flash 1017 (max) (FAST); host Baseten (FAST); standard uncached input/output pricing. TTFT is the first streamed token, which may be reasoning. Retrieval time is not measurement time; region/concurrency/quantization not specified in this record. | SourceChecked 2026-11-02 |
| DeepSeek V4.1 Flash | 0.311 s | Together AIDeepSeek V4.1 Flash (max) | longmedianArtificial Analysis: DeepSeek V4.1 Flash (max); host Together AI; standard uncached input/output pricing. TTFT is the first streamed token, which may be reasoning. Retrieval time is not measurement time; region/concurrency/quantization not specified in this record. | SourceChecked 2026-11-02 |
| MiniMax M3.1 Flash | 0.584 s | Together AIMiniMax-M3.1-Flash | longmedianArtificial Analysis: MiniMax-M3.1-Flash; host Together AI; standard uncached input/output pricing. TTFT is the first streamed token, which may be reasoning. Retrieval time is not measurement time; region/concurrency/quantization not specified in this record. | SourceChecked 2026-11-02 |
| GLM-5.3 | 0.597 s | Together AIGLM-5.3 (max) | longmedianArtificial Analysis: GLM-5.3 (max); host Together AI; standard uncached input/output pricing. TTFT is the first streamed token, which may be reasoning. Retrieval time is not measurement time; region/concurrency/quantization not specified in this record. | SourceChecked 2026-11-02 |
| MiniMax M3 | 0.699 s | MiniMaxMiniMax-M3 | source aggregatemedianExplicit first-party AA measurement; reasoning configuration retained. Cross-provider medians are excluded. Retrieval time is not measurement time. | SourceChecked 2026-11-02 |
| DeepSeek V4 Pro 0813 | 0.832 s | DigitalOceanDeepSeek V4 Pro 0813 (max) | longmedianArtificial Analysis: DeepSeek V4 Pro 0813 (max); host DigitalOcean; standard uncached input/output pricing. TTFT is the first streamed token, which may be reasoning. Retrieval time is not measurement time; region/concurrency/quantization not specified in this record. | SourceChecked 2026-11-02 |
| Kimi K3 | 0.87 s | DatabricksKimi K3 (max) | longmedianArtificial Analysis: Kimi K3 (max); host Databricks; standard uncached input/output pricing. TTFT is the first streamed token, which may be reasoning. Retrieval time is not measurement time; region/concurrency/quantization not specified in this record. | SourceChecked 2026-11-02 |
| Claude Opus 4.6 | 1.35 s | AmazonClaude Opus 4.6 (non-reasoning, high) | longmedianArtificial Analysis: Claude Opus 4.6 (non-reasoning, high); host Amazon; standard uncached input/output pricing. TTFT is the first streamed token, which may be reasoning. Retrieval time is not measurement time; region/concurrency/quantization not specified in this record. | SourceChecked 2026-11-02 |
| GLM-5.3-Flash | 1.546 s | Z.aiGLM-5.3-Flash | source aggregatemedianExplicit first-party AA measurement; reasoning configuration retained. Cross-provider medians are excluded. Retrieval time is not measurement time. | SourceChecked 2026-11-02 |
| GLM-5.2 | 1.777 s | Z.aiGLM-5.2 (Max) | source aggregatemedianExplicit first-party AA measurement; reasoning configuration retained. Cross-provider medians are excluded. Retrieval time is not measurement time. | SourceChecked 2026-11-02 |
Benchmark performance
Claude Fable 5.2 in color, alongside leading models. Latest stored reports, best effort per report date.
AutomationBench 1.0.6 ↗
Knowledge Work
220 tasks; Stirrup agent; pairwise Elo
- 01
- 02
- 03
- 04
- 05
Reported benchmarks
All published results in our dataset, with versions, evaluation settings, and sources.
| Benchmark | Score | Evaluation setup | Source |
|---|---|---|---|
| AA-Briefcase1.1 / combined EloCombined Elo · Higher is better | 1,562.73 Elo | effort: low · Claude Fable 5.2 (Low, Default Fallback); combined Elo; not rubric percentage. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| AA-Briefcase1.1 / combined EloCombined Elo · Higher is better | 1,622.24 Elo | effort: medium · Claude Fable 5.2 (Medium, Default Fallback); combined Elo; not rubric percentage. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| AA-Briefcase1.1 / combined EloCombined Elo · Higher is better | 1,677.44 Elo | effort: high · Claude Fable 5.2 (High, Default Fallback); combined Elo; not rubric percentage. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| AA-Briefcase1.1 / combined EloCombined Elo · Higher is better | 1,737.78 Elo | effort: xhigh · Claude Fable 5.2 (Xhigh, Default Fallback); combined Elo; not rubric percentage. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| AA-Briefcase1.1 / combined EloCombined Elo · Higher is better | 1,785.32 Elo | effort: max · Claude Fable 5.2 (Max, Default Fallback); combined Elo; not rubric percentage. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| AA-Briefcase1.1 / rubric pass rateRubric pass rate · Higher is better | 56.869% | effort: low · Claude Fable 5.2 (Low, Default Fallback); rubric pass rate; not combined Elo. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| AA-Briefcase1.1 / rubric pass rateRubric pass rate · Higher is better | 61.01% | effort: medium · Claude Fable 5.2 (Medium, Default Fallback); rubric pass rate; not combined Elo. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| AA-Briefcase1.1 / rubric pass rateRubric pass rate · Higher is better | 62.929% | effort: high · Claude Fable 5.2 (High, Default Fallback); rubric pass rate; not combined Elo. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| AA-Briefcase1.1 / rubric pass rateRubric pass rate · Higher is better | 65.152% | effort: xhigh · Claude Fable 5.2 (Xhigh, Default Fallback); rubric pass rate; not combined Elo. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| AA-Briefcase1.1 / rubric pass rateRubric pass rate · Higher is better | 66.082% | effort: max · Claude Fable 5.2 (Max, Default Fallback); rubric pass rate; not combined Elo. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| AutomationBench1.0.6 / Zapier held-out private setStrict task completion · Higher is better | 51.4% | effort: max · Zapier private held-out 657 tasks; API mode; default fallbacks; strict completion. System card attributes run to October 8, published November 9. · Tools enabled | Zapier via Anthropic system card ↗ (opens in a new tab)benchmark publisher · 2026-11-09 |
| AutomationBench-AA1.0.6 / AA private 657Objectives completed without guardrail violations · Higher is better | 67.997% | effort: low · Claude Fable 5.2 (Low, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| AutomationBench-AA1.0.6 / AA private 657Objectives completed without guardrail violations · Higher is better | 68.823% | effort: medium · Claude Fable 5.2 (Medium, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| AutomationBench-AA1.0.6 / AA private 657Objectives completed without guardrail violations · Higher is better | 71.36% | effort: high · Claude Fable 5.2 (High, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| AutomationBench-AA1.0.6 / AA private 657Objectives completed without guardrail violations · Higher is better | 74.163% | effort: xhigh · Claude Fable 5.2 (Xhigh, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| AutomationBench-AA1.0.6 / AA private 657Objectives completed without guardrail violations · Higher is better | 76.422% | effort: max · Claude Fable 5.2 (Max, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| BenchCAD2.1 / Core / standardMean partial-credit score · Higher is better | 75.4% | effort: max · 100 Core cases; five runs; mean partial credit; official scorer. Extra tool-assisted setup has no token/round cap; standard setup has 60 rounds and 500K output tokens. No numeric confidence interval inferred from chart. · No tools | Anthropic ↗ (opens in a new tab)provider reported · 2026-11-09 |
| BenchCAD2.1 / Core / with toolsMean partial-credit score · Higher is better | 83.5% | effort: max · 100 Core cases; five runs; mean partial credit; official scorer. Extra tool-assisted setup has no token/round cap; standard setup has 60 rounds and 500K output tokens. No numeric confidence interval inferred from chart. · Tools enabled | Anthropic ↗ (opens in a new tab)provider reported · 2026-11-09 |
| ChartographyAnthropic implementation / Gemini 3.5 Flash graderAccuracy · Higher is better | 91.6% | effort: max · 100 tasks; expert answer ranges; Gemini 3.5 Flash grader. Tool-enabled variant has container, standard libraries and image cropping. · Tools enabled | Anthropic ↗ (opens in a new tab)provider reported · 2026-11-09 |
| ChartographyAnthropic implementation / Gemini 3.5 Flash graderAccuracy · Higher is better | 73.4% | effort: max · 100 tasks; expert answer ranges; Gemini 3.5 Flash grader. Tool-enabled variant has container, standard libraries and image cropping. · No tools | Anthropic ↗ (opens in a new tab)provider reported · 2026-11-09 |
| CursorBench4.0 / Cursor agentScore · Higher is better | 53.2% | cost usd per task: 3.94 · effort: low · harness: Cursor production agent · output tokens per task: 36616 · Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant. · steps per task: 30 · Tools enabled | Cursor ↗ (opens in a new tab)benchmark publisher |
| CursorBench4.0 / Cursor agentScore · Higher is better | 54.8% | cost usd per task: 4.66 · effort: medium · harness: Cursor production agent · output tokens per task: 44103 · Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant. · steps per task: 35 · Tools enabled | Cursor ↗ (opens in a new tab)benchmark publisher |
| CursorBench4.0 / Cursor agentScore · Higher is better | 56.3% | cost usd per task: 15.48 · effort: xhigh · harness: Cursor production agent · output tokens per task: 161251 · Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant. · steps per task: 108 · Tools enabled | Cursor ↗ (opens in a new tab)benchmark publisher |
| CursorBench4.0 / Cursor agentScore · Higher is better | 56.9% | cost usd per task: 9.7 · effort: high · harness: Cursor production agent · output tokens per task: 99259 · Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant. · steps per task: 69 · Tools enabled | Cursor ↗ (opens in a new tab)benchmark publisher |
| CursorBench4.0 / Cursor agentScore · Higher is better | 60% | cost usd per task: 32.79 · effort: max · harness: Cursor production agent · output tokens per task: 352794 · Publisher table observed November 9; row publication date unspecified; current harness may differ from earlier system cards; small score differences may not be significant. · steps per task: 218 · Tools enabled | Cursor ↗ (opens in a new tab)benchmark publisher |
| DeepSWEv1.1Pass@1 · Higher is better | 74.3% | effort: max · 113 tasks; mean of five trials. Harness not established in the system-card section; not treated as the mini-swe-agent leaderboard. | Anthropic ↗ (opens in a new tab)provider reported · 2026-11-09 |
| FrontierCode2.0 / Cognition native agentsAccuracy · Higher is better | 49.1% | effort: xhigh · Cognition evaluation; 132 tasks, mean@5; Claude Code; highest-performing effort is xhigh, not max. · Tools enabled | Anthropic ↗ (opens in a new tab)provider reported · 2026-11-09 |
| FrontierCode2.0 / independent mini-swe-agentAccuracy · Higher is better | 38.44% | effort: low · Claude Fable 5.2 (Low, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| FrontierCode2.0 / independent mini-swe-agentAccuracy · Higher is better | 41.298% | effort: medium · Claude Fable 5.2 (Medium, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| FrontierCode2.0 / independent mini-swe-agentAccuracy · Higher is better | 49.128% | effort: high · Claude Fable 5.2 (High, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| FrontierCode2.0 / independent mini-swe-agentAccuracy · Higher is better | 47.581% | effort: xhigh · Claude Fable 5.2 (Xhigh, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| FrontierCode2.0 / independent mini-swe-agentAccuracy · Higher is better | 48.76% | effort: max · Claude Fable 5.2 (Max, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| FrontierSWE2 / Proximal harnessAccuracy · Higher is better | 69.9% | effort: max · Proximal harness; 34 tasks; 20h/trial; mean of five trials. · Tools enabled | Anthropic ↗ (opens in a new tab)provider reported · 2026-11-09 |
| GDP.pdf100 tasks / All-pass / five attemptsAccuracy · Higher is better | 26.8% | effort: low · Claude Fable 5.2 (Low, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · No tools | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| GDP.pdf100 tasks / All-pass / five attemptsAccuracy · Higher is better | 23.8% | effort: medium · Claude Fable 5.2 (Medium, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · No tools | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| GDP.pdf100 tasks / All-pass / five attemptsAccuracy · Higher is better | 21.8% | effort: high · Claude Fable 5.2 (High, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · No tools | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| GDP.pdf100 tasks / All-pass / five attemptsAccuracy · Higher is better | 21.4% | effort: xhigh · Claude Fable 5.2 (Xhigh, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · No tools | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| GDP.pdf100 tasks / All-pass / five attemptsAccuracy · Higher is better | 24.2% | effort: max · Claude Fable 5.2 (Max, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · No tools | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| GDPval-AA2.1Elo · Higher is better | 1,545.68 Elo | effort: low · Claude Fable 5.2 (Low, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| GDPval-AA2.1Elo · Higher is better | 1,591.16 Elo | effort: medium · Claude Fable 5.2 (Medium, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| GDPval-AA2.1Elo · Higher is better | 1,718.13 Elo | effort: high · Claude Fable 5.2 (High, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| GDPval-AA2.1Elo · Higher is better | 1,750.65 Elo | effort: xhigh · Claude Fable 5.2 (Xhigh, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| GDPval-AA2.1Elo · Higher is better | 1,831.33 Elo | effort: max · Claude Fable 5.2 (Max, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| Harvey LAB-AA1.1 / private 120 / hallucination-gatedHallucination-gated all-pass rate · Higher is better | 7.2% | effort: max · harness: Stirrup · 120 private tasks; v1.1 three-judge hallucination-gated all-pass rate. Distinct from ungated 10.8% Fable all-pass and earlier v1.0. · Tools enabled | Artificial Analysis via Anthropic system card ↗ (opens in a new tab)independent · 2026-11-09 |
| HLE-Diamond1,000 questions / no toolsAccuracy · Higher is better | 60.8% | effort: low · Claude Fable 5.2 (Low, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · No tools | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| HLE-Diamond1,000 questions / no toolsAccuracy · Higher is better | 66.3% | effort: medium · Claude Fable 5.2 (Medium, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · No tools | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| HLE-Diamond1,000 questions / no toolsAccuracy · Higher is better | 73.8% | effort: high · Claude Fable 5.2 (High, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · No tools | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| HLE-Diamond1,000 questions / no toolsAccuracy · Higher is better | 75.8% | effort: xhigh · Claude Fable 5.2 (Xhigh, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · No tools | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| HLE-Diamond1,000 questions / no toolsAccuracy · Higher is better | 80.2% | effort: max · Claude Fable 5.2 (Max, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · No tools | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| Humanity's Last ExamStable HLE — full setAccuracy · Higher is better | 73.2% | effort: max · Full 2,500 questions; auto thinking; 980K task budget; no compaction; Opus 4.6 judge. · subset: full · No tools | Anthropic ↗ (opens in a new tab)provider reported · 2026-11-09 |
| Humanity's Last ExamStable HLE — full setAccuracy · Higher is better | 74.2% | effort: max · Full 2,500 questions; auto thinking; 980K task budget; no compaction; Opus 4.6 judge; HLE source blocklist and cheating audit. · subset: full · Tools enabled | Anthropic ↗ (opens in a new tab)provider reported · 2026-11-09 |
| Humanity's Last ExamStable HLE — text-only subsetAccuracy · Higher is better | 59.731% | effort: low · Claude Fable 5.2 (Low, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · No tools | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| Humanity's Last ExamStable HLE — text-only subsetAccuracy · Higher is better | 61.909% | effort: medium · Claude Fable 5.2 (Medium, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · No tools | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| Humanity's Last ExamStable HLE — text-only subsetAccuracy · Higher is better | 66.172% | effort: high · Claude Fable 5.2 (High, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · No tools | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| Humanity's Last ExamStable HLE — text-only subsetAccuracy · Higher is better | 68.86% | effort: xhigh · Claude Fable 5.2 (Xhigh, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · No tools | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| Humanity's Last ExamStable HLE — text-only subsetAccuracy · Higher is better | 74.282% | effort: max · Claude Fable 5.2 (Max, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · No tools | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| LiveBench: Agentic2.0 / 2026-11-06Accuracy · Higher is better | 82.9% | release: 2026-11-06 · Official CSV exact model row; November 6 task set, observed November 9; effort not labeled. Not a pinned Agent Index component. · suite: agentic | LiveBench ↗ (opens in a new tab)benchmark publisher · 2026-11-09 |
| LiveBench: Browser Use2.0 / 2026-11-06Accuracy · Higher is better | 86.9% | release: 2026-11-06 · Official CSV exact model row; November 6 task set, observed November 9; effort not labeled. Not a pinned Agent Index component. · suite: browser-use | LiveBench ↗ (opens in a new tab)benchmark publisher · 2026-11-09 |
| LiveBench: Coding2.0 / 2026-11-06Accuracy · Higher is better | 79.2% | release: 2026-11-06 · Official CSV exact model row; November 6 task set, observed November 9; effort not labeled. Not a pinned Agent Index component. · suite: coding | LiveBench ↗ (opens in a new tab)benchmark publisher · 2026-11-09 |
| LiveBench: Data Analysis2.0 / 2026-11-06Accuracy · Higher is better | 81.5% | release: 2026-11-06 · Official CSV exact model row; November 6 task set, observed November 9; effort not labeled. Not a pinned Agent Index component. · suite: data-analysis | LiveBench ↗ (opens in a new tab)benchmark publisher · 2026-11-09 |
| LiveBench: Design2.0 / 2026-11-06Accuracy · Higher is better | 85.1% | release: 2026-11-06 · Official CSV exact model row; November 6 task set, observed November 9; effort not labeled. Not a pinned Agent Index component. · suite: design | LiveBench ↗ (opens in a new tab)benchmark publisher · 2026-11-09 |
| LiveBench: Reasoning2.0 / 2026-11-06Accuracy · Higher is better | 88.8% | release: 2026-11-06 · Official CSV exact model row; November 6 task set, observed November 9; effort not labeled. Not a pinned Agent Index component. · suite: reasoning | LiveBench ↗ (opens in a new tab)benchmark publisher · 2026-11-09 |
| LiveBench: Research2.0 / 2026-11-06Accuracy · Higher is better | 94.5% | release: 2026-11-06 · Official CSV exact model row; November 6 task set, observed November 9; effort not labeled. Not a pinned Agent Index component. · suite: research | LiveBench ↗ (opens in a new tab)benchmark publisher · 2026-11-09 |
| OSWorld2.1 / offline 82 / partial creditPartial-credit score · Higher is better | 89.3% | effort: max · 82 offline tasks out of 108; 1080p; 500 action steps; Pass@1 averaged over five attempts; Opus 4.8 grades applicable tasks. · Tools enabled | Anthropic ↗ (opens in a new tab)provider reported · 2026-11-09 |
| OSWorld2.1 / offline 82 / strict passStrict pass rate · Higher is better | 54.6% | effort: max · 82 offline tasks out of 108; 1080p; 500 action steps; Pass@1 averaged over five attempts; Opus 4.8 grades applicable tasks. · Tools enabled | Anthropic ↗ (opens in a new tab)provider reported · 2026-11-09 |
| Real-World Finance2 / November 2026 graderElo · Higher is better | 1,637 Elo | effort: max · 123 tasks; one attempt; three-hour deadline; Opus 5.5 grader; rescaled Elo. Fable missed six tasks; Opus completed all 123. Elo is not completion percentage. · Tools enabled | Anthropic ↗ (opens in a new tab)provider reported · 2026-11-09 |
| Remote Labor Index240 freelance projectsAutomation rate · Higher is better | 41.3% | effort: high · Scale AI run; 99/240 projects; human evaluation; 200K context and 32K response limit. · Tools enabled | Anthropic ↗ (opens in a new tab)provider reported · 2026-11-09 |
| RoboDojo SimAnthropic internal simulationAccuracy · Higher is better | 64.9% | effort: highest available · Internal simulation run; highest available reasoning effort. | Anthropic ↗ (opens in a new tab)provider reported · 2026-11-09 |
| SWE-bench MultilingualAnthropic November 2026 / five trialsAccuracy · Higher is better | 96.6% | effort: max · Mean of five trials; variant as stated in system card. No Verified or version-specific Pro score inferred. · Tools enabled | Anthropic ↗ (opens in a new tab)provider reported · 2026-11-09 |
| SWE-bench MultimodalAnthropic November 2026 / five trialsAccuracy · Higher is better | 74.1% | effort: max · Mean of five trials; variant as stated in system card. No Verified or version-specific Pro score inferred. · Tools enabled | Anthropic ↗ (opens in a new tab)provider reported · 2026-11-09 |
| SWE-bench ProAnthropic November 2026 / five trialsAccuracy · Higher is better | 93% | effort: max · Mean of five trials; variant as stated in system card. No Verified or version-specific Pro score inferred. · Tools enabled | Anthropic ↗ (opens in a new tab)provider reported · 2026-11-09 |
| Terminal-Bench4.0Task success · Higher is better | 67.6% | effort: max · harness: Claude Code --bare · 66 tasks; 10 trials/task; Claude Code --bare; no internet, cached resources and offline search; standard error ±1.8 points. Fable uses default server-side fallbacks. · Tools enabled | Anthropic ↗ (opens in a new tab)provider reported · 2026-11-09 |
| Terminal-Bench4.0 / mini-swe-agentTask success · Higher is better | 54.545% | effort: low · Claude Fable 5.2 (Low, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| Terminal-Bench4.0 / mini-swe-agentTask success · Higher is better | 55.051% | effort: medium · Claude Fable 5.2 (Medium, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| Terminal-Bench4.0 / mini-swe-agentTask success · Higher is better | 59.596% | effort: high · Claude Fable 5.2 (High, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| Terminal-Bench4.0 / mini-swe-agentTask success · Higher is better | 61.616% | effort: xhigh · Claude Fable 5.2 (Xhigh, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| Terminal-Bench4.0 / mini-swe-agentTask success · Higher is better | 59.596% | effort: max · Claude Fable 5.2 (Max, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| Terminal-Bench-Science0.1 / Claude Code / offlineAccuracy · Higher is better | 74.6% | effort: max · 70 tasks; 10 trials/task; Claude Code --bare; offline/cached resources; standard error ±4.3 points clustered by task. · Tools enabled | Anthropic ↗ (opens in a new tab)provider reported · 2026-11-09 |
| Terminal-Bench-Science0.1 / independent mini-swe-agentAccuracy · Higher is better | 53.81% | effort: low · Claude Fable 5.2 (Low, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| Terminal-Bench-Science0.1 / independent mini-swe-agentAccuracy · Higher is better | 60.476% | effort: medium · Claude Fable 5.2 (Medium, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| Terminal-Bench-Science0.1 / independent mini-swe-agentAccuracy · Higher is better | 70% | effort: high · Claude Fable 5.2 (High, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| Terminal-Bench-Science0.1 / independent mini-swe-agentAccuracy · Higher is better | 67.619% | effort: xhigh · Claude Fable 5.2 (Xhigh, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| Terminal-Bench-Science0.1 / independent mini-swe-agentAccuracy · Higher is better | 73.333% | effort: max · Claude Fable 5.2 (Max, Default Fallback); exact independent record; implementation per archived benchmarking methodology. · Tools enabled | Artificial Analysis (opens in a new tab)independent · 2026-11-09 |
| WANDRAnthropic offline / relaxed soft F1Soft F1 · Higher is better | 82.1% | effort: max · 500 tasks; modified soft F1; frozen web index; Opus 4.6 grader; 980K budget. Not directly comparable with Perplexity live-web runs. · Tools enabled | Anthropic ↗ (opens in a new tab)provider reported · 2026-11-09 |