Reasoning
198-question Diamond subset; independent AA evaluation
- 01
- 02
- 03
- 03
- 03
- 20
Muse Spark 1.2
Across the task benchmarks we track, Muse Spark 1.2’s strongest relative results are in knowledge work, work automations, and reasoning. On GDPval-AA v2.1, it scores 1,480 Elo, ranking #19 among 38 models with reported results in our dataset.
Updated: Data as of:
Strongest results
| Benchmark | Measures | Score |
|---|---|---|
| Humanity's Last Exam ↗Stable HLE — text-only subset | Expert reasoning | 45.5%#20 of 68 |
| GDP.pdf ↗100 tasks / All-pass / five attempts | Professional document reasoning | 17.4%#27 of 55 |
| GDPval-AA ↗2.1 | Knowledge work | 1,480.04 Elo#19 of 38 |
| AA-Briefcase ↗1.1 / combined Elo | Business deliverables | 1,325 Elo#29 of 57 |
Model details
| Model | Muse Spark 1.2 |
|---|---|
| Agent Index v1.2 | Insufficient coverage (3/7) · Preview |
| Creator | Meta (opens in a new tab) |
| Availability | Proprietary |
| Context size | 1,048,576 tokens (opens in a new tab) |
| Max output | Not reported (opens in a new tab) |
| Knowledge cutoff | Not reported |
| License | Not reported |
Pricing and performance
Per provider and reported configuration. Prices are USD per 1M tokens.
| Provider / configuration | Contexttokens | InputUSD / 1M tokens | OutputUSD / 1M tokens | LatencyTTFT · seconds | Speedtokens / sec |
|---|---|---|---|---|---|
| Muse Spark 1.2 (xhigh) | 1,048,576 | $1.25 (opens in a new tab) | $4.25 (opens in a new tab) | 12.279 (opens in a new tab) | 277.574 (opens in a new tab) |
Benchmark performance
Muse Spark 1.2 in color, alongside leading models. Latest stored reports, best effort per report date.
GPQA Diamond ↗
Coding
113 tasks; mini-swe-agent
- 01
- 01
- 01
- 04
- 05
- 10
Work Automations
Private 657 tasks; 50 turns; objective completion
- 01
- 02
- 03
- 04
- 05
- 34
Knowledge Work
220 tasks; Stirrup agent; pairwise Elo
- 01
- 02
- 03
- 04
- 05
- 19
Reported benchmarks
All published results in our dataset, with versions, evaluation settings, and sources.
| Benchmark | Score | Evaluation setup | Source |
|---|---|---|---|
| AA-Briefcase1.1 / combined EloCombined Elo · Higher is better | 1,325 Elo | selection: best reported configuration per release · Muse Spark 1.2 (xhigh) · tools: benchmark-defined | Artificial Analysis (opens in a new tab)independent |
| AutomationBench-AA1.0.6 / AA private 657Objectives completed without guardrail violations · Higher is better | 40.6% | reported precision decimals: 1 · result quality: reported · Muse Spark 1.2 (xhigh); private held-out 657 tasks; 50 turns; objective completion with zero credit for any guardrail violation. · source model href: /models/muse-spark-1-2 · task count: 657 · Tools enabled · turn cap: 50 | Artificial Analysis (opens in a new tab)independent |
| DeepSWEv1.1Pass@1 · Higher is better | 55% | mini-swe-agent; effort=xhigh; ±2 percentage points | Datacurve ↗ (opens in a new tab)benchmark publisher |
| GDP.pdf100 tasks / All-pass / five attemptsAccuracy · Higher is better | 17.4% | selection: best reported configuration per release · Muse Spark 1.2 (xhigh) · tools: benchmark-defined | Artificial Analysis (opens in a new tab)independent |
| GDPval-AA2.1Elo · Higher is better | 1,480.04 Elo | Muse Spark 1.2 (Xhigh); 220 tasks; Stirrup agent; pairwise Elo | Artificial Analysis (opens in a new tab)independent |
| GPQADiamond / Artificial AnalysisAccuracy · Higher is better | 90.404% | Muse Spark 1.2 (Xhigh); 198-question Diamond subset; independent AA evaluation | Artificial Analysis (opens in a new tab)independent |
| Humanity's Last ExamStable HLE — text-only subsetAccuracy · Higher is better | 45.5% | question count: 2158 · reported precision decimals: 1 · Muse Spark 1.2 (xhigh); original HLE; 2,158 text-only questions; no tools; pass@1; chart labels rounded to one decimal. · source model href: /models/muse-spark-1-2 · subset: text · No tools | Artificial Analysis (opens in a new tab)independent |
| OSWorld2.1 / full setBinary accuracy · Higher is better | 15.84% | xhigh; batch tool; v2.1; full set | OSWorld / XLang ↗ (opens in a new tab)benchmark publisher |
| Terminal-Bench4.0 / mini-swe-agentTask success · Higher is better | 7.071% | harness: mini-swe-agent · repeats: 3 · selection: best reported configuration per release · Muse Spark 1.2 (Xhigh) | Artificial Analysis (opens in a new tab)independent |