Work Automations
Private 657 tasks; 50 turns; objective completion
- 01
- 02
- 03
- 04
- 05
- 30
Muse Spark Open
Across the task benchmarks we track, Muse Spark Open’s strongest relative results are in work automations. On AutomationBench-AA, it scores 49.7%, ranking #30 among 55 models with reported results in our dataset.
Updated: Data as of:
Strongest results
| Benchmark | Measures | Score |
|---|---|---|
| Agent Index ↗v1.2 Preview | Overall agent capability | 8.0 / 100#24 of 26 |
| Humanity's Last Exam ↗Stable HLE — text-only subset | Expert reasoning | 43.9%#22 of 68 |
| LiveBench: Research ↗2.0 / 2026-10-25 | Research reports | 80.3%#10 of 25 |
| GDP.pdf ↗100 tasks / All-pass / five attempts | Professional document reasoning | 18.6%#25 of 55 |
Model details
| Model | Muse Spark Open |
|---|---|
| Agent Index v1.2 | 8.0 / 100 · Rank 24 · Preview |
| Creator | Meta (opens in a new tab) |
| Availability | Open source / open weights |
| Context size | Not reported |
| Max output | Not reported |
| Knowledge cutoff | Not reported |
| License | Muse Model License · Model weights ↗ (opens in a new tab) |
Pricing and performance
Per provider and reported configuration. Prices are USD per 1M tokens.
No provider pricing or performance reports have been added for this model yet.
Benchmark performance
Muse Spark Open in color, alongside leading models. Latest stored reports, best effort per report date.
AutomationBench-AA ↗
Reported benchmarks
All published results in our dataset, with versions, evaluation settings, and sources.
| Benchmark | Score | Evaluation setup | Source |
|---|---|---|---|
| AA-Briefcase1.1 / combined EloCombined Elo · Higher is better | 1,279 Elo | selection: best reported configuration per release · Muse Spark Open (xhigh) · tools: benchmark-defined | Artificial Analysis (opens in a new tab)independent |
| AutomationBench-AA1.0.6 / AA private 657Objectives completed without guardrail violations · Higher is better | 49.7% | reported precision decimals: 1 · result quality: reported · Muse Spark Open (xhigh); private held-out 657 tasks; 50 turns; objective completion with zero credit for any guardrail violation. · source model href: /models/muse-spark-open · task count: 657 · Tools enabled · turn cap: 50 | Artificial Analysis (opens in a new tab)independent |
| GDP.pdf100 tasks / All-pass / five attemptsAccuracy · Higher is better | 18.6% | selection: best reported configuration per release · Muse Spark Open (xhigh) · tools: benchmark-defined | Artificial Analysis (opens in a new tab)independent |
| HLE-Diamond1,000 questions / no toolsAccuracy · Higher is better | 26.3% | selection: best reported configuration per release · Muse Spark Open (xhigh) · No tools | Artificial Analysis (opens in a new tab)independent |
| Humanity's Last ExamStable HLE — text-only subsetAccuracy · Higher is better | 43.9% | question count: 2158 · reported precision decimals: 1 · Muse Spark Open (xhigh); original HLE; 2,158 text-only questions; no tools; pass@1; chart labels rounded to one decimal. · source model href: /models/muse-spark-open · subset: text · No tools | Artificial Analysis (opens in a new tab)independent |
| LiveBench: Agentic2.0 / 2026-10-25Accuracy · Higher is better | 11.7% | release: 2026-10-25 · Highest supported effort unless publisher reports an exception; single run · suite: agentic | LiveBench ↗ (opens in a new tab)benchmark publisher |
| LiveBench: Browser Use2.0 / 2026-10-25Accuracy · Higher is better | 26.2% | release: 2026-10-25 · Highest supported effort unless publisher reports an exception; single run · suite: browser-use | LiveBench ↗ (opens in a new tab)benchmark publisher |
| LiveBench: Coding2.0 / 2026-10-25Accuracy · Higher is better | 73.6% | release: 2026-10-25 · Highest supported effort unless publisher reports an exception; single run · suite: coding | LiveBench ↗ (opens in a new tab)benchmark publisher |
| LiveBench: Data Analysis2.0 / 2026-10-25Accuracy · Higher is better | 60.2% | release: 2026-10-25 · Highest supported effort unless publisher reports an exception; single run · suite: data-analysis | LiveBench ↗ (opens in a new tab)benchmark publisher |
| LiveBench: Design2.0 / 2026-10-25Accuracy · Higher is better | 60.3% | release: 2026-10-25 · Highest supported effort unless publisher reports an exception; single run · suite: design | LiveBench ↗ (opens in a new tab)benchmark publisher |
| LiveBench: Reasoning2.0 / 2026-10-25Accuracy · Higher is better | 73.7% | release: 2026-10-25 · Highest supported effort unless publisher reports an exception; single run · suite: reasoning | LiveBench ↗ (opens in a new tab)benchmark publisher |
| LiveBench: Research2.0 / 2026-10-25Accuracy · Higher is better | 80.3% | release: 2026-10-25 · Highest supported effort unless publisher reports an exception; single run · suite: research | LiveBench ↗ (opens in a new tab)benchmark publisher |
| Terminal-Bench4.0 / mini-swe-agentTask success · Higher is better | 8.1% | harness: mini-swe-agent · repeats: 3 · selection: best reported configuration per release · Muse Spark Open (xhigh) | Artificial Analysis (opens in a new tab)independent |