Work Automations
Private 657 tasks; 50 turns; objective completion
- 01
- 02
- 03
- 04
- 05
- 43
GLM-5.1
Across the task benchmarks we track, GLM-5.1’s strongest relative results are in work automations. On AutomationBench-AA, it scores 20.3%, ranking #43 among 55 models with reported results in our dataset.
Updated: Data as of:
Strongest results
| Benchmark | Measures | Score |
|---|---|---|
| IFBench ↗58 constraint types | Precise instruction following | 76.3%#7 of 17 |
| Humanity's Last Exam ↗Stable HLE — text-only subset | Expert reasoning | 30.1%Joint #49 of 68 |
| AA-Briefcase ↗1.1 / combined Elo | Business deliverables | 965 Elo#42 of 57 |
| Terminal-Bench ↗4.0 / mini-swe-agent | Working in a terminal | 2%Joint #43 of 56 |
Model details
| Model | GLM-5.1 |
|---|---|
| Agent Index v1.2 | Insufficient coverage (3/7) · Preview |
| Creator | Z.ai (opens in a new tab) |
| Availability | Open source / open weights |
| Context size | Not reported |
| Max output | Not reported |
| Knowledge cutoff | Not reported |
| License | Not reported · Model weights ↗ (opens in a new tab) |
Pricing and performance
Per provider and reported configuration. Prices are USD per 1M tokens.
No provider pricing or performance reports have been added for this model yet.
Benchmark performance
GLM-5.1 in color, alongside leading models. Latest stored reports, best effort per report date.
AutomationBench-AA ↗
Reported benchmarks
All published results in our dataset, with versions, evaluation settings, and sources.
| Benchmark | Score | Evaluation setup | Source |
|---|---|---|---|
| AA-Briefcase1.1 / combined EloCombined Elo · Higher is better | 965 Elo | selection: best reported configuration per release · GLM-5.1 · tools: benchmark-defined | Artificial Analysis (opens in a new tab)independent |
| AutomationBench-AA1.0.6 / AA private 657Objectives completed without guardrail violations · Higher is better | 20.3% | reported precision decimals: 1 · result quality: reported · GLM-5.1; private held-out 657 tasks; 50 turns; objective completion with zero credit for any guardrail violation. · source model href: /models/glm-5-1 · task count: 657 · Tools enabled · turn cap: 50 | Artificial Analysis (opens in a new tab)independent |
| Gaia2CLI · OpenClaw 2026.4.1Pass@1 · Higher is better | 50.5% | api host: OpenRouter · capabilities: search, execution, adaptability, ambiguity, time · edition: Gaia2-CLI · harness: OpenClaw 2026.4.1 · reasoning: enabled · reasoning context: Not round-tripped by this harness for OpenRouter; may affect multi-step performance | Meta Agents Research Environments ↗ (opens in a new tab)benchmark publisher · 2026-04-13 |
| GDP.pdf100 tasks / All-pass / five attemptsAccuracy · Higher is better | 8.4% | selection: best reported configuration per release · GLM-5.1 · tools: benchmark-defined | Artificial Analysis (opens in a new tab)independent |
| Humanity's Last ExamStable HLE — text-only subsetAccuracy · Higher is better | 30.1% | question count: 2158 · reported precision decimals: 1 · GLM-5.1; original HLE; 2,158 text-only questions; no tools; pass@1; chart labels rounded to one decimal. · source model href: /models/glm-5-1 · subset: text · No tools | Artificial Analysis (opens in a new tab)independent |
| IFBench58 constraint typesAccuracy · Higher is better | 76.3% | selection: publisher default displayed configuration · GLM-5.1 | Artificial Analysis (opens in a new tab)independent |
| Terminal-Bench4.0 / mini-swe-agentTask success · Higher is better | 2% | harness: mini-swe-agent · repeats: 3 · selection: best reported configuration per release · GLM-5.1 | Artificial Analysis (opens in a new tab)independent |
| π-BenchOverall · PROCProactivity (PROC) · Higher is better | 58.4% | completeness score: 63.6 · completeness uncertainty: 2.9 · metric: Avg PROC · personas: 5 · proc uncertainty: 0.8 · selection: Overall rows; not per-persona averages. COMP is a separate metric and does not determine this ranking. · tasks: 100 | π-Bench research team ↗ (opens in a new tab)benchmark publisher |