Long-horizon coding tasks
- 01
- 02
- 03
GPT-6 Astra74%
- 04
GPT-5.6 Sol73%
- 05
Compare models on software engineering, coding, and terminal tasks using publicly reported benchmark results.
| Model | DeepSWE ↗v1.1 | SWE-bench Verified ↗Shared harness preferred; provider reports otherwise | Terminal-Bench ↗4.0 preferred; 2.1 fallback | LiveBench: Coding ↗2.0 / 2026-10-25 |
|---|---|---|---|---|
| 74% | — | 53.9%4.0 native | — | |
| 74% | — | 19.1%4.0 native | 82.2% | |
| 74% | — | 58.2%4.0 native | 93.7% | |
| 73% | — | 37.3%4.0 native | — | |
| 70% | — | 44.5%4.0 native | — | |
| 69% | — | 41.8%4.0 native | — | |
| 69% | — | 12.6%4.0 shared | 59.3% | |
| 67% | — | 14.6%4.0 shared | — | |
| 59% | — | 23.6%4.0 native | — | |
| 55% | — | 7.1%4.0 shared | — | |
| 54% | — | 12.4%4.0 native | — | |
| — | — | 57.9%4.0 native | 97.7% | |
| — | — | 32.8%4.0 shared | 91.4% | |
| — | 76.8%Shared harness | — | — | |
| — | 75.6%Shared harness | — | — | |
| — | — | 64.8%4.0 native | 96.3% | |
| — | 71.4%Shared harness | 0%4.0 shared | — | |
| — | — | 3%4.0 shared | — | |
| — | — | 61.8%4.0 native | 94.7% | |
| — | 49.2%Provider report | 0%4.0 shared | — | |
| — | — | 3%4.0 shared | — | |
| — | 79%Provider report | 3%4.0 shared | — | |
| — | 80.6%Provider report | 14.6%4.0 shared | — | |
| — | — | 14.1%4.0 shared | — | |
| — | — | 26.8%4.0 shared | — | |
| — | — | 33.8%4.0 shared | 96% | |
| — | 75.8%Shared harness | — | — | |
| — | 69.6%Shared harness | — | — | |
| — | — | 4%4.0 shared | — | |
| — | — | 57.1%4.0 shared | 89.7% | |
| — | — | 65.2%4.0 shared | — | |
| — | — | 2%4.0 shared | — | |
| — | — | 1%4.0 shared | — | |
| — | — | 32.8%4.0 shared | 80.6% | |
| — | 72.8%Shared harness | — | — | |
| — | — | 2%4.0 shared | — | |
| — | — | 11.6%4.0 shared | — | |
| — | — | 35.4%4.0 shared | 86.2% | |
| — | — | 12.6%4.0 shared | 84.5% | |
| — | — | 58.2%4.0 native | 96% | |
| — | — | 55.6%4.0 shared | 94.3% | |
| — | — | 0%4.0 shared | — | |
| — | — | 0%4.0 shared | — | |
| — | — | 37.6%4.0 native | 84.5% | |
| — | — | 1%4.0 shared | 64% | |
| — | 71.3%Provider report | — | — | |
| — | 70.8%Shared harness | 45.7%2.1 legacy | — | |
| — | 80.2%Provider report | 0.5%4.0 shared | — | |
| — | — | 67.2%4.0 shared | 81.8% | |
| — | — | 34.8%4.0 shared | 81.5% | |
| — | — | 0%4.0 shared | — | |
| — | 80.5%Provider report | 2%4.0 shared | — | |
| — | — | 25.8%4.0 shared | 83.9% | |
| — | — | 6.1%4.0 shared | — | |
| — | — | 14.5%4.0 native | 90.2% | |
| — | — | 8.1%4.0 shared | 73.6% | |
| — | — | 52%4.0 shared | 86.2% | |
| — | — | 0%4.0 shared | 71.3% | |
| — | — | 0.5%4.0 shared | — | |
| — | — | 0%4.0 shared | — | |
| — | — | 0%4.0 shared | — | |
| — | — | 5.6%4.0 shared | 86.8% | |
| — | — | 33.3%4.0 shared | 85.6% |
Sorted by DeepSWE (11 reported models), followed by models with no score on that benchmark. SWE-bench Verified prefers shared-harness results and uses labeled provider reports when unavailable. Terminal-Bench prefers native 4.0, then shared 4.0, then legacy 2.1; each score is labeled. Scores across different benchmarks are not directly comparable. Hover a result to see its evaluation settings.