mini-swe-agent; effort=max; ±4 percentage points
Datacurve ↗Best LLMs for long-horizon coding tasks
DeepSWEDeepSWE evaluates coding agents on original, long-horizon software engineering tasks. It tests whether an agent can understand a codebase, make changes, and complete engineering work beyond short coding questions.
Updated: Data as of:
| Model / creator | Version | Reported score |
|---|---|---|
 GPT-6 AstraOpenAI · Proprietary | v1.1 | 74.1% |
 Claude Opus 5Anthropic · Proprietary | v1.1 | 74% |
 Gemini 3.8 FlashGoogle · Proprietary | v1.1 | 74% |
| GPT-6 AstraOpenAI · Proprietary | v1.1 | 74% |
| Gemini 3.8 FlashGoogle · Proprietary | v1.1 | 73.8% |
| Claude Opus 5Anthropic · Proprietary | v1.1 | 73.7% |
| GPT-5.6 SolOpenAI · Proprietary | v1.1 | 73% |
| GPT-5.6 SolOpenAI · Proprietary | v1.1 | 72.7% |
| Claude Fable 5Anthropic · Proprietary | v1.1 | 70% |
| Claude Fable 5Anthropic · Proprietary | v1.1 | 69.9% |
 GLM-5.3Z.ai · Open source / open weights | v1.1 | 69% |
 Kimi K3Moonshot AI · Open source / open weights | v1.1 | 69% |
| Claude Fable 5.1Anthropic · Proprietary | v1.1 | 67.4% |
| GPT-5.5OpenAI · Proprietary | v1.1 | 67% |
| Claude Opus 4.8Anthropic · Proprietary | v1.1 | 59% |
 Muse Spark 1.2Meta · Proprietary | v1.1 | 55% |
| Claude Sonnet 5Anthropic · Proprietary | v1.1 | 54% |
View all 23 published reports
mini-swe-agent; effort=high; ±1 percentage points
Datacurve ↗GPT-6 Astra74%
v1.1mini-swe-agent; effort=xhigh; ±3 percentage points
Datacurve ↗max; ±4 percentage points
Datacurve ↗high; ±1 percentage point
Datacurve ↗GPT-6 Astra74%
v1.1xhigh; ±3 percentage points
Datacurve ↗GPT-5.6 Sol73%
v1.1mini-swe-agent; effort=max; ±3 percentage points
Datacurve ↗GPT-5.6 Sol73%
v1.1max; ±3 percentage points
Datacurve ↗mini-swe-agent; effort=xhigh; ±3 percentage points
Datacurve ↗xhigh; ±3 percentage points
Datacurve ↗Kimi K369%
v1.1mini-swe-agent; effort=max; ±5 percentage points
Datacurve ↗GLM-5.369%
v1.1mini-swe-agent; effort=max; ±3 percentage points
Datacurve ↗GLM-5.369%
v1.1max; ±3 percentage points
Datacurve ↗GPT-5.567%
v1.1mini-swe-agent; effort=xhigh; ±6 percentage points
Datacurve ↗mini-swe-agent; effort=max; ±2 percentage points
Datacurve ↗mini-swe-agent; effort=xhigh; ±2 percentage points
Datacurve ↗mini-swe-agent; effort=max; ±4 percentage points
Datacurve ↗GPT-6 Astra74.1%
v1.1OpenAI GPT-6 Astra launch comparison; best reported reasoning setting; see source evaluation footnotes
OpenAI ↗Gemini 3.8 Flash73.8%
v1.1OpenAI GPT-6 Astra launch comparison; best reported reasoning setting; see source evaluation footnotes
OpenAI ↗Claude Opus 573.7%
v1.1OpenAI GPT-6 Astra launch comparison; best reported reasoning setting; see source evaluation footnotes
OpenAI ↗GPT-5.6 Sol72.7%
v1.1OpenAI GPT-6 Astra launch comparison; best reported reasoning setting; see source evaluation footnotes
OpenAI ↗Claude Fable 569.9%
v1.1OpenAI GPT-6 Astra launch comparison; best reported reasoning setting; see source evaluation footnotes
OpenAI ↗Claude Fable 5.167.4%
v1.1OpenAI GPT-6 Astra launch comparison; best reported reasoning setting; see source evaluation footnotes
OpenAI ↗