SWE-bench Verified measures whether an agent can resolve real GitHub issues across 500 human-validated tasks. The primary track uses mini-SWE-agent 2.0.0; provider-reported setups remain separate.
| Model / creator | Version | Reported score |
|---|---|---|
 DeepSeek V4 Pro 0424DeepSeek · Open source / open weights | Verified | 80.6% |
| Verified | 80.5% | |
 Kimi K2.6Moonshot AI · Open source / open weights | Verified | 80.2% |
| DeepSeek V4 Flash 0420DeepSeek · Open source / open weights | Verified | 79% |
 Claude Opus 4.5Anthropic · Proprietary | Verified / mini-SWE-agent 2.0.0 / Bash Only | 76.8% |
| Kimi K2.5Moonshot AI · Open source / open weights | Verified | 76.8% |
 Gemini 3 FlashGoogle · Proprietary | Verified / mini-SWE-agent 2.0.0 / Bash Only | 75.8% |
| Claude Opus 4.6Anthropic · Proprietary | Verified / mini-SWE-agent 2.0.0 / Bash Only | 75.6% |
 GPT-5.2OpenAI · Proprietary | Verified / mini-SWE-agent 2.0.0 / Bash Only | 72.8% |
| Claude Sonnet 4.5Anthropic · Proprietary | Verified / mini-SWE-agent 2.0.0 / Bash Only | 71.4% |
| Kimi K2 ThinkingMoonshot AI · Open source / open weights | Verified | 71.3% |
| Kimi K2.5Moonshot AI · Open source / open weights | Verified / mini-SWE-agent 2.0.0 / Bash Only | 70.8% |
| Gemini 3 ProGoogle · Proprietary | Verified / mini-SWE-agent 2.0.0 / Bash Only | 69.6% |
| DeepSeek R1 (Jan 2025)DeepSeek · Open source / open weights | Verified | 49.2% |
attempts: 1 · harness: mini-SWE-agent · harness version: 2.0.0 · reasoning effort: high · Official Bash Only leaderboard; mini-SWE-agent 2.0.0; 500 tasks; one attempt; effort high · tasks: 500
SWE-bench ↗attempts: 1 · harness: mini-SWE-agent · harness version: 2.0.0 · reasoning effort: high · Official Bash Only leaderboard; mini-SWE-agent 2.0.0; 500 tasks; one attempt; effort high · tasks: 500
SWE-bench ↗attempts: 1 · harness: mini-SWE-agent · harness version: 2.0.0 · reasoning effort: high · Official Bash Only leaderboard; mini-SWE-agent 2.0.0; 500 tasks; one attempt; effort high · tasks: 500
SWE-bench ↗attempts: 1 · harness: mini-SWE-agent · harness version: 2.0.0 · Official Bash Only leaderboard; mini-SWE-agent 2.0.0; 500 tasks; one attempt; effort not specified · tasks: 500
SWE-bench ↗attempts: 1 · harness: mini-SWE-agent · harness version: 2.0.0 · reasoning effort: high · Official Bash Only leaderboard; mini-SWE-agent 2.0.0; 500 tasks; one attempt; effort high · tasks: 500
SWE-bench ↗attempts: 1 · harness: mini-SWE-agent · harness version: 2.0.0 · reasoning effort: high · Official Bash Only leaderboard; mini-SWE-agent 2.0.0; 500 tasks; one attempt; effort high · tasks: 500
SWE-bench ↗attempts: 1 · harness: mini-SWE-agent · harness version: 2.0.0 · reasoning effort: high · Official Bash Only leaderboard; mini-SWE-agent 2.0.0; 500 tasks; one attempt; effort high · tasks: 500
SWE-bench ↗harness comparison: provider-reported; settings differ across models · DeepSeek original V4 release; Think Max; published SWE Verified resolved score; original release, not 0813
DeepSeek ↗harness comparison: provider-reported; settings differ across models · MiniMax primary benchmark table; internal Claude Code harness; system prompt override; average of four runs
MiniMax ↗harness comparison: provider-reported; settings differ across models · Kimi K2.6 Thinking; quoted comparison table; follow original Kimi SWE harness
MiniMax ↗harness comparison: provider-reported; settings differ across models · DeepSeek original V4 release; Think Max; published SWE Verified resolved score; original release, not 0813
DeepSeek ↗harness comparison: provider-reported; settings differ across models · Internal minimal coding agent; non-thinking; average of five independent runs
Moonshot AI ↗harness comparison: provider-reported; settings differ across models · Moonshot in-house SWE-agent-derived harness; clamped Bash/Edit context and tailored prompt
Moonshot AI ↗harness comparison: provider-reported; settings differ across models · Original January 2025 R1 model card; SWE Verified resolved; not R1-0528
DeepSeek ↗