OpenCodePapers

code-generation-on-swe-bench-multilingual

Code IntelligenceCode Generation
Dataset Link
Results over time
Click legend items to toggle metrics. Hover points for model names.
Leaderboard
PaperCodeAccuracyModelNameReleaseDate
Introducing Claude Opus 5.5✓ Link93.9Claude Opus 5.52026-09-22
Claude Fable 5 and Claude Mythos 5✓ Link92.2Claude Mythos 52026-06-09
Introducing Claude Sonnet 5.5✓ Link90.3Claude Sonnet 5.52026-09-28
Introducing Claude Opus 5✓ Link89.5Claude Opus 52026-07-24
Introducing Claude Fable 5.1 and Claude Mythos 5.189.1Claude Fable 5.1 (max effort)2026-09-01
Introducing Claude Opus 4.884.4Claude Opus 4.82026-05-28
Introducing Claude Haiku 5.583.7Claude Haiku 5.5 (max effort)2026-10-07
Hy4 preview✓ Link82.9Hy4 preview (swe-agent scaffold)2026-08-28
Intern-S2-Preview: Scientific Agentic Foundation Model✓ Link81.67Intern-S2-Preview-397B (Mini-SWE-Agent)2026-08-13
On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability✓ Link81.0Qwen3.8-Flash-Next (mini-SWE-agent)2026-08-31
Introducing Claude Opus 4.780.5Claude Opus 4.72026-04-16
Composer 2 Technical Report79.8Composer 2.5 (Cursor harness)2026-03-25
Ornith-1.0: Self-Scaffolding LLMs for Agentic Coding✓ Link78.9Ornith-1.0-397B (OpenHands)2026-06-21
Introducing Laguna S 2.1✓ Link78.5Laguna S 2.1 (pool)2026-07-21
Introducing LongCat-2.0✓ Link78.3LongCat-2.02026-07-05
Introducing Grok 4.578.0Grok 4.52026-07-08
Introducing Beam: Reflection's 501B open-weight model78.0Beam2026-10-05
Introducing Claude Opus 4.677.83Claude Opus 4.62026-02-05
Kimi K2.6✓ Link76.7Kimi K2.6 (In-house SWE-agent-adapted framework)2026-04-20
The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligence✓ Link76.5MiniMax-M2.72026-05-26
Introducing Claude Opus 4.576.2Claude Opus 4.52025-11-24
DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence✓ Link76.2DeepSeek-V4-Pro Max2026-04-26
Introducing Claude Sonnet 4.675.9Claude Sonnet 4.62026-02-17
Hy3✓ Link75.8Hy3 (SWE-agent)2026-07-02
dots3-note Preview: A Small but Mighty Step Toward Long-Horizon Agency in Real Life✓ Link75.67dots3-note Preview (live-swe-agent)2026-08-13
KAT-Coder-V2 Technical Report75.4KAT-Coder-V2 (Claude Code)2026-03-29
Composer 2 Technical Report73.7Composer 2 (Cursor harness)2026-03-25
GLM-5: From Vibe Coding to Agentic Engineering✓ Link73.3GLM-5 (OpenHands)2026-02-11
DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence✓ Link73.3DeepSeek-V4-Flash Max2026-04-26
Kimi K2.5: Visual Agentic Intelligence✓ Link73.0Kimi K2.5 (Internal agent scaffold)2026-02-02
Ling-3.0-flash72.4Ling-3.0-flash (OpenHands)2026-07-23
MiMo-V2-Flash Technical Report✓ Link71.7MiMo-V2-Flash (Thinking)2026-01-06
Ornith-1.5: From Self-Scaffolding to Self-Improvement71.4Ornith-1.5-35B-A3B (OpenHands)2026-08-20
Qwen3.6✓ Link71.3Qwen3.6-27B (Internal agent scaffold)2026-04-21
DeepSeek-V3.2 Technical Report✓ Link70.2DeepSeek-V3.2 (Internal framework)2025-12-02
Qwen3.5: Towards Native Multimodal Agents✓ Link69.3Qwen3.5-397B-A17B (Internal agent scaffold)2026-02-16
Ornith-1.0: Self-Scaffolding LLMs for Agentic Coding✓ Link69.3Ornith-1.0-35B (OpenHands)2026-06-21
MindForge: Teaching Small Language Models Whole-Life-Cycle Software Engineering via Source-Free Program Synthesis67.77MindForge-27B (Mini-SWE-Agent)2026-07-29
Priming Base Models for Agentic Post-Training: An Open Midtraining Recipe67.72NVIDIA Nemotron 3.5 Super VL 120B-A12B BF162026-10-15
Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning✓ Link67.7Nemotron 3 Ultra (OpenHands)2026-06-03
Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters✓ Link67.4Step-3.5-Flash2026-02-11
Qwen3.6✓ Link67.2Qwen3.6-35B-A3B (Internal agent scaffold)2026-04-21
Qwen3-Coder-Next Technical Report✓ Link64.3Qwen3-Coder-Next (OpenHands)2026-02-28
Laguna M.1/XS.2 Technical Report✓ Link63.1Laguna M.1 (pool)2026-05-26
Introducing Laguna XS 2.1✓ Link63.1Laguna XS 2.1 (pool)2026-07-02
Self-Evolving Coding Agents via Harness-Policy Optimization✓ Link59.7Self-HPO-Qwen3.5-35B-A3B2026-10-16
One to More, More to One: Category-Aware Iterative Expert Training for Software Engineering Agents✓ Link59.0Logics-SWE-Qwen3.6-27B (SWE-agent)2026-09-20
Laguna M.1/XS.2 Technical Report✓ Link57.7Laguna XS.2 (pool)2026-05-26
Ornith-1.0: Self-Scaffolding LLMs for Agentic Coding✓ Link52.0Ornith-1.0-9B (OpenHands)2026-06-21
Dockerless: Environment-Free Program Verifier for Coding Agents50.0Dockerless-RL-9B (OpenHands)2026-06-26
Kimi K2: Open Agentic Intelligence✓ Link47.3Kimi K22025-07-28
NVIDIA Nemotron 3: Efficient and Open Intelligence✓ Link45.78Nemotron 3 Super (OpenHands)2025-12-24
Granite 4.2 LLMs: How They're Built✓ Link41.89Granite-4.2-30B2026-08-25
Let It Flow: Agentic Crafting on Rock and Roll, Building the ROME Model within an Open Agentic Learning Ecosystem✓ Link40.0ROME (30B MoE, activated 3B) (iFlow CLI)2025-12-31
NVIDIA Nemotron 3.5 Lightning✓ Link39.33NVIDIA Nemotron 3.5 Lightning 30B-A3B (NeMo Evaluator)2026-08-11
Scaling Embeddings Outperforms Scaling Experts in Language Models38.1LongCat-Flash-Lite (68.5B)2026-01-29
LoopCoder-v2: Only Loop Once for Efficient Test-Time Computation Scaling✓ Link31.0LoopCoder-v22026-06-16
SWE-smith: Scaling Data for Software Engineering Agents✓ Link8.4SWE-agent-LM-32B2025-04-30