OpenCodePapers

code-generation-on-swe-bench-verified

Code IntelligenceCode Generation
Dataset Link
Results over time
Click legend items to toggle metrics. Hover points for model names.
Leaderboard
PaperCodeAccuracyModelNameReleaseDate
Introducing Claude Opus 5✓ Link96.0Claude Opus 52026-07-24
Claude Fable 5 and Claude Mythos 5✓ Link95.5Claude Mythos 52026-06-09
Introducing Ember-192.2Ember-12026-09-23
Introducing Claude Opus 4.888.6Claude Opus 4.82026-05-28
Introducing Claude Opus 4.787.6Claude Opus 4.72026-04-16
Macaron-V1: Towards Open Continual Learning with Self-Improvement and Mixture-of-LoRA✓ Link85.6Macaron-V1-Venti (Claude Code)2026-08-10
Lean4AgenticRL: Advancing Agentic RL with Verifiable Multi-level Process Rewards✓ Link84.2LAR-27B (Claude-code)2026-10-15
Ornith-1.0: Self-Scaffolding LLMs for Agentic Coding✓ Link82.4Ornith-1.0-397B (OpenHands)2026-06-21
NVIDIA-labs OO Agents: Native Python Object-Oriented Agents82.2NOOA agent (GPT-5.5, xhigh)2026-07-22
Introducing Claude Opus 4.580.9Claude Opus 4.52025-11-24
Introducing Beam: Reflection's 501B open-weight model80.9Beam2026-10-05
Introducing Claude Opus 4.680.84Claude Opus 4.62026-02-05
Gemini 3.1 Pro: A smarter model for your most complex tasks80.6Gemini 3.1 Pro2026-02-19
DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence✓ Link80.6DeepSeek-V4-Pro Max2026-04-26
MiniMax M3: Frontier Coding, 1M Context, Native Multimodality — All in One Model✓ Link80.5MiniMax M3 (Claude Code)2026-05-31
MiniMax-M2.5✓ Link80.2MiniMax-M2.5 (Claude Code)2026-02-12
Kimi K2.6✓ Link80.2Kimi K2.6 (In-house SWE-agent-adapted framework)2026-04-20
Inkling-Small✓ Link80.2Inkling-Small (Bash-only)2026-07-27
Introducing GPT-5.280.0GPT-5.22025-12-11
Introducing Claude Sonnet 4.679.6Claude Sonnet 4.62026-02-17
KAT-Coder-V2 Technical Report79.6KAT-Coder-V2 (Claude Code)2026-03-29
DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence✓ Link79.0DeepSeek-V4-Flash Max2026-04-26
Ornith-1.5: From Self-Scaffolding to Self-Improvement79.0Ornith-1.5-35B-A3B (OpenHands)2026-08-20
MiMo-V2.5-Pro✓ Link78.9MiMo-V2.5-Pro2026-04-27
Qwen3.6-Plus: Towards Real World Agents78.8Qwen3.6-Plus2026-04-02
dots3-note Preview: A Small but Mighty Step Toward Long-Horizon Agency in Real Life✓ Link78.4dots3-note Preview (live-swe-agent)2026-08-13
Hy3✓ Link78.0Hy3 (SWE-agent)2026-07-02
GLM-5: From Vibe Coding to Agentic Engineering✓ Link77.8GLM-5 (OpenHands)2026-02-11
Inkling: Our open-weights model✓ Link77.6Inkling (bash-only harness)2026-07-15
Marathoner: Ultra-Long-Horizon Autonomous Intelligence77.5Marathoner-9B (Claude Code)2026-09-28
Introducing Claude Sonnet 4.577.2Claude Sonnet 4.52025-09-29
Qwen3.6✓ Link77.2Qwen3.6-27B (Internal agent scaffold)2026-04-21
Kimi K2.5: Visual Agentic Intelligence✓ Link76.8Kimi K2.5 (Internal agent scaffold)2026-02-02
ModularRSI: Modular and Generalizable Recursive Harness Self-Improvement✓ Link76.45ModularRSI (Harbor)2026-09-14
Qwen3.5: Towards Native Multimodal Agents✓ Link76.4Qwen3.5-397B-A17B (Internal agent scaffold)2026-02-16
A new era of intelligence with Gemini 376.2Gemini 3 Pro (single attempt)2025-11-18
Motif 3: Technical Report✓ Link76.2Motif 3 (mini-SWE-agent)2026-08-10
Muse Glimmer 30B: An On-Device Agentic Model✓ Link76.0Muse Glimmer-30B (High)2026-10-17
Ornith-1.0: Self-Scaffolding LLMs for Agentic Coding✓ Link75.6Ornith-1.0-35B (OpenHands)2026-06-21
Introducing GPT-574.9GPT-5 (High)2025-08-07
InCoder-32B: Code Foundation Model for Industrial Scenarios✓ Link74.8InCoder-32B (32B)2026-03-17
Laguna M.1/XS.2 Technical Report✓ Link74.6Laguna M.1 (pool)2026-05-26
Claude Opus 4.174.5Claude Opus 4.12025-08-05
Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters✓ Link74.4Step-3.5-Flash2026-02-11
Hy3 preview✓ Link74.4Hy3 preview2026-04-23
Ling and Ring 2.6 Technical Report: Efficient and Instant Agentic Intelligence at Trillion-Parameter Scale✓ Link74.00Ring-2.6-1T (High) (Claude Code)2026-06-13
MindForge: Teaching Small Language Models Whole-Life-Cycle Software Engineering via Source-Free Program Synthesis73.84MindForge-27B (Mini-SWE-Agent)2026-07-29
MAI-Thinking-1: Building a Hill-Climbing Machine73.5MAI-Thinking-12026-06-02
MiMo-V2-Flash Technical Report✓ Link73.4MiMo-V2-Flash (Thinking)2026-01-06
Qwen3.6✓ Link73.4Qwen3.6-35B-A3B (Internal agent scaffold)2026-04-21
DeepSeek-V3.2 Technical Report✓ Link73.1DeepSeek-V3.2 (Internal framework)2025-12-02
Introducing Claude 472.7Claude Sonnet 42025-05-22
Introducing Claude 472.5Claude Opus 42025-05-22
Qwen3.5: Towards Native Multimodal Agents✓ Link72.4Qwen3.5-27B2026-02-16
Ling-2.6-1T: A Trillion-Parameter Comprehensive Flagship Model for Complex Tasks72.20Ling-2.6-1T (OpenHands)2026-04-29
SkillForge: Self-Distilling Agents for Project-Specific Issue Resolution✓ Link72.2SkillForge (DeepSeek-V3.2) (Mini-SWE-Agent) (extra data)2026-08-19
Bilevel Coordinated Reflection: A Game-Theoretic Approach to Multi-Agent LLM Systems✓ Link72.2Bilevel SRMA (Kimi K2.5) (official test harness)2026-09-02
Qwen3.5: Towards Native Multimodal Agents✓ Link72.0Qwen3.5-122B-A10B2026-02-16
Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning✓ Link71.9Nemotron 3 Ultra (OpenHands)2026-06-03
Qwen3-Coder-Next Technical Report✓ Link71.3Qwen3-Coder-Next (OpenHands)2026-02-28
Priming Base Models for Agentic Post-Training: An Open Midtraining Recipe71.1NVIDIA Nemotron 3.5 Super VL 120B-A12B BF162026-10-15
Introducing Laguna XS 2.1✓ Link70.9Laguna XS 2.1 (pool)2026-07-02
PreciseCoder: Faithful and Generalizable Debugging Beyond Unit-Test Rewards70.8PreciseCoder-27B (SWE-agent)2026-10-13
Mach-Mind-4-Flash Technical Report70.60Mach-Mind-4-Flash2026-07-10
Introducing K2 Horizon: Frontier Performance, Radically Open✓ Link70.6K2-Horizon-7B2026-09-03
InCoder-32B-Thinking: Industrial Code World Model for Thinking✓ Link70.4InCoder-32B-Thinking2026-04-03
Solar Open 2 Technical Report✓ Link70.4Solar Open 2 250B-A15B2026-07-22
LEGO-RL: Harness-Native Reinforcement Learning for Coding Agents✓ Link70.4Lego-RL-Qwen3.5-35B-A3B (OpenHands SDK) (extra data)2026-08-18
Laguna M.1/XS.2 Technical Report✓ Link69.9Laguna XS.2 (pool)2026-05-26
MemGovern: Enhancing Code Agents through Learning from Governed Human Experiences✓ Link69.8MemGovern (Claude-4-Sonnet) (SWE-Agent)2026-01-11
Self-Evolving Coding Agents via Harness-Policy Optimization✓ Link69.5Self-HPO-Qwen3.5-35B-A3B2026-10-16
Ornith-1.0: Self-Scaffolding LLMs for Agentic Coding✓ Link69.4Ornith-1.0-9B (OpenHands)2026-06-21
Qwen3.5: Towards Native Multimodal Agents✓ Link69.2Qwen3.5-35B-A3B2026-02-16
UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning✓ Link68.7UI-TARS-2 (w/ GUI-SDK)2025-09-02
ROSS: Relearning from Self-Generated Rollouts through Selective Supervision68.4ROSS (Qwen3.6-35B-A3B) (Codex CLI)2026-09-28
Qwen-AgentWorld: Language World Models for General Agents✓ Link67.86w/ LWM RL on Qwen3.5-35B-A3B-SFT (Internal agent scaffold (bash and file-edit tools))2026-06-23
Introducing North Mini Code: Cohere's first model for developers✓ Link67.6North Mini Code (SWE-agent)2026-06-09
Orchard: An Open-Source Agentic Modeling Framework✓ Link67.5Qwen3-30B-A3B-Thinking (Orchard-SWE, SFT+RL) (mini-swe-agent)2026-05-14
PACT: From Credit Assignment to Critic Alignment✓ Link67.4PACT (Qwen3.6-35B-A3B) (Codex through Harbor)2026-09-22
Kolibri: A Sovereign European Model on the Pareto Frontier✓ Link66.4Kolibri (OpenCode)2026-10-03
daVinci-Env: Open SWE Environment Synthesis at Scale✓ Link66.0OpenSWE-72B (SWE-Agent)2026-03-13
PivotOPD: Learning to Recover from Pivotal Mistakes in Multi-Turn Agents66.0PivotOPD (SWE-Bench Verified)2026-09-30
Rufus-Air: An Open LLM Post-Training Recipe65.6Rufus-Air2026-09-24
LoopCoder-v2: Only Loop Once for Efficient Test-Time Computation Scaling✓ Link64.4LoopCoder-v22026-06-16
GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models✓ Link64.2GLM-4.5 (355B) (OpenHands)2025-08-08
Nanbeige4.2-3B: Unlocking Agentic Capabilities in a Compact Model✓ Link63.6Nanbeige4.2-3B (OpenHands)2026-07-21
gpt-oss-120b & gpt-oss-20b Model Card62.4gpt-oss-120b (High)2025-08-08
daVinci-Env: Open SWE Environment Synthesis at Scale✓ Link62.4OpenSWE-32B (SWE-Agent)2026-03-13
Claude 3.7 Sonnet and Claude Code62.3Claude 3.7 Sonnet2025-02-24
Kwai Keye-VL-2.0 Technical Report✓ Link62.0Keye-VL-2.0-30B-A3B2026-06-09
Dockerless: Environment-Free Program Verifier for Coding Agents62.0Dockerless-RL-9B (OpenHands)2026-06-26
Closing the Loop: Practical Training Recipes for Looped Language Models61.8Dory1.0-4B (NeMo Gym / Nemo Evaluator SDK)2026-09-30
FrogNano: Training a 4B Coding Agent via Online Task Synthesis✓ Link61.5FrogNano-4B-2609 (Leaf)2026-09-07
SWE-Master: Unleashing the Potential of Software Engineering Agents via Post-Training✓ Link61.4SWE-Master-32B-RL (R2E-Gym)2026-02-03
Ling and Ring 2.6 Technical Report: Efficient and Instant Agentic Intelligence at Trillion-Parameter Scale✓ Link61.20Ling-2.6-flash (Claude Code)2026-06-13
NVIDIA Nemotron 3: Efficient and Open Intelligence✓ Link60.47Nemotron 3 Super (OpenHands)2025-12-24
LongCat-Flash Technical Report✓ Link60.40LongCat-Flash (OpenHands)2025-09-01
SWE-Lego: Pushing the Limits of Supervised Fine-tuning for Software Issue Resolving✓ Link60.4SWE-Lego-Qwen3-32B (OpenHands)2026-01-04
NVIDIA Nemotron 3.5 Lightning✓ Link60.0NVIDIA Nemotron 3.5 Lightning 30B-A3B (OpenCode v1.17.8)2026-08-11
Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities59.6Gemini 2.5 Pro (single attempt)2025-07-07
ReTerm: Recovery Training for Self-Improving Terminal Agents✓ Link58.4ReTerm-9B (ReAct)2026-10-13
GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models✓ Link57.6GLM-4.5-Air (OpenHands)2025-08-08
Let It Flow: Agentic Crafting on Rock and Roll, Building the ROME Model within an Open Agentic Learning Ecosystem✓ Link57.4ROME (30B MoE, activated 3B) (iFlow CLI)2025-12-31
Granite 4.2 LLMs: How They're Built✓ Link57Granite-4.2-30B2026-08-25
Agent Lightning v1.0: Towards Harnessed Agentic RL✓ Link56.4Agent Lightning v1.0 (Qwen3.5-9B) (mini-SWE-agent) (extra data)2026-08-18
SERA: Soft-Verified Efficient Repository Agents✓ Link54.2SERA-32B (SWE-agent)2026-01-28
OpenThoughts-Agent: Data Recipes for Agentic Models✓ Link54.0OpenThinkerAgent-32B (Terminus-2)2026-06-23
Tmax: A simple recipe for terminal agents✓ Link53.5Tmax 9B (Vanillux2Agent)2026-06-22
LFM2.5-24B-A2B: Built for Fast On-Device Agents51.00LFM2.5-24B-A2B (OpenCode)2026-10-13
K-EXAONE Technical Report✓ Link49.4K-EXAONE (236B, reasoning) (mini-SWE-agent)2026-01-05
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning✓ Link49.2DeepSeek-R12025-01-22
Introducing OpenAI o148.9OpenAI o1 (High)2024-09-12
Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities48.9Gemini 2.5 Flash (single attempt)2025-07-07
Unified Audio Intelligence Without Regressing on Text Intelligence48.2Audex 30B-A3B (OpenHands)2026-07-06
MiniCPM5-2B✓ Link46.4MiniCPM5-2B2026-09-07
AEM: Adaptive Entropy Modulation for Multi-Turn Agentic Reinforcement Learning43.7DeepSWE + AEM (Qwen3-32B)2026-05-08
DeepSeek-V3 Technical Report✓ Link42.0DeepSeek-V3 (Agentless)2024-12-27
FC-SWE: Failure-Conditioned RL for Long-Horizon Software Engineering Agents41.7FC-SWE (Qwen3.5-4B) (SWE-agent)2026-10-06
Polar: Agentic RL on Any Harness at Scale✓ Link40.4Polar RL (Qwen3.5-4B) (Pi)2026-05-22
SWE-smith: Scaling Data for Software Engineering Agents✓ Link40.2SWE-agent-LM-32B2025-04-30
Function-Aware Fill-in-the-Middle as Mid-Training for Coding Agent Foundation Models✓ Link35.0Qwen3-8B + FIM-Midtrain + SWE-Lego (OpenHands)2026-07-14
PivotRL: High Accuracy Agentic Post-Training at Low Compute Cost32.67Qwen3-30B-A3B-Thinking-2507 + PivotRL (OpenHands)2026-03-22
SERA: Soft-Verified Efficient Repository Agents✓ Link31.7SERA-8B (SWE-agent)2026-01-28
Context-Aware RL for Agentic and Multimodal LLMs✓ Link30.2ContextRL (trained from Klear-AgentForge-8B) (mini-SWE-agent)2026-06-15
SWE-AGILE: A Software Agent Framework for Efficiently Managing Dynamic Reasoning Context✓ Link30.06SWE-AGILE (SFT) (Qwen3-14B) (R2EGym)2026-04-13
AgentDevel: Reframing Self-Evolving LLM Agents as Release Engineering30.0AgentDevel (final) (Claude Code)2026-01-08
Revisiting DAgger in the Era of LLM-Agents29.8Qwen3-8B + DAgger-style Training (OpenHands)2026-05-13
Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning29.8Qwen3-30B-A3B + SAO (OpenHands)2026-07-08
ProRL Agent: Rollout-as-a-Service for RL Training of Multi-Turn LLM Agents✓ Link23.6ProRL Agent-14B (RL)2026-03-19
What Should Agents Say? Action-state Communication for Efficient Multi-Agent Systems✓ Link23.0PACT (Qwen3-14B) — OpenHands2026-06-03
Youtu-LLM: Unlocking the Native Agentic Potential for Lightweight Large Language Models✓ Link17.7Youtu-LLM 2B2025-12-31