OpenCodePapers

code-generation-on-swe-bench-pro

Code Generation
Dataset Link
Results over time
Click legend items to toggle metrics. Hover points for model names.
Leaderboard
Click a sortable column header to sort.
PaperCodeResolvedModelNameReleaseDate
Claude Fable 5 and Claude Mythos 580.3Claude Mythos 52026-06-09
Introducing Claude Opus 4.869.2Claude Opus 4.82026-05-28
Introducing Grok 4.564.7Grok 4.52026-07-08
GPT-5.6: Frontier intelligence that scales with your ambition64.6GPT-5.6 Sol2026-07-09
Introducing Claude Opus 4.764.3Claude Opus 4.72026-04-16
Ornith-1.0: Self-Scaffolding LLMs for Agentic Coding✓ Link62.2Ornith-1.0-397B2026-06-21
GLM-5.2: Built for Long-Horizon Tasks✓ Link62.1GLM-5.22026-06-16
Introducing Muse Spark 1.161.50Muse Spark 1.12026-07-09
Introducing LongCat-2.0✓ Link59.5LongCat-2.02026-07-05
Introducing GPT-5.459.10gpt-5.4 (xHigh)2026-03-05
MiniMax M3: Frontier Coding, 1M Context, Native Multimodality — All in One Model✓ Link59.0MiniMax M32026-05-31
Kimi K2.6✓ Link58.6Kimi K2.62026-04-20
Introducing GPT-5.558.6GPT-5.52026-04-23
GLM-5.1: Towards Long-Horizon Tasks✓ Link58.4GLM-5.12026-04-03
Hy3✓ Link57.9Hy32026-07-02
Introducing GPT-5.457.7GPT-5.42026-03-05
Seed2.1 Model Card: Agentic Intelligence for Productivity57.5Seed2.1 Pro2026-06-23
MiMo-V2.5-Pro✓ Link57.2MiMo-V2.5-Pro2026-04-27
Introducing GPT-5.3-Codex56.8GPT-5.3-Codex2026-02-05
Introducing GPT-5.2-Codex56.4GPT-5.2-Codex2026-01-14
Step 3.7 Flash✓ Link56.3Step 3.7 Flash2026-05-29
The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligence✓ Link56.2MiniMax-M2.72026-05-26
Introducing GPT-5.255.6GPT-5.22025-12-11
DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence✓ Link55.4DeepSeek-V4-Pro Max2026-04-26
The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligence✓ Link55.4MiniMax-M2.52026-05-26
Introducing Muse Spark55.00Muse Spark2026-04-08
Gemini 3.1 Pro: A smarter model for your most complex tasks54.2Gemini 3.1 Pro2026-02-19
Introducing Claude Opus 4.651.90claude-opus-4-6 (thinking)2026-02-05
Gemini 3.1 Pro: A smarter model for your most complex tasks46.10gemini-3.1-pro (thinking)2026-02-19
Introducing Claude Opus 4.545.89claude-opus-4-5-202511012025-11-24
Introducing Claude Sonnet 4.543.60claude-4-5-Sonnet2025-09-29
Gemini 3: Try Google's newest, most intelligent AI model43.30gemini-3-pro-preview2025-11-18
Introducing Claude 442.70claude-4-Sonnet2025-05-22
Introducing GPT-541.78gpt-5-2025-08-07 (High)2025-08-07
Introducing GPT-5.2-Codex41.04gpt-5.2-codex2026-01-14
Claude Haiku 4.539.45claude-4-5-haiku2025-10-15
Qwen3-Coder: Agentic Coding in the World✓ Link38.70qwen3-coder-480b-a35b2025-07-22
MiniMax-M2.1✓ Link36.81minimax-2.12025-12-20
Gemini 3 Flash34.63gemini-3-flash2025-12-17
Introducing GPT-5.229.94gpt-5.22025-12-11
Kimi K2: Open Agentic Intelligence✓ Link27.67kimi-k2-instruct2025-07-28
Qwen3 Technical Report✓ Link21.41qwen3-235b-a22b2025-05-14
Introducing gpt-oss✓ Link16.20gpt-oss-120b2025-08-05
DeepSeek-V3.2 Technical Report✓ Link15.56deepseek-v3p22025-12-02
Gemma 3 Technical Report✓ Link11.38gemma-3-27b-it2025-03-25
Introducing Llama 3.1: Our most capable models to date✓ Link11.18llama3-1-405b-instruct2024-07-23
GLM-4.6✓ Link9.67glm-4.62025-09-30
The Llama 4 herd✓ Link5.24llama4-maverick-17b-instruct2025-04-05
Codestral: Hello, World!✓ Link1.51codestral-24052024-05-29
Check out how to contribute new results. Then edit this file.