| code-generation-on-frontiercode-1-1-extended | Claude Opus 5.5 (Claude Code) | #1 | Score: 65.27Pass Rate: 70.8 |
| code-generation-on-frontiercode-1-1-extended | Claude Fable 5 (Claude Code) | #2 | Score: 64.94Pass Rate: 70.86 |
| code-generation-on-frontiercode-1-1-extended | GPT-6 Astra (Codex) | #3 | Score: 64.48Pass Rate: 70.63 |
| code-generation-on-frontiercode-1-1-extended | Claude Sonnet 5.5 (Claude Code) | #4 | Score: 64.43Pass Rate: 69.87 |
| code-generation-on-frontiercode-1-1-extended | Claude Opus 5 (Claude Code) | #5 | Score: 63.63Pass Rate: 69.63 |
| code-generation-on-frontiercode-1-1-extended | Gemini 4 Argon (Chisel) | #6 | Score: 60.58Pass Rate: 66.0 |
| code-generation-on-frontiercode-1-1-extended | GPT-5.6 Sol (Codex) | #7 | Score: 60.55Pass Rate: 66.61 |
| code-generation-on-frontiercode-1-1-extended | Kimi K3 (MiniSWE-Agent) | #8 | Score: 58.19Pass Rate: 63.58 |
| code-generation-on-frontiercode-1-1-extended | DeepSeek V4.1 Flash (Chisel) | #9 | Score: 56.62Pass Rate: 62.03 |
| code-generation-on-frontiercode-1-1-extended | Gemini 3.7 Flash (Chisel) | #10 | Score: 56.27Pass Rate: 62.33 |
| code-generation-on-frontiercode-1-1-extended | GPT-5.6 Terra (Codex) | #11 | Score: 55.84Pass Rate: 61.75 |
| code-generation-on-frontiercode-1-1-extended | GPT-5.6 Luna (Codex) | #12 | Score: 55.06Pass Rate: 60.89 |
| code-generation-on-frontiercode-1-1-extended | Gemini 3.8 Flash (Chisel) | #13 | Score: 53.45Pass Rate: 59.62 |
| code-generation-on-frontiercode-1-1-extended | GLM 5.3 (Chisel) | #14 | Score: 52.63Pass Rate: 57.73 |
| code-generation-on-frontiercode-1-1-extended | GLM 5.3 Flash (Chisel) | #15 | Score: 46.21Pass Rate: 51.08 |
| code-generation-on-frontiercode-1-1-extended | DeepSeek V4 Pro 0813 (Chisel) | #16 | Score: 44.45Pass Rate: 48.93 |
| code-generation-on-frontiercode-1-1-extended | GLM 5.2 (MiniSWE-Agent) | #17 | Score: 40.05Pass Rate: 44.13 |
| code-generation-on-frontiercode-1-1-extended | DeepSeek V4 Flash 0731 (Chisel) | #18 | Score: 31.7Pass Rate: 34.96 |
| code-generation-on-frontiercode-1-1-extended | DeepSeek V4 Pro (MiniSWE-Agent) | #19 | Score: 31.01Pass Rate: 34.49 |
| code-generation-on-frontiercode-1-1-extended | MiniMax M3 (MiniSWE-Agent) | #20 | Score: 28.54Pass Rate: 31.46 |
| code-generation-on-frontiercode-1-1-extended | Nemotron 3 Ultra (Chisel) | #21 | Score: 27.21Pass Rate: 30.27 |
| code-generation-on-frontiercode-1-1-extended | Inkling (MiniSWE-Agent) | #22 | Score: 24.8Pass Rate: 27.8 |
| code-generation-on-frontiercode-1-1-extended | Claude Fable 5.1 (Claude Code) | – | Score: 63.6Pass Rate: 68.8 |
| code-generation-on-frontiercode-1-1-extended | Claude Haiku 5.5 (Claude Code) | – | Score: 58.43Pass Rate: 64.7 |
| code-generation-on-frontiercode-1-1-extended | Claude Opus 4.6 (Claude Code) | – | Score: 43.67Pass Rate: 48.0 |
| code-generation-on-frontiercode-1-1-extended | Claude Opus 4.7 (Claude Code) | – | Score: 53.93Pass Rate: 59.09 |
| code-generation-on-frontiercode-1-1-extended | Claude Opus 4.8 (Claude Code) | – | Score: 59.6Pass Rate: 65.49 |
| code-generation-on-frontiercode-1-1-extended | Claude Sonnet 4.6 (Claude Code) | – | Score: 40.0Pass Rate: 44.4 |
| code-generation-on-frontiercode-1-1-extended | Claude Sonnet 5 (Claude Code) | – | Score: 56.18Pass Rate: 61.66 |
| code-generation-on-frontiercode-1-1-extended | Composer 2.5 (Cursor CLI) | – | Score: 40.75Pass Rate: 45.13 |
| code-generation-on-frontiercode-1-1-extended | GPT-5.4-mini (Codex) | – | Score: 43.01Pass Rate: 47.94 |
| code-generation-on-frontiercode-1-1-extended | GPT-5.5 (Codex) | – | Score: 56.66Pass Rate: 62.76 |
| code-generation-on-frontiercode-1-1-extended | GPT-6 Luna (Codex) | – | Score: 56.1Pass Rate: 62.27 |
| code-generation-on-frontiercode-1-1-extended | GPT-6 Sol (Codex) | – | Score: 60.69Pass Rate: 66.33 |
| code-generation-on-frontiercode-1-1-extended | GPT-6.1 Sol (Codex) | – | Score: 60.4Pass Rate: 66.53 |
| code-generation-on-frontiercode-1-1-extended | Gemini 3.6 Flash (Chisel) | – | Score: 48.02Pass Rate: 53.27 |
| code-generation-on-frontiercode-1-1-extended | Grok 4.5 (Grok Build) | – | Score: 56.55Pass Rate: 62.27 |
| code-generation-on-frontiercode-1-1-extended | Grok 4.6 (Grok Build) | – | Score: 61.31Pass Rate: 67.05 |
| code-generation-on-frontiercode-1-1-extended | Grok 4.7 (Grok Build) | – | Score: 59.35Pass Rate: 65.23 |
| code-generation-on-frontiercode-1-1-extended | Kimi K2.7 (MiniSWE-Agent) | – | Score: 45.38Pass Rate: 50.0 |
| code-generation-on-frontiercode-1-1-extended | Mistral 3.5 Medium (Chisel) | – | Score: 18.66Pass Rate: 20.6 |
| code-generation-on-frontiercode-1-1-extended | Mistral Large 4 (Preview) (Chisel) | – | Score: 53.21Pass Rate: 58.07 |
| code-generation-on-frontiercode-1-1-extended | Qwen 3.7 Plus (MiniSWE-Agent) | – | Score: 21.78Pass Rate: 24.13 |
| code-generation-on-frontiercode-1-1-extended | SWE-1.6 (Chisel) | – | Score: 20.5Pass Rate: 22.67 |
| code-generation-on-frontiercode-1-1-extended | SWE-1.7 (Chisel) | – | Score: 54.26Pass Rate: 60.33 |
| code-generation-on-frontiercode-1-1-extended | SWE-2 (devin) | – | Score: 62.46Pass Rate: 68.44 |
| code-generation-on-frontiercode-1-1-main | Claude Opus 5.5 (Claude Code) | #1 | Score: 54.64Pass Rate: 59.6 |
| code-generation-on-frontiercode-1-1-main | Claude Fable 5 (Claude Code) | #2 | Score: 53.48Pass Rate: 58.85 |
| code-generation-on-frontiercode-1-1-main | Claude Opus 5 (Claude Code) | #3 | Score: 53.38Pass Rate: 58.85 |
| code-generation-on-frontiercode-1-1-main | GPT-6 Astra (Codex) | #4 | Score: 53.26Pass Rate: 58.75 |
| code-generation-on-frontiercode-1-1-main | Claude Sonnet 5.5 (Claude Code) | #5 | Score: 52.09Pass Rate: 57.2 |
| code-generation-on-frontiercode-1-1-main | Gemini 4 Argon (Chisel) | #6 | Score: 47.55Pass Rate: 52.8 |
| code-generation-on-frontiercode-1-1-main | GPT-5.6 Sol (Codex) | #7 | Score: 47.49Pass Rate: 52.91 |
| code-generation-on-frontiercode-1-1-main | Kimi K3 (MiniSWE-Agent) | #8 | Score: 44.17Pass Rate: 48.91 |
| code-generation-on-frontiercode-1-1-main | DeepSeek V4.1 Flash (Chisel) | #9 | Score: 43.87Pass Rate: 48.63 |
| code-generation-on-frontiercode-1-1-main | Gemini 3.7 Flash (Chisel) | #10 | Score: 43.59Pass Rate: 48.9 |
| code-generation-on-frontiercode-1-1-main | GPT-5.6 Terra (Codex) | #11 | Score: 41.31Pass Rate: 46.32 |
| code-generation-on-frontiercode-1-1-main | Gemini 3.8 Flash (Chisel) | #12 | Score: 41.19Pass Rate: 46.73 |
| code-generation-on-frontiercode-1-1-main | GLM 5.3 (Chisel) | #13 | Score: 40.14Pass Rate: 44.7 |
| code-generation-on-frontiercode-1-1-main | GPT-5.6 Luna (Codex) | #14 | Score: 39.81Pass Rate: 44.74 |
| code-generation-on-frontiercode-1-1-main | GLM 5.3 Flash (Chisel) | #15 | Score: 31.83Pass Rate: 35.67 |
| code-generation-on-frontiercode-1-1-main | DeepSeek V4 Pro 0813 (Chisel) | #16 | Score: 28.55Pass Rate: 31.8 |
| code-generation-on-frontiercode-1-1-main | GLM 5.2 (MiniSWE-Agent) | #17 | Score: 24.5Pass Rate: 27.4 |
| code-generation-on-frontiercode-1-1-main | DeepSeek V4 Flash 0731 (Chisel) | #18 | Score: 18.8Pass Rate: 21.08 |
| code-generation-on-frontiercode-1-1-main | DeepSeek V4 Pro (MiniSWE-Agent) | #19 | Score: 17.64Pass Rate: 20.0 |
| code-generation-on-frontiercode-1-1-main | MiniMax M3 (MiniSWE-Agent) | #20 | Score: 14.7Pass Rate: 16.64 |
| code-generation-on-frontiercode-1-1-main | Inkling (MiniSWE-Agent) | #21 | Score: 14.0Pass Rate: 15.9 |
| code-generation-on-frontiercode-1-1-main | Nemotron 3 Ultra (Chisel) | #22 | Score: 13.61Pass Rate: 15.4 |
| code-generation-on-frontiercode-1-1-main | Claude Fable 5.1 (Claude Code) | – | Score: 50.91Pass Rate: 55.45 |
| code-generation-on-frontiercode-1-1-main | Claude Haiku 5.5 (Claude Code) | – | Score: 46.36Pass Rate: 51.65 |
| code-generation-on-frontiercode-1-1-main | Claude Opus 4.6 (Claude Code) | – | Score: 26.64Pass Rate: 29.7 |
| code-generation-on-frontiercode-1-1-main | Claude Opus 4.7 (Claude Code) | – | Score: 38.54Pass Rate: 42.83 |
| code-generation-on-frontiercode-1-1-main | Claude Opus 4.8 (Claude Code) | – | Score: 46.5Pass Rate: 51.65 |
| code-generation-on-frontiercode-1-1-main | Claude Sonnet 4.6 (Claude Code) | – | Score: 24.31Pass Rate: 27.5 |
| code-generation-on-frontiercode-1-1-main | Claude Sonnet 5 (Claude Code) | – | Score: 42.73Pass Rate: 47.59 |
| code-generation-on-frontiercode-1-1-main | Composer 2.5 (Cursor CLI) | – | Score: 25.64Pass Rate: 29.34 |
| code-generation-on-frontiercode-1-1-main | GPT-5.4-mini (Codex) | – | Score: 27.04Pass Rate: 30.8 |
| code-generation-on-frontiercode-1-1-main | GPT-5.5 (Codex) | – | Score: 42.96Pass Rate: 48.24 |
| code-generation-on-frontiercode-1-1-main | GPT-6 Luna (Codex) | – | Score: 42.42Pass Rate: 47.8 |
| code-generation-on-frontiercode-1-1-main | GPT-6 Sol (Codex) | – | Score: 49.27Pass Rate: 54.3 |
| code-generation-on-frontiercode-1-1-main | GPT-6.1 Sol (Codex) | – | Score: 50.23Pass Rate: 55.8 |
| code-generation-on-frontiercode-1-1-main | Gemini 3.6 Flash (Chisel) | – | Score: 34.37Pass Rate: 38.9 |
| code-generation-on-frontiercode-1-1-main | Grok 4.5 (Grok Build) | – | Score: 42.44Pass Rate: 47.2 |
| code-generation-on-frontiercode-1-1-main | Grok 4.6 (Grok Build) | – | Score: 48.01Pass Rate: 53.15 |
| code-generation-on-frontiercode-1-1-main | Grok 4.7 (Grok Build) | – | Score: 47.59Pass Rate: 53.1 |
| code-generation-on-frontiercode-1-1-main | Kimi K2.7 (MiniSWE-Agent) | – | Score: 30.06Pass Rate: 33.6 |
| code-generation-on-frontiercode-1-1-main | Mistral 3.5 Medium (Chisel) | – | Score: 8.0Pass Rate: 9.0 |
| code-generation-on-frontiercode-1-1-main | Mistral Large 4 (Preview) (Chisel) | – | Score: 39.43Pass Rate: 43.5 |
| code-generation-on-frontiercode-1-1-main | Qwen 3.7 Plus (MiniSWE-Agent) | – | Score: 10.2Pass Rate: 11.46 |
| code-generation-on-frontiercode-1-1-main | SWE-1.6 (Chisel) | – | Score: 9.39Pass Rate: 10.45 |
| code-generation-on-frontiercode-1-1-main | SWE-1.7 (Chisel) | – | Score: 41.99Pass Rate: 47.4 |
| code-generation-on-frontiercode-1-1-main | SWE-2 (devin) | – | Score: 50.0Pass Rate: 55.47 |