OpenCodePapers

agents-on-automationbench-public

Agents
Dataset Link
Results over time
Click legend items to toggle metrics. Hover points for model names.
Leaderboard
PaperCodeSuccess RateModelNameReleaseDate
DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression✓ Link60.5DeepSeek-V4.1-Flash-10172026-09-17
DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression✓ Link54.8DeepSeek-V4.1-Flash Max2026-09-17
Atria Dawn: The Dawn of Agentic Superintelligence✓ Link53.8Atria Dawn Preview2026-09-14
MiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement✓ Link53.1MiMo-V2.6-Pro-RL2026-10-08
GLM-5.3-Flash: Frontier Intelligence, Flash Cost✓ Link48.8GLM-5.3-Flash2026-08-26
GLM-5.3: Frontier Coding with Emergent Cyber Capabilities✓ Link48.2GLM-5.32026-08-14
REMORY: Learning Residual Memory for Context Compaction✓ Link45.3Qwen3.8-27B + REMORY2026-10-08
Introducing Beam: Reflection's 501B open-weight model37.0Beam2026-10-05
CompoWorld: Compositional Environment Scaling for General Agents✓ Link32.33CompoWorld2026-09-27
Hy4 preview✓ Link32.1Hy4 preview2026-08-28
DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence✓ Link31.8DeepSeek-V4-Pro-0813 (Max)2026-04-26
WorkRover: Scaling Long-Horizon Cowork Agent Training Via Recursive Environment Evolution28.67WorkRover-35B-A3B2026-10-13
LFM2.5-24B-A2B: Built for Fast On-Device Agents27.69LFM2.5-24B-A2B2026-10-13
Occamy-1.0: Open Pareto-frontier 35B Intelligence for Co-work✓ Link27.6Occamy-1.02026-09-04
Introducing K2 Horizon: Frontier Performance, Radically Open✓ Link25.3K2-Horizon-375B-A23B2026-09-03
DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence✓ Link25.1DeepSeek-V4-Flash-0731 (Max)2026-04-26
Interlock: From Local Correctness to Task Completion in Long-Horizon Agents18.33Qwen3.6-27B (Interlock-RFT)2026-10-14
Skill2Env: Capability-Oriented Environment Synthesis from Skills for General Agents17.7Skill2Env (ours / 35B-A3B)2026-09-27
Trajectory-Level Speculation with Semantic Verification for Tool-Using Agents✓ Link10.5SpecAgent2026-10-15