| LLaDA2.0: Scaling Up Diffusion Language Models to 100B | ✓ Link | 95.44 | | LLaDA2.0-flash | 2025-12-10 |
| Heterogeneous Agent Collaborative Reinforcement Learning | | 94.3 | | Qwen3-4B-Instruct (HACPO) | 2026-03-03 |
| Self-Distillation Zero: Self-Revision Turns Binary Rewards into Dense Supervision | ✓ Link | 94.0 | | SD-Zero (Olmo-3-7B-Instruct) | 2026-04-13 |
| Ministral 3 | | 90.4 | | Ministral 3 (14B, Instruct) | 2026-01-13 |
| | 89.7 | | Gemini 2.0 Flash Experimental | |
| Gemma 3 Technical Report | ✓ Link | 89.0 | | Gemma 3 (27B, IT) (4-shot) | 2025-03-25 |
| Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement | | 88.1 | 72 | Qwen2.5-Math-72B-Instruct(TIR,Greedy) | 2024-09-18 |
| MACM: Utilizing a Multi-Agent System for Condition Mining in Solving Complex Mathematical Problems | ✓ Link | 87.920 | | GPT-4 Turbo (MACM, w/code, voting) | 2024-04-06 |
| You Only Need Minimal RLVR Training: Extrapolating LLMs via Rank-1 Trajectories | ✓ Link | 87.4 | | RELEX (Qwen3-8B-Base) | 2026-05-20 |
| Value-Aware Stochastic KV Cache Eviction for Reasoning Models | ✓ Link | 86.47 | | VaSE-DKV (Qwen3-14B) | 2026-06-02 |
| Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement | | 85.9 | 72 | Qwen2.5-Math-72B-Instruct(COT,Greedy) | 2024-09-18 |
| On the Step Length Confounding in LLM Reasoning Data Selection | ✓ Link | 85.4 | | Qwen3-8B-Base (Aslec-casl) | 2026-04-08 |
| Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement | | 85.2 | 7 | Qwen2.5-Math-7B-Instruct(TIR,Greedy) | 2024-09-18 |
| dots.llm1 Technical Report | ✓ Link | 85.0 | | dots.llm1.inst | 2025-06-06 |
| Value-Aware Stochastic KV Cache Eviction for Reasoning Models | ✓ Link | 84.72 | | VaSE-DKV (Qwen3-4B) | 2026-06-02 |
| SAGE: Mitigating Long-Horizon Reasoning Biases via Topological Guidance | ✓ Link | 84.72 | | Qwen3.5-35B w/SAGE | 2026-09-24 |
| Qwen2.5 Technical Report | ✓ Link | 84.7 | | Qwen2.5-Plus | 2024-12-19 |
| STRIDE: Learnable Stepwise Language Feedback for LLM Reasoning | | 84.6 | | STRIDE-Qwen-7B | 2026-05-13 |
| Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verification | ✓ Link | 84.3 | | GPT-4-code model (CSV, w/ code, SC, k=16) | 2023-08-15 |
| Qwen2 Technical Report | ✓ Link | 84.0 | 72 | Qwen2-Math-72B-Instruct(greedy) | 2024-07-15 |
| From Next-Token to Next-Block: A Principled Adaptation Path for Diffusion LLMs | ✓ Link | 84.0 | | NBDiff-7B (Instruct) | 2025-12-07 |
| Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement | | 83.6 | 7 | Qwen2.5-Math-7B-Instruct(COT,Greedy) | 2024-09-18 |
| Qwen2.5-VL Technical Report | ✓ Link | 83.0 | | Qwen2.5-VL-72B | 2025-02-19 |
| DARC: Decoupled Asymmetric Reasoning Curriculum for LLM Evolution | ✓ Link | 83.0 | | DARC (Qwen3-8B-Base) | 2026-01-20 |
| Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning | ✓ Link | 82.88 | | Nemotron 3 Nano 30B-A3B Base | 2025-12-23 |
| Beyond Policy Rollouts: Embedding-Space Directional Exploration with Anchored Noise for LLM RLVR | | 81.8 | | EDEN (Qwen2.5-Math-7B) | 2026-10-12 |
| LLaDA MoE v2: Scaling Mixture-of-Experts Diffusion Language Models | | 80.02 | | LLaDA MoE v2 Instruct | 2026-08-04 |
| Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement | | 79.9 | 1.5 | Qwen2.5-Math-1.5B-Instruct(TIR,Greedy) | 2024-09-18 |
| OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data | ✓ Link | 79.6 | | OpenMath2-Llama3.1-70B (majority@256) | 2024-10-02 |
| InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models | ✓ Link | 78.9 | | InternVL3-78B | 2025-04-14 |
| Questioning the Questions: Sustaining Self-Evolution in Reasoning Models | ✓ Link | 77.6 | | R-Quest (Qwen3-4B-Base) | 2026-10-03 |
| MiniMax-01: Scaling Foundation Models with Lightning Attention | ✓ Link | 77.4 | | MiniMax-Text-01 | 2025-01-14 |
| Hello GPT-4o | | 76.6 | | GPT-4o | 2024-05-13 |
| OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data | ✓ Link | 76.1 | | OpenMath2-Llama3.1-8B (majority@256) | 2024-10-02 |
| Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement | | 75.8 | 1.5 | Qwen2.5-Math-1.5B-Instruct(COT,Greedy) | 2024-09-18 |
| ERNIE 5.0 Technical Report | | 73.89 | | ERNIE 5.0-Base | 2026-02-04 |
| Introducing Llama 3.1: Our most capable models to date | ✓ Link | 73.8 | | Llama 3.1 (405B, Instruct) | 2024-07-23 |
| Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verification | ✓ Link | 73.5 | | GPT-4-code model (CSV, w/ code) | 2023-08-15 |
| On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability | ✓ Link | 72.78 | | Qwen3.8-Flash-Next-Base | 2026-08-31 |
| Cumulative Reasoning with Large Language Models | ✓ Link | 72.2 | | CR (GPT-4-turbo model, w/ code) | 2023-08-08 |
| OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data | ✓ Link | 71.9 | | OpenMath2-Llama3.1-70B | 2024-10-02 |
| Qwen3 Technical Report | ✓ Link | 71.84 | | Qwen3-235B-A22B (Base) | 2025-05-14 |
| Qwen2.5-Omni Technical Report | ✓ Link | 71.5 | | Qwen2.5-Omni-7B | 2025-03-26 |
| Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verification | ✓ Link | 71.2 | | LogicNet (with code interpreter) | 2023-08-15 |
| Introducing Claude 3.5 Sonnet | | 71.1 | | Claude 3.5 Sonnet | 2024-06-20 |
| MiMo-V2-Flash Technical Report | ✓ Link | 71.0 | | MiMo-V2-Flash (Base) | 2026-01-06 |
| Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs | ✓ Link | 70.8 | | Qwen2-72B-Instruct-Step-DPO (0-shot CoT, w/o code) | 2024-06-26 |
| Motif 3: Technical Report | ✓ Link | 70.58 | | Motif 3 Base (4-shot CoT) | 2026-08-10 |
| STRIDE: Learnable Stepwise Language Feedback for LLM Reasoning | | 70.4 | | STRIDE-Llama-8B | 2026-05-13 |
| GPT-4o mini: advancing cost-efficient intelligence | ✓ Link | 70.2 | | GPT-4o mini | 2024-07-18 |
| Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verification | ✓ Link | 69.7 | | GPT-4-code model (w/ code) | 2023-08-15 |
| Introducing Llama 3.1: Our most capable models to date | ✓ Link | 68.0 | | Llama 3.1 (70B, Instruct) | 2024-07-23 |
| Llama 3.2: Revolutionizing edge AI and vision with open, customizable models | ✓ Link | 68.0 | | Llama 3.2 90B Vision Instruct | 2024-09-25 |
| OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data | ✓ Link | 67.8 | | OpenMath2-Llama3.1-8B | 2024-10-02 |
| Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context | ✓ Link | 67.7 | | Gemini 1.5 Pro (4-shot) | 2024-03-08 |
| LongCat-Flash-Omni Technical Report | ✓ Link | 66.8 | | LongCat-Flash-Omni (Base) | 2025-10-31 |
| Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters | ✓ Link | 66.8 | | Step-3.5-Flash Base | 2026-02-11 |
| AlphaMath Almost Zero: Process Supervision without Process | ✓ Link | 66.3 | | AlphaMath-7B-SBS@3 | 2024-05-06 |
| Solving Quantitative Reasoning Problems with Language Models | ✓ Link | 64.9 | 62 | Minerva 62B (maj5@256) | 2022-06-29 |
| | 64.5 | 7 | DAMOMath-7B | |
| Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs | ✓ Link | 64.0 | | Phi-4-Mini (0-shot, CoT) | 2025-02-24 |
| An Empirical Study of Data Ability Boundary in LLMs' Math Reasoning | ✓ Link | 63.7 | 7 | MMOS-DeepSeekMath-7B(0-shot,k=50) | 2024-02-23 |
| daVinci-LLM:Towards the Science of Pretraining | ✓ Link | 62.8 | | daVinci-LLM-3B | 2026-03-28 |
| Every Token Counts: Generalizing 16M Ultra-Long Context in Large Language Models | ✓ Link | 61.56 | | HSA-UltraLong-Inst | 2025-11-28 |
| ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools | ✓ Link | 61.3 | | GLM-4 (0520) | 2024-06-18 |
| Can Large Models Teach Student Models to Solve Mathematical Problems Like Human Beings? A Reasoning Distillation Method via Multi-LoRA Interaction | ✓ Link | 61.2 | | LoRID (Qwen2.5-Math) | 2025-08-18 |
| DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression | ✓ Link | 61.1 | | DeepSeek-V4.1-Flash Base | 2026-09-17 |
| Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verification | ✓ Link | 60.8 | | GPT-4-code model (w/o code) | 2023-08-15 |
| OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset | ✓ Link | 60.4 | 70 | OpenMath-CodeLlama-70B (w/ code, SC, k=50) | 2024-02-15 |
| OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset | ✓ Link | 60.2 | 34 | OpenMath-CodeLlama-34B (w/ code, SC, k=50) | 2024-02-15 |
| Introducing the next generation of Claude | | 60.1 | | Claude 3 Opus | 2024-03-04 |
| ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving | ✓ Link | 60.0 | 34 | ToRA-Code 34B model (w/ code, SC, k=50) | 2023-09-29 |
| Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone | ✓ Link | 59.5 | | Phi-3.5-MoE | 2024-04-22 |
| Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Models | ✓ Link | 59.4 | | Nemotron-H-56B-Base (4-shot) | 2025-04-04 |
| DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models | ✓ Link | 58.8 | 7 | DeepSeekMATH-RL-7B (w/ code, greedy decoding) | 2024-02-05 |
| Laguna M.1/XS.2 Technical Report | ✓ Link | 58.8 | | Laguna XS.2 | 2026-05-26 |
| OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset | ✓ Link | 58.3 | 70 | OpenMath-Llama2-70B (w/ code, SC, k=50) | 2024-02-15 |
| Cumulative Reasoning with Large Language Models | ✓ Link | 58.0 | | CR (GPT-4 model, w/o code) | 2023-08-08 |
| OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset | ✓ Link | 57.6 | 13 | OpenMath-CodeLlama-13B (w/ code, SC, k=50) | 2024-02-15 |
| OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset | ✓ Link | 57.2 | 7 | OpenMath-Mistral-7B (w/ code, SC, k=50) | 2024-02-15 |
| Attention Residuals | ✓ Link | 57.1 | | Kimi Linear (48B) with Block AttnRes | 2026-03-16 |
| ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving | ✓ Link | 56.9 | 70 | ToRA 70B (w/ code, SC, k=50) | 2023-09-29 |
| Improved Large Language Diffusion Models | ✓ Link | 56.7 | | iLLaDA 8B Instruct | 2026-06-24 |
| Skills-in-Context Prompting: Unlocking Compositionality in Large Language Models | | 56.4 | | SKiC (GPT-4 model) | 2023-08-01 |
| DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solving | ✓ Link | 56.1 | 70 | DART-Math-Llama3-70B-Prop2Diff (0-shot CoT, w/o code) | 2024-06-18 |
| OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset | ✓ Link | 55.6 | 7 | OpenMath-CodeLlama-7B (w/ code, SC, k=50) | 2024-02-15 |
| EpiCaR: Knowing What You Don't Know Matters for Better Reasoning in LLMs | | 55.56 | | EpiCaR (Qwen-3, 8B) + Slow Thinking (ICL) | 2026-01-11 |
| BEAM: Binary Expert Activation Masking for Dynamic Routing in MoE | | 55.16 | | Qwen3-30B-A3B (BEAM, beta=0.01) | 2026-05-14 |
| An Empirical Study of Data Ability Boundary in LLMs' Math Reasoning | ✓ Link | 55.0 | 7 | MMOS-DeepSeekMath-7B(0-shot) | 2024-02-23 |
| DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solving | ✓ Link | 54.9 | 70 | DART-Math-Llama3-70B-Uniform (0-shot CoT, w/o code) | 2024-06-18 |
| Can Large Models Teach Student Models to Solve Mathematical Problems Like Human Beings? A Reasoning Distillation Method via Multi-LoRA Interaction | ✓ Link | 54.8 | | LoRID (DeepSeekMath) | 2025-08-18 |
| ReFusion: A Diffusion Large Language Model with Parallel Autoregressive Decoding | ✓ Link | 54.22 | | ReFusion | 2025-12-15 |
| Progressive-Hint Prompting Improves Reasoning in Large Language Models | ✓ Link | 53.9 | | PHP (GPT-4 model) | 2023-04-19 |
| DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model | ✓ Link | 53.9 | | DeepSeek-V2 Chat (RL) (4-shot) | 2024-05-07 |
| DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solving | ✓ Link | 53.6 | 7 | DART-Math-DSMath-7B-Prop2Diff (0-shot CoT, w/o code) | 2024-06-18 |
| Gemini: A Family of Highly Capable Multimodal Models | ✓ Link | 53.2 | | Gemini Ultra (4-shot) | 2023-12-19 |
| DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solving | ✓ Link | 52.9 | 7 | DART-Math-DSMath-7B-Uniform (0-shot CoT, w/o code) | 2024-06-18 |
| Introducing Llama 3.1: Our most capable models to date | ✓ Link | 51.9 | | Llama 3.1 (8B, Instruct) | 2024-07-23 |
| The Llama 3 Herd of Models | ✓ Link | 51.9 | | Llama 3.1 (8B, Instruct) | 2024-07-31 |
| PAL: Program-aided Language Models | ✓ Link | 51.8 | | GPT-4 model (w/ code, PAL) | 2022-11-18 |
| DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models | ✓ Link | 51.7 | 7 | DeepSeekMATH-RL-7B (greedy decoding) | 2024-02-05 |
| Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing | ✓ Link | 51 | | AlphaLLM (with MCTS) | 2024-04-18 |
| ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving | ✓ Link | 50.8 | 34 | ToRA-Code 34B (w/ code) | 2023-09-29 |
| Aria: An Open Multimodal Native Mixture-of-Experts Model | ✓ Link | 50.8 | | Aria | 2024-10-08 |
| OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset | ✓ Link | 50.7 | 70 | OpenMath-CodeLlama-70B (w/ code) | 2024-02-15 |
| CharacterFlywheel: Scaling Iterative Improvement of Engaging and Steerable LLMs in Production | | 50.5 | | CharacterFlywheel V7 (Llama 3.1 70B) | 2026-03-02 |
| Solving Quantitative Reasoning Problems with Language Models | ✓ Link | 50.3 | | Minerva 540B (maj1@k, k=64) | 2022-06-29 |
| FourierSampler: Unlocking Non-Autoregressive Potential in Diffusion Language Models via Frequency-Guided Generation | ✓ Link | 50.0 | | SDAR-4B-Chat (with FourierSampler) | 2026-01-30 |
| ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving | ✓ Link | 49.7 | 70 | ToRA 70B (w/ code) | 2023-09-29 |
| 2 OLMo 2 Furious | ✓ Link | 49.7 | | OLMo 2 32B Instruct (4-shot) | 2024-12-31 |
| Dynin-Omni: Omnimodal Unified Large Diffusion Language Model | ✓ Link | 49.6 | | Dynin-Omni (8B) | 2026-03-09 |
| An Empirical Study of Data Ability Boundary in LLMs' Math Reasoning | ✓ Link | 49.5 | 34 | MMOS-CODE-34B(0-shot) | 2024-02-23 |
| PaLM 2 Technical Report | ✓ Link | 48.8 | | PaLM 2 (few-shot, k=4, SC) | 2023-05-17 |
| Key-Point-Driven Data Synthesis with its Enhancement on Mathematical Reasoning | | 48.8 | 7 | DeepSeekMath-7B-KPMath-Plus | 2024-03-04 |
| Key-Point-Driven Data Synthesis with its Enhancement on Mathematical Reasoning | | 48.6 | 34 | Llemma-34B-KPMath-Plus | 2024-03-04 |
| OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset | ✓ Link | 48.3 | 34 | OpenMath-CodeLlama-34B (w/ code) | 2024-02-15 |
| ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving | ✓ Link | 48.1 | 13 | ToRA-Code 13B (w/ code) | 2023-09-29 |
| Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations | ✓ Link | 48.1 | 67 | Shepherd + DeepSeek-67B (SFT on MetaMATH + PRM rerank, k=256) | 2023-12-14 |
| Pixtral 12B | ✓ Link | 48.1 | | Pixtral 12B | 2024-10-09 |
| Llama 3.2: Revolutionizing edge AI and vision with open, customizable models | ✓ Link | 48.0 | | Llama 3.2 3B Instruct | 2024-09-25 |
| Solving Quantitative Reasoning Problems with Language Models | ✓ Link | 47.6 | 8 | Minerva 8B (maj5@256) | 2022-06-29 |
| Key-Point-Driven Data Synthesis with its Enhancement on Mathematical Reasoning | | 46.8 | 7 | Mistral-7B-KPMath-Plus | 2024-03-04 |
| DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solving | ✓ Link | 46.6 | 8 | DART-Math-Llama3-8B-Prop2Diff (0-shot CoT, w/o code) | 2024-06-18 |
| Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Models | ✓ Link | 46.5 | | Nemotron-H-8B-Base (4-shot) | 2025-04-04 |
| OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset | ✓ Link | 46.3 | 70 | OpenMath-Llama2-70B (w/ code) | 2024-02-15 |
| SmolLM3: smol, multilingual, long-context reasoner | ✓ Link | 46.1 | | SmolLM3 3B Base (4-shot) | 2025-07-08 |
| OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset | ✓ Link | 45.5 | 13 | OpenMath-CodeLlama-13B (w/ code) | 2024-02-15 |
| DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solving | ✓ Link | 45.5 | 7 | DART-Math-Mistral-7B-Prop2Diff (0-shot CoT, w/o code) | 2024-06-18 |
| DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solving | ✓ Link | 45.3 | 8 | DART-Math-Llama3-8B-Uniform (0-shot CoT, w/o code) | 2024-06-18 |
| MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning | ✓ Link | 45.2 | 34 | MathCoder-CL-34B | 2023-10-05 |
| MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning | ✓ Link | 45.1 | 34 | MathCoder-L-34B | 2023-10-05 |
| Augmenting Math Word Problems via Iterative Question Composing | ✓ Link | 45.0 | 72 | MMIQC-72B | 2024-01-17 |
| ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving | ✓ Link | 44.6 | 7 | ToRA-Code 7B (w/ code) | 2023-09-29 |
| OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset | ✓ Link | 44.5 | 7 | OpenMath-Mistral-7B (w/ code) | 2024-02-15 |
| Youtu-LLM: Unlocking the Native Agentic Potential for Lightweight Large Language Models | ✓ Link | 44.4 | | Youtu-LLM (2B, base) | 2025-12-31 |
| An Empirical Study of Data Ability Boundary in LLMs' Math Reasoning | ✓ Link | 44.3 | 7 | MMOS-CODE-7B(0-shot) | 2024-02-23 |
| OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset | ✓ Link | 43.6 | 7 | OpenMath-CodeLlama-7B (w/ code) | 2024-02-15 |
| Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations | ✓ Link | 43.5 | 7 | Shepherd+Mistral-7B (SFT on MetaMATH + PRM RL+ PRM rerank, k=256) | 2023-12-14 |
| DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solving | ✓ Link | 43.5 | 7 | DART-Math-Mistral-7B-Uniform (0-shot CoT, w/o code) | 2024-06-18 |
| Solving Quantitative Reasoning Problems with Language Models | ✓ Link | 43.4 | 62 | Minerva 62B (maj1@k, k=64) | 2022-06-29 |
| Llama-3.1-FoundationAI-SecurityLLM-Reasoning-8B Technical Report | | 43.3 | | Foundation-Sec-8B-Reasoning (8B) | 2026-01-28 |
| Introducing the next generation of Claude | | 43.1 | | Claude 3 Sonnet | 2024-03-04 |
| ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving | ✓ Link | 43.0 | 13 | ToRA 13B (w/ code) | 2023-09-29 |
| Sparks of Artificial General Intelligence: Early experiments with GPT-4 | ✓ Link | 42.5 | | GPT-4 | 2023-03-22 |
| Gemma 2: Improving Open Language Models at a Practical Size | ✓ Link | 42.3 | | Gemma 2 27B (4-shot) | 2024-07-31 |
| Harder Is Better: Boosting Mathematical Reasoning via Difficulty-Aware GRPO and Multi-Aspect Question Reformulation | ✓ Link | 42.17 | | Qwen2.5-Math-7B (MathForge) | 2026-01-28 |
| | 41.8 | 7 | SFT-Mistral-7B | |
| Key-Point-Driven Data Synthesis with its Enhancement on Mathematical Reasoning | | 41 | 13 | Llama2-13B-KPMath-Plus | 2024-03-04 |
| ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving | ✓ Link | 40.1 | 7 | ToRA 7B (w/ code) | 2023-09-29 |
| 2 OLMo 2 Furious | ✓ Link | 39.2 | | OLMo 2 13B Instruct (4-shot) | 2024-12-31 |
| Introducing the next generation of Claude | | 38.9 | | Claude 3 Haiku | 2024-03-04 |
| One Sample to Rule Them All: Extreme Data Efficiency in RL Scaling | ✓ Link | 38.8 | | Qwen2.5-7b-base (Polymath Learning - Natural sample: Probability) | 2026-01-06 |
| Can Large Models Teach Student Models to Solve Mathematical Problems Like Human Beings? A Reasoning Distillation Method via Multi-LoRA Interaction | ✓ Link | 38.7 | | LoRID (Mistral) | 2025-08-18 |
| Improved Large Language Diffusion Models | ✓ Link | 38.4 | | iLLaDA 8B Base | 2026-06-24 |
| Queryable LoRA: Instruction-Regularized Routing Over Shared Low-Rank Update Atoms | | 37.5 | | Instruction-Queryable LoRA | 2026-05-08 |
| Flash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memory-Efficient Diffusion LLMs | ✓ Link | 36.56 | | Flash-dLLM | 2026-09-22 |
| MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning | ✓ Link | 35.9 | 13 | MathCoder-CL-13B | 2023-10-05 |
| MuggleMath: Assessing the Impact of Query and Response Augmentation on Math Reasoning | ✓ Link | 35.6 | 70 | MuggleMATH-70B | 2023-10-09 |
| PaLM 2 Technical Report | ✓ Link | 34.3 | | PaLM 2 (few-shot, k=4, CoT) | 2023-05-17 |
| Solving Quantitative Reasoning Problems with Language Models | ✓ Link | 33.6 | 540 | Minerva 540B | 2022-06-29 |
| Galactica: A Large Language Model for Science | ✓ Link | 33.6 | 540 | Minerva 540B (5-shot) mCoT | 2022-11-16 |
| WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct | ✓ Link | 33.0 | 7 | WizardMath-7B-V1.1 | 2023-08-18 |
| Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations | ✓ Link | 33.0 | 7 | Shepherd + Mistral-7B (SFT on MetaMATH + PRM RL) | 2023-12-14 |
| Gemini: A Family of Highly Capable Multimodal Models | ✓ Link | 32.6 | | Gemini Pro (4-shot) | 2023-12-19 |
| DeepSeek LLM: Scaling Open-Source Language Models with Longtermism | ✓ Link | 32.6 | | DeepSeek LLM 67B Chat | 2024-01-05 |
| 2 OLMo 2 Furious | ✓ Link | 32.5 | | OLMo 2 7B Instruct (4-shot) | 2024-12-31 |
| The Master Key Hypothesis: Unlocking Cross-Model Capability Transfer via Linear Subspace Alignment | ✓ Link | 31.2 | | Qwen1.5 (14B) + UNLOCK from 7B | 2026-04-07 |
| Apertus: Democratizing Open and Compliant LLMs for Global Language Environments | ✓ Link | 30.8 | | Apertus-70B-Instruct | 2025-09-17 |
| HySparse: A Hybrid Sparse Attention Architecture with Oracle Token Selection and KV Cache Sharing | | 30.8 | | HySparse (80B MoE) | 2026-02-03 |
| MuggleMath: Assessing the Impact of Query and Response Augmentation on Math Reasoning | ✓ Link | 30.7 | 13 | MuggleMATH-13B | 2023-10-09 |
| MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning | ✓ Link | 30.2 | 7 | MathCoder-CL-7B | 2023-10-05 |
| MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning | ✓ Link | 29.9 | 13 | MathCoder-L-13B | 2023-10-05 |
| Parameter-Efficient Sparsity Crafting from Dense to Mixture-of-Experts for Instruction Tuning on General Tasks | ✓ Link | 29.9 | | Qwen2idae-16x14B (4-shot) | 2024-01-05 |
| OpenChat: Advancing Open-source Language Models with Mixed-Quality Data | ✓ Link | 28.9 | 7 | OpenChat-3.5-1210 7B | 2023-09-20 |
| OpenChat: Advancing Open-source Language Models with Mixed-Quality Data | ✓ Link | 28.6 | 7 | OpenChat-3.5 7B | 2023-09-20 |
| Mixtral of Experts | ✓ Link | 28.4 | | Mixtral 8x7B (maj@4) | 2024-01-08 |
| Solving Quantitative Reasoning Problems with Language Models | ✓ Link | 27.6 | 62 | Minerva 62B (4-shot) | 2022-06-29 |
| River-LLM: Large Language Model Seamless Exit Based on KV Share | | 26.6 | | River-LLM (Llama3.1 8B, τ=0.5) | 2026-04-20 |
| Coupling Experts and Routers in Mixture-of-Experts via an Auxiliary Loss | | 26.1 | | MoE + ERC | 2025-12-29 |
| MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models | ✓ Link | 26.0 | 70 | MetaMath 70B | 2023-09-21 |
| mHC: Manifold-Constrained Hyper-Connections | | 26.0 | | mHC (27B) | 2025-12-31 |
| MuggleMath: Assessing the Impact of Query and Response Augmentation on Math Reasoning | ✓ Link | 25.8 | 7 | MuggleMATH 7B | 2023-10-09 |
| Solving Quantitative Reasoning Problems with Language Models | ✓ Link | 25.4 | 8 | Minerva 8B (maj1@k, k=64) | 2022-06-29 |
| Can Large Models Teach Student Models to Solve Mathematical Problems Like Human Beings? A Reasoning Distillation Method via Multi-LoRA Interaction | ✓ Link | 25.2 | | LoRID (LLaMA-2) | 2025-08-18 |
| Gemma: Open Models Based on Gemini Research and Technology | ✓ Link | 24.3 | | Gemma 7B (4-shot) | 2024-03-13 |
| RecurrentGemma: Moving Past Transformers for Efficient Open Language Models | ✓ Link | 23.8 | | RecurrentGemma-9B (4-shot) | 2024-04-11 |
| Triplet-Block Diffusion RWKV | ✓ Link | 23.8 | | B3D-RWKV-7.2B | 2026-05-25 |
| MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning | ✓ Link | 23.3 | 7 | MathCoder-L-7B | 2023-10-05 |
| WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct | ✓ Link | 22.7 | 70 | WizardMath-70B-V1.0 | 2023-08-18 |
| Parameter-Efficient Sparsity Crafting from Dense to Mixture-of-Experts for Instruction Tuning on General Tasks | ✓ Link | 22.6 | | Camelidae-8×34B (4-shot) | 2024-01-05 |
| MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models | ✓ Link | 22.5 | 13 | MetaMath 13B | 2023-09-21 |
| Chameleon: Mixed-Modal Early-Fusion Foundation Models | ✓ Link | 22.5 | | Chameleon 34B | 2024-05-16 |
| Knowledge Distillation During Mid-Training Favors Reasoning over Factual Recall | ✓ Link | 22.4 | | Switch Distillation (1B; 7B teacher, full post-training) | 2026-09-01 |
| T5Gemma 2: Seeing, Reading, and Understanding Longer | | 21.2 | | T5Gemma 2 4B-4B | 2025-12-16 |
| SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model | ✓ Link | 21.0 | | SmolLM2 1.7B Instruct | 2025-02-04 |
| LLaMA: Open and Efficient Foundation Language Models | ✓ Link | 20.5 | 65 | LLaMA 65B (maj1@k) | 2023-02-27 |
| Galactica: A Large Language Model for Science | ✓ Link | 20.4 | 120 | GAL 120B (5-shot) mCoT | 2022-11-16 |
| Cross-Tokenizer LLM Distillation through a Byte-Level Interface | | 20.08 | | Gemma2 (2B) with BLD (distilled from OpenMath2-Llama3.1-8B) | 2026-04-13 |
| MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models | ✓ Link | 19.4 | 7 | MetaMath 7B | 2023-09-21 |
| Solving Quantitative Reasoning Problems with Language Models | ✓ Link | 19.1 | 175 | davinci-002 175B | 2022-06-29 |
| Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM | ✓ Link | 17.8 | | Branch-Train-MiX 4x7B (sampling top-2 experts) | 2024-03-12 |
| Copy-Paste Language Models | ✓ Link | 17.5 | | Copy-Paste Language Model (1.7B, Full) | 2026-10-13 |
| Galactica: A Large Language Model for Science | ✓ Link | 16.6 | 120 | GAL 120B <work> | 2022-11-16 |
| LLaMA: Open and Efficient Foundation Language Models | ✓ Link | 15.2 | 33 | LLaMA 33B-maj1@k | 2023-02-27 |
| Power Scheduler: A Batch Size and Token Number Agnostic Learning Rate Scheduler | ✓ Link | 15.2 | | PowerLM-3B (4-shot) | 2024-08-23 |
| Solving Quantitative Reasoning Problems with Language Models | ✓ Link | 14.1 | 8 | Minerva 8B | 2022-06-29 |
| WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct | ✓ Link | 14.0 | 13 | WizardMath-13B-V1.0 | 2023-08-18 |
| dLLM: Simple Diffusion Language Modeling | ✓ Link | 13.9 | | Qwen3-0.6B-diffusion-bd3lm-v0.1 | 2026-02-26 |
| Mistral 7B | ✓ Link | 13.1 | 7 | Mistral 7B (maj@4) | 2023-10-10 |
| Galactica: A Large Language Model for Science | ✓ Link | 12.7 | 30 | GAL 30B (5-shot) mCoT | 2022-11-16 |
| Mixtral of Experts | ✓ Link | 12.7 | 7 | Mistral 7B (maj@4) | 2024-01-08 |
| SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model | ✓ Link | 11.6 | | SmolLM2 1.7B | 2025-02-04 |
| Galactica: A Large Language Model for Science | ✓ Link | 11.4 | 30 | GAL 30B <work> | 2022-11-16 |
| RecurrentGemma: Moving Past Transformers for Efficient Open Language Models | ✓ Link | 11.0 | | RecurrentGemma-2B (4-shot) | 2024-04-11 |
| WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct | ✓ Link | 10.7 | 7 | WizardMath-7B-V1.0 | 2023-08-18 |
| LLaMA: Open and Efficient Foundation Language Models | ✓ Link | 10.6 | 65 | LLaMA 65B | 2023-02-27 |
| Solving Quantitative Reasoning Problems with Language Models | ✓ Link | 8.8 | 540 | PaLM 540B | 2022-06-29 |
| Galactica: A Large Language Model for Science | ✓ Link | 8.8 | 540 | PaLM 540B (5-shot) mCoT | 2022-11-16 |
| LLaMA: Open and Efficient Foundation Language Models | ✓ Link | 8.8 | 13 | LLaMA 13B-maj1@k | 2023-02-27 |
| Scaling Inherently Interpretable Language Models | ✓ Link | 8.0 | | Steerling-8B | 2026-08-06 |
| LLaMA: Open and Efficient Foundation Language Models | ✓ Link | 7.1 | 33 | LLaMA 33B | 2023-02-27 |
| Measuring Mathematical Problem Solving With the MATH Dataset | ✓ Link | 6.9 | 1.5 | GPT-2 (1.5B) | 2021-03-05 |
| LLaMA: Open and Efficient Foundation Language Models | ✓ Link | 6.9 | 7 | LLaMA 7B-maj1@k | 2023-02-27 |
| Measuring Mathematical Problem Solving With the MATH Dataset | ✓ Link | 6.4 | 0.7 | GPT-2 (0.7B) | 2021-03-05 |
| Measuring Mathematical Problem Solving With the MATH Dataset | ✓ Link | 6.2 | 0.3 | GPT-2 (0.3B) | 2021-03-05 |
| Measuring Mathematical Problem Solving With the MATH Dataset | ✓ Link | 5.6 | 13 | GPT-3 13B | 2021-03-05 |
| Solving Quantitative Reasoning Problems with Language Models | ✓ Link | 5.6 | 8 | PaLM 8B (fine-tuned) | 2022-06-29 |
| Measuring Mathematical Problem Solving With the MATH Dataset | ✓ Link | 5.4 | 0.1 | GPT-2 (0.1B) | 2021-03-05 |
| Measuring Mathematical Problem Solving With the MATH Dataset | ✓ Link | 5.2 | 175 | GPT-3-175B (few-shot) | 2021-03-05 |
| Galactica: A Large Language Model for Science | ✓ Link | 5.2 | 175 | GPT-3 175B (8-shot) | 2022-11-16 |
| Solving Quantitative Reasoning Problems with Language Models | ✓ Link | 4.4 | 62 | PaLM 62B | 2022-06-29 |
| LLaMA: Open and Efficient Foundation Language Models | ✓ Link | 3.9 | 13 | LLaMA 13B | 2023-02-27 |
| Measuring Mathematical Problem Solving With the MATH Dataset | ✓ Link | 3.0 | 13 | GPT-3-13B (few-shot) | 2021-03-05 |
| Measuring Mathematical Problem Solving With the MATH Dataset | ✓ Link | 2.9 | 2.7 | GPT-3 2.7B | 2021-03-05 |
| LLaMA: Open and Efficient Foundation Language Models | ✓ Link | 2.9 | 7 | LLaMA 7B | 2023-02-27 |
| Solving Quantitative Reasoning Problems with Language Models | ✓ Link | 1.5 | 8 | PaLM 8B | 2022-06-29 |