| Hierarchical Prompting Taxonomy: A Universal Evaluation Framework for Large Language Models Aligned with Human Cognitive Principles | ✓ Link | 97.72 | | Claude 3.5 Sonnet (HPT) | 2024-06-18 |
| Learning More by Training Less: Uncertainty-Guided Progressive Contraction for Efficient Reasoning Post-Training | ✓ Link | 97.7 | | Qwen3.5-9B (rho=5%) | 2026-10-09 |
| SecOPD: Mitigating Adaptive Prompt Injections by On-Policy Distillation | ✓ Link | 97.4 | | SecOPD (Qwen3.6-27B) | 2026-08-21 |
| Achieving >97% on GSM8K: Deeply Understanding the Problems Makes LLMs Better Solvers for Math Word Problems | ✓ Link | 97.1 | | DUP prompt upon GPT-4 | 2024-04-23 |
| Introducing Llama 3.1: Our most capable models to date | ✓ Link | 96.8 | | Llama 3.1 (405B, Instruct) (8-shot) | 2024-07-23 |
| ThoughtFold: Folding Reasoning Chains via Introspective Preference Learning | ✓ Link | 96.8 | | ThoughtFold (Qwen3-14B) | 2026-06-02 |
| Qwen2 Technical Report | ✓ Link | 96.7 | 72 | Qwen2-Math-72B-Instruct
(greedy) | 2024-07-15 |
| ERNIE 4.5 Technical Report | ✓ Link | 96.6 | | ERNIE-4.5-300B-A47B | 2025-06-30 |
| Introducing efficiency focused "Thinking CAP" model series | ✓ Link | 96.5 | | ThinkingCap-Qwen3.6-27B | 2026-07-06 |
| | 96.4 | 7 | SFT-Mistral-7B (Metamath, OVM, Smart Ensemble) | |
| Introducing Claude 3.5 Sonnet | | 96.4 | | Claude 3.5 Sonnet | 2024-06-20 |
| CopT: Contrastive On-Policy Thinking with Continuous Spaces for General and Agentic Reasoning | ✓ Link | 96.36 | | CopT (Qwen3-8B) | 2026-05-19 |
| LLaDA2.0: Scaling Up Diffusion Language Models to 100B | ✓ Link | 96.06 | | LLaDA2.0-flash | 2025-12-10 |
| OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data | ✓ Link | 96.0 | | OpenMath2-Llama3.1-70B (majority@256) | 2024-10-02 |
| Qwen2.5 Technical Report | ✓ Link | 96.0 | | Qwen2.5-Plus | 2024-12-19 |
| Gemma 3 Technical Report | ✓ Link | 95.9 | | Gemma 3 (27B, IT) | 2025-03-25 |
| Geometry Conflict: Explaining and Controlling Forgetting in LLM Continual Post-Training | ✓ Link | 95.8 | | Qwen3-14B with GCWM | 2026-05-10 |
| Rethinking Training-Inference Mismatch in LLM Reinforcement Learning: Where It Arises and How to Correct It | ✓ Link | 95.6 | | Qwen3-30B-A3B + CIS | 2026-09-26 |
| HyperCLOVA X THINK Technical Report | ✓ Link | 95.53 | | HyperCLOVA X SEED 14B Think | 2025-06-27 |
| Post-Trained MoE Can Skip Half Experts via Self-Distillation | ✓ Link | 95.5 | | ZEDA (Qwen3-30B-A3B) | 2026-05-18 |
| Explore Broadly, Reason Sharply: Push Small Models toward the Frontier via Sampling | | 95.5 | | Qwen3-8B | 2026-09-29 |
| CritICL: Inference-Time Weak-to-Strong Generalization from Small Language Model Failure Modes | ✓ Link | 95.4 | | CritICL-static (Qwen2.5-72B-Instruct) | 2026-08-27 |
| Unlocking Lossless Speedups in LLMs via Discrete Diffusion | ✓ Link | 95.4 | | Uno (K2-Horizon-7B-Uno) | 2026-09-03 |
| Qwen2.5-VL Technical Report | ✓ Link | 95.3 | | Qwen2.5-VL-72B | 2025-02-19 |
| How2Everything: Mining the Web for How-To Procedures to Evaluate and Improve LLMs | ✓ Link | 95.3 | | Qwen3-8B (Instruct) + RL | 2026-02-09 |
| | 95.2 | 75 | Jiutian-大模型 | |
| | 95.1 | 7 | DAMOMath-7B(MetaMath, OVM, BS, Ensemble) | |
| Introducing Llama 3.1: Our most capable models to date | ✓ Link | 95.1 | | Llama 3.1 (70B, Instruct) (8-shot) | 2024-07-23 |
| The Claude 3 Model Family: Opus, Sonnet, Haiku | | 95 | | Claude 3 Opus (0-shot chain-of-thought) | 2024-03-04 |
| Introducing the next generation of Claude | | 95.0 | | Claude 3 Opus | 2024-03-04 |
| Introspective Diffusion Language Models | ✓ Link | 95.0 | | I-DLM (8B) | 2026-04-13 |
| OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data | ✓ Link | 94.9 | | OpenMath2-Llama3.1-70B | 2024-10-02 |
| Teaching-Inspired Integrated Prompting Framework: A Novel Approach for Enhancing Reasoning in Large Language Models | ✓ Link | 94.8 | | GPT-4 (Teaching-Inspired) | 2024-10-10 |
| MiniMax-01: Scaling Foundation Models with Lightning Attention | ✓ Link | 94.8 | | MiniMax-Text-01 | 2025-01-14 |
| LYNX: Learning Dynamic Exits for Confidence-Controlled Reasoning | ✓ Link | 94.77 | | LYNX (QwQ-32B) | 2025-12-05 |
| iGRPO: Self-Feedback-Driven LLM Reasoning | | 94.69 | | OpenMath-Nemotron-14B + iGRPO | 2026-02-09 |
| Heterogeneous Agent Collaborative Reinforcement Learning | | 94.6 | | Qwen3-4B-Instruct (HACPO) | 2026-03-03 |
| MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interaction | ✓ Link | 94.5 | | MiniCPM-o 4.5-Instruct | 2026-04-30 |
| When Should Models Change Their Minds? Contextual Belief Management in Large Language Models | | 94.5 | | Qwen3.5-9B (RL-RD) | 2026-05-28 |
| Think Again or Think Longer? Selective Verification for Budget-Aware Reasoning | ✓ Link | 94.47 | | Selective active verify (Qwen3-1.7B gate) | 2026-06-18 |
| PretrainZero: Reinforcement Active Pretraining | | 94.4 | | PretrainZero (Qwen3-30B-A3B-MoE-Base) | 2025-12-03 |
| Cactus: Accelerating Auto-Regressive Decoding with Constrained Acceptance Speculative Sampling | ✓ Link | 94.4 | | Cactus (δ=1.0; Qwen 3 32B verifier + Qwen 3 1.7B drafter) | 2026-04-05 |
| Qwen3 Technical Report | ✓ Link | 94.39 | | Qwen3-235B-A22B (Base) | 2025-05-14 |
| SAGE: Mitigating Long-Horizon Reasoning Biases via Topological Guidance | ✓ Link | 94.16 | | Qwen3.5-35B w/SAGE | 2026-09-24 |
| | 94.13 | 7 | SFT-Mistral-7B (Metamath + ovm +ensemble) | |
| OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data | ✓ Link | 94.1 | | OpenMath2-Llama3.1-8B (majority@256) | 2024-10-02 |
| DiagEvo: Diagnosis-Guided Self-Evolution via Hierarchical Error Memory | | 94.1 | | DiagEvo (Qwen3-8B, 235B-A22B diagnostician) | 2026-09-01 |
| See What I See, Know What I Think: Dense Latent Communication Across Heterogeneous Agents | | 94.09 | | Dense alignment (cross-model KV-cache transformation) | 2026-06-11 |
| Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs | ✓ Link | 94.0 | | Qwen2-72B-Instruct-Step-DPO (0-shot CoT) | 2024-06-26 |
| Guided Self-Evolving LLMs with Minimal Human Supervision | | 94.0 | | Qwen3-8B-Base (R-Few 1%) | 2025-12-02 |
| DARC: Decoupled Asymmetric Reasoning Curriculum for LLM Evolution | ✓ Link | 94.0 | | DARC (Qwen3-8B-Base) | 2026-01-20 |
| Motif 3: Technical Report | ✓ Link | 93.93 | | Motif 3 Base (8-shot CoT) | 2026-08-10 |
| Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillation | ✓ Link | 93.8 | | OPDLM-MATH-8B-Thinking | 2026-06-04 |
| Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decoding | | 93.71 | | Nemotron-Labs-Diffusion-14B (Diff.) | 2026-07-07 |
| Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Models | ✓ Link | 93.7 | | Nemotron-H-56B-Base (8-shot) | 2025-04-04 |
| Rethinking Cross-Tokenizer On-Policy Distillation: From Alignment Coverage to Supervision Reliability | | 93.69 | | Strict top-16 (Qwen3-235B-A22B-Instruct-2507 -> Granite-4.1-8B) | 2026-10-06 |
| CERA-MoA: Co-Evolving Routing Mechanisms with Continually Learning LLM Agents | ✓ Link | 93.5 | | CERA-MoA (Qwen3-4B) | 2026-09-16 |
| ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools | ✓ Link | 93.3 | | GLM-4 (0520) | 2024-06-18 |
| MixSD: Mixed Contextual Self-Distillation for Knowledge Injection | | 93.3 | | Qwen3-8B with MixSD (lambda=0.5) | 2026-05-16 |
| Cliff: Learning Process Rewards from the First Mistake | | 93.252 | | Qwen3-4B + Cliff (lambda1) | 2026-09-02 |
| | 93.2 | 7 | DAMOMath-7B(MetaMath, OVM, Ensemble) | |
| APPO: Agentic Procedural Policy Optimization | ✓ Link | 93.2 | | APPO (Qwen2.5-7B-Instruct) | 2026-06-10 |
| DiRL: An Efficient Post-Training Framework for Diffusion Language Models | ✓ Link | 93.1 | | DiRL-8B-Instruct | 2025-12-23 |
| Reasoning Palette: Modulating Reasoning via Latent Contextualization for Controllable Exploration for (V)LMs | | 93.03 | | Qwen3-8B-Base with Reasoning Palette (RLOO, Linear Decay) | 2025-12-19 |
| DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression | ✓ Link | 93.0 | | DeepSeek-V4.1-Flash Base | 2026-09-17 |
| Reading, Not Thinking: Understanding and Bridging the Modality Gap When Text Becomes Pixels in Multimodal LLMs | | 92.72 | | Qwen3-VL-8B (LM-only, self-distilled) | 2026-03-10 |
| Sharpen Without Search: On-Policy Distillation of Sequence-Level Power Distribution | ✓ Link | 92.63 | | OPPD (Qwen2.5-Math-7B, seed 1) | 2026-10-05 |
| Questioning the Questions: Sustaining Self-Evolution in Reasoning Models | ✓ Link | 92.57 | | R-Quest (Qwen3-4B-Base) | 2026-10-03 |
| Weak-Driven Learning: How Weak Agents make Strong Agents Stronger | ✓ Link | 92.5 | | WMSS (Qwen3-8B-Base) | 2026-02-09 |
| Off-the-Shelf LLMs as Process Scorers: Training-Free Alternative to PRMs for Mathematical Reasoning | | 92.5 | | CGS (Qwen2.5-1.5B → Qwen2.5-32B) | 2026-06-01 |
| DEVA: Fine-tuning Multimodal Large Language Models for Visual Perception Tasks | | 92.5 | | DEVA (Qwen2.5-14B) | 2026-06-03 |
| Stabilizing Reinforcement Learning for Diffusion Language Models | ✓ Link | 92.4 | | StableDRL (SDAR-8B, static sampling) | 2026-03-06 |
| Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning | ✓ Link | 92.34 | | Nemotron 3 Nano 30B-A3B Base | 2025-12-23 |
| The Claude 3 Model Family: Opus, Sonnet, Haiku | | 92.3 | | Claude 3 Sonnet (0-shot chain-of-thought) | 2024-03-04 |
| Introducing the next generation of Claude | | 92.3 | | Claude 3 Sonnet | 2024-03-04 |
| Nemotron-4 340B Technical Report | ✓ Link | 92.3 | | Nemotron-4-340B-Instruct | 2024-06-17 |
| MiMo-V2-Flash Technical Report | ✓ Link | 92.3 | | MiMo-V2-Flash (Base) | 2026-01-06 |
| CharacterFlywheel: Scaling Iterative Improvement of Engaging and Steerable LLMs in Production | | 92.3 | | CharacterFlywheel V7 (Llama 3.1 70B) | 2026-03-02 |
| Understanding Evolution Strategies for LLM Reasoning: Broader Reasoning Coverage than GRPO | | 92.3 | | Qwen2.5-7B-Instruct (GRPO to ES) | 2026-08-27 |
| HuatuoGPT-3: RL-Only Domain Adaptation from Base Models | ✓ Link | 92.3 | | OnePO (Qwen3-8B-Base) (extra data) | 2026-10-05 |
| DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model | ✓ Link | 92.2 | | DeepSeek-V2 Chat (RL) (8-shot) | 2024-05-07 |
| DMax: Aggressive Parallel Decoding for dLLMs | ✓ Link | 92.1 | | DMax-Math | 2026-04-09 |
| Representation-Space MMD for Diffusion Language Models | ✓ Link | 92.1 | | DMax-Math-MMD | 2026-10-05 |
| Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing | ✓ Link | 92 | 70 | AlphaLLM (with MCTS) | 2024-04-18 |
| Beyond Correctness: Learning Robust Reasoning via Transfer | | 92.0 | | RLTR (Ours) | 2026-02-09 |
| Multi-Block Diffusion Language Models | ✓ Link | 91.96 | | MBD-LLaDA2-Mini | 2026-06-30 |
| Rethinking RL for LLM Reasoning: It's Sparse Policy Selection, Not Capability Learning | ✓ Link | 91.9 | | ReasonMaxxer (Qwen3-4B) | 2026-05-07 |
| NGM: A Plug-and-Play Training-Free Memory Module for LLMs | ✓ Link | 91.74 | | Qwen3-14B + NGM | 2026-05-16 |
| OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data | ✓ Link | 91.7 | | OpenMath2-Llama3.1-8B | 2024-10-02 |
| Can Large Models Teach Student Models to Solve Mathematical Problems Like Human Beings? A Reasoning Distillation Method via Multi-LoRA Interaction | ✓ Link | 91.7 | | LoRID (Qwen2.5-Math) | 2025-08-18 |
| Scalable Prompt Routing via Fine-Grained Latent Task Discovery | | 91.6 | | FineRouter | 2026-03-19 |
| Guiding LLM Post-training Data Engineering with Model Internals from Sparse Autoencoders | | 91.5 | | Qwen2.5-Math-7B (SaeRL, GRPO) | 2026-05-26 |
| PaLM 2 Technical Report | ✓ Link | 91.0 | | PaLM 2 (few-shot, k=8, SC) | 2023-05-17 |
| Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone | ✓ Link | 91.0 | | Phi-3-Medium (8-shot) | 2024-04-22 |
| From Next-Token to Next-Block: A Principled Adaptation Path for Diffusion LLMs | ✓ Link | 91.0 | | NBDiff-7B (Instruct) | 2025-12-07 |
| Illusions of Confidence? Diagnosing LLM Truthfulness via Neighborhood Consistency | ✓ Link | 91.0 | | Qwen-2.5-32B-Instruct (Structure-Aware Training) | 2026-01-09 |
| Breaking the Ceiling of the LLM Community by Treating Token Generation as a Classification for Ensembling | ✓ Link | 90.91 | | GaC(Qwen2-72B-Instruct + Llama-3-70B-Instruct) | 2024-06-18 |
| OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset | ✓ Link | 90.8 | 70 | OpenMath-CodeLlama-70B (w/ code, SC, k=50) | 2024-02-15 |
| Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context | ✓ Link | 90.8 | | Gemini 1.5 Pro (11-shot) | 2024-03-08 |
| Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning | ✓ Link | 90.67 | | Nemotron 3 Super (120B-A12B Base) | 2026-04-14 |
| InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models | ✓ Link | 90.5 | | InternVL3-78B | 2025-04-14 |
| DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solving | ✓ Link | 90.4 | 70 | DART-Math-Llama3-70B-Uniform (0-shot CoT, w/o code) | 2024-06-18 |
| The Molecular Structure of Thought: Mapping the Topology of Long Chain-of-Thought Reasoning | | 90.3 | | Llama-3.1-8B-Instruct + 35K QwQ-Mole-Syn + RL | 2026-01-09 |
| Beyond Policy Rollouts: Embedding-Space Directional Exploration with Anchored Noise for LLM RLVR | | 90.3 | | EDEN (Qwen2.5-Math-7B) | 2026-10-12 |
| OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset | ✓ Link | 90.1 | 70 | OpenMath-Llama2-70B (w/ code, SC, k=50) | 2024-02-15 |
| Can Large Models Teach Student Models to Solve Mathematical Problems Like Human Beings? A Reasoning Distillation Method via Multi-LoRA Interaction | ✓ Link | 90.0 | | LoRID (DeepSeekMath) | 2025-08-18 |
| The Flexibility Trap: Why Arbitrary Order Limits Reasoning Potential in Diffusion Language Models | ✓ Link | 89.8 | | JustGRPO (LLaDA 8B Instruct) | 2026-01-21 |
| S2D2: Fast Decoding for Diffusion LLMs via Training-Free Self-Speculation | ✓ Link | 89.8 | | S2D2 (LLaDA2.1-Mini) | 2026-03-26 |
| Residual Context Diffusion Language Models | ✓ Link | 89.76 | | SDAR (8B, b=32) with RCD | 2026-01-30 |
| ProFit: Leveraging High-Value Signals in SFT via Probability-Guided Token Selection | ✓ Link | 89.62 | | ProFit (Qwen3-14B-Base) | 2026-01-14 |
| DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solving | ✓ Link | 89.6 | 70 | DART-Math-Llama3-70B-Prop2Diff (0-shot CoT, w/o code) | 2024-06-18 |
| When Reasoning Meets Its Laws | ✓ Link | 89.5 | | DeepSeek-R1-8B (SFT-Compo) | 2025-12-19 |
| Thinking Before Constraining: A Unified Decoding Framework for Large Language Models | ✓ Link | 89.5 | | Qwen3-8B (In-Writing-Base) | 2026-05-28 |
| EpiCaR: Knowing What You Don't Know Matters for Better Reasoning in LLMs | | 89.46 | | EpiCaR (Qwen-3, 8B) | 2026-01-11 |
| T3D: Few-Step Diffusion Language Models via Trajectory Self-Distillation with Direct Discriminative Optimization | ✓ Link | 89.31 | | T3D (SDAR-4B-Chat) | 2026-02-12 |
| Prefill-Free Cross-Family KV Cache Transfer for Heterogeneous Multi-Agent LLMs | ✓ Link | 89.23 | | HeteroFold (Ministral-3-14B to Qwen3-4B) | 2026-09-29 |
| Demystifying Hidden-State Recurrence: Switchable Latent Reasoning with On-Policy Reinforcement Learning | | 89.2 | | Switch (ours, after Switch-GRPO) | 2026-06-11 |
| daVinci-Env: Open SWE Environment Synthesis at Scale | ✓ Link | 89.16 | | OpenSWE-72B (Qwen2.5-72B-Base) | 2026-03-13 |
| Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations | ✓ Link | 89.1 | 7 | Shepherd+Mistral-7B (SFT on MetaMATH + PRM RL+ PRM rerank, k=256) | 2023-12-14 |
| | 89.0 | 13 | Llama SFT (Metamath ToRA Ensemble) | |
| Solving Quantitative Reasoning Problems with Language Models | ✓ Link | 89 | 62 | Minerva 62B (maj5@100) | 2022-06-29 |
| Improved Large Language Diffusion Models | ✓ Link | 89.0 | | iLLaDA 8B Instruct | 2026-06-24 |
| Distillation Defenses Easily Break After Reinforcement Learning | | 89.0 | | Qwen2.5-Math-7B (Distill) (extra data) | 2026-09-28 |
| The Claude 3 Model Family: Opus, Sonnet, Haiku | | 88.9 | | Claude 3 Haiku (0-shot chain-of-thought) | 2024-03-04 |
| Introducing the next generation of Claude | | 88.9 | | Claude 3 Haiku | 2024-03-04 |
| From Imitation to Discrimination: Toward A Generalized Curriculum Advantage Mechanism Enhancing Cross-Domain Reasoning Tasks | | 88.9 | | GRPO (with CAPO) (Qwen2.5-7B-Math) | 2025-12-02 |
| Multi-Stream LLMs: Unblocking Language Models with Parallel Streams of Thoughts, Inputs and Outputs | ✓ Link | 88.85 | | Qwen3-4B (Stream) | 2026-05-12 |
| Qwen2.5-Omni Technical Report | ✓ Link | 88.7 | | Qwen2.5-Omni-7B | 2025-03-26 |
| Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs | ✓ Link | 88.6 | | Phi-4-Mini (8-shot, CoT) | 2025-02-24 |
| Encoder-Decoder Gemma: Improving the Quality-Efficiency Trade-Off via Adaptation | ✓ Link | 88.6 | | T5Gemma 9B-9B PrefixLM IT | 2025-04-08 |
| T5Gemma 2: Seeing, Reading, and Understanding Longer | | 88.6 | | T5Gemma 2 4B-4B (post-trained) | 2025-12-16 |
| ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving | ✓ Link | 88.3 | 70 | ToRA-70B (SC, k=50) | 2023-09-29 |
| LightThinker++: From Reasoning Compression to Memory Management | | 88.25 | | LThinker (tho) (Llama3.1-8B) | 2026-04-04 |
| DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models | ✓ Link | 88.2 | 7 | DeepSeekMATH-RL-7B | 2024-02-05 |
| DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solving | ✓ Link | 88.2 | 7 | DART-Math-DSMath-7B-Uniform (0-shot CoT, w/o code) | 2024-06-18 |
| Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters | ✓ Link | 88.2 | | Step-3.5-Flash Base | 2026-02-11 |
| CLIPO: Contrastive Learning in Policy Optimization Generalizes RLVR | ✓ Link | 88.02 | | GRPO + CLIPO | 2026-03-10 |
| PRISM: PRIvileged Sibling-guided Multi-Agent Topology Design | | 88.02 | | PRISM | 2026-09-29 |
| Model Card and Evaluations for Claude Models | | 88 | | Claude 2 (0-shot chain-of-thought) | 2023-07-11 |
| Claude 2 | | 88.0 | | Claude 2 | 2023-07-11 |
| OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset | ✓ Link | 88.0 | 34 | OpenMath-CodeLlama-34B (w/ code, SC, k=50) | 2024-02-15 |
| WildReward: Learning Reward Models from In-the-Wild Human Interactions | ✓ Link | 87.9 | | Llama3.1-8B-Instruct (online DPO with WildReward) | 2026-02-09 |
| FourierSampler: Unlocking Non-Autoregressive Potential in Diffusion Language Models via Frequency-Guided Generation | ✓ Link | 87.64 | | SDAR-4B-Chat (with FourierSampler) | 2026-01-30 |
| 2 OLMo 2 Furious | ✓ Link | 87.6 | | OLMo 2 32B Instruct (8-shot) | 2024-12-31 |
| Dynin-Omni: Omnimodal Unified Large Diffusion Language Model | ✓ Link | 87.6 | | Dynin-Omni (8B) | 2026-03-09 |
| | 87.41 | 4 | Shivaay-4B (8-shot chain-of-thought) | |
| 2 OLMo 2 Furious | ✓ Link | 87.4 | | OLMo 2 13B Instruct (8-shot) | 2024-12-31 |
| Solving math word problems with process- and outcome-based feedback | | 87.3 | 70 | DeepMind 70B Model (SFT+ORM-RL, ORM reranking) | 2022-11-25 |
| An Empirical Study of Data Ability Boundary in LLMs' Math Reasoning | ✓ Link | 87.2 | 7 | MMOS-DeepSeekMath-7B(0-shot,k=50) | 2024-02-23 |
| Solving math word problems with process- and outcome-based feedback | | 87.1 | 70 | DeepMind 70B Model (SFT+PRM-RL, PRM reranking) | 2022-11-25 |
| Sparks of Artificial General Intelligence: Early experiments with GPT-4 | ✓ Link | 87.1 | | GPT-4 | 2023-03-22 |
| Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Models | ✓ Link | 87.1 | | Nemotron-H-8B-Base (8-shot) | 2025-04-04 |
| Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weights | ✓ Link | 87.1 | | Qwen2.5-3B-Inst with RandOpt | 2026-03-12 |
| OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset | ✓ Link | 86.9 | 7 | OpenMath-Mistral-7B (w/ code, SC, k=50) | 2024-02-15 |
| OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset | ✓ Link | 86.8 | 13 | OpenMath-CodeLlama-13B (w/ code, SC, k=50) | 2024-02-15 |
| Orca-Math: Unlocking the potential of SLMs in Grade School Math | | 86.8 | 7 | Orca-Math 7B (fine-tuned) | 2024-02-16 |
| DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solving | ✓ Link | 86.8 | 7 | DART-Math-DSMath-7B-Prop2Diff (0-shot CoT, w/o code) | 2024-06-18 |
| Gemini: A Family of Highly Capable Multimodal Models | ✓ Link | 86.5 | | Gemini Pro (maj1@32) | 2023-12-19 |
| Deep Fusion Memory: A Chunk-Level Parametric Memory for Large Language Models | | 86.43 | | DFM (Qwen3-8B-Base) | 2026-10-18 |
| A Sovereign, Open-Source Foundation Model for German and English | ✓ Link | 86.1 | | Soofi S 30B-A3B Base | 2026-07-10 |
| NeuPAT: Neuron-aware Plasticity Allocation Tuning for Language-Preserving MLLMs | | 85.97 | | NeuPAT (Qwen3-4B-Instruct-2507) (extra data) | 2026-08-08 |
| LFM2.5-1.2B-Thinking: On-Device Reasoning Under 1GB | | 85.60 | | LFM2.5-1.2B-Thinking | 2026-01-20 |
| BEAM: Binary Expert Activation Masking for Dynamic Routing in MoE | | 85.52 | | Qwen3-30B-A3B (BEAM, beta=0.01) | 2026-05-14 |
| | 85.5 | | Codex (Self-Evaluation Guided Decoding, PAL, multiple reasoning chains, 9-shot gen, 5-shot eval) | |
| Co-RL: Unsupervised Reasoning Emerges from Diverse Cohort in Multi-agent RL | ✓ Link | 85.4 | | Co-RL (Llama-3.1-8B-Instruct, Different family+) | 2026-08-19 |
| DTCA: Denoising Token Credit Assignment for Reinforcement Learning in Diffusion Language Models | ✓ Link | 85.4 | | DTCA (ours) | 2026-10-26 |
| FineRMoE: Dimension Expansion for Finer-Grained Expert with Its Upcycling Approach | ✓ Link | 85.37 | | FineRMoE (Qwen2.5-7B) | 2026-03-09 |
| LoopRPT: Reinforcement Pre-Training for Looped Language Models | | 85.36 | | Ouro (2.6B) LoopRPT | 2026-03-20 |
| Prism: Efficient Test-Time Scaling via Hierarchical Search and Self-Verification for Discrete Diffusion Language Models | ✓ Link | 85.3 | | Prism (LLaDA-8B-Instruct, K=8) | 2026-02-02 |
| Model Card and Evaluations for Claude Models | | 85.2 | | Claude 1.3 (0-shot chain-of-thought) | 2023-07-11 |
| ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving | ✓ Link | 85.1 | 34 | ToRA-Code-34B (SC, k=50) | 2023-09-29 |
| 2 OLMo 2 Furious | ✓ Link | 85.1 | | OLMo 2 7B Instruct (8-shot) | 2024-12-31 |
| ReFusion: A Diffusion Large Language Model with Parallel Autoregressive Decoding | ✓ Link | 84.91 | | ReFusion | 2025-12-15 |
| Safety at One Shot: Patching Fine-Tuned LLMs with A Single Instance | | 84.9 | | Llama3 8B (One-shot FT) | 2026-01-05 |
| OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset | ✓ Link | 84.8 | 7 | OpenMath-CodeLlama-7B (w/ code, SC, k=50) | 2024-02-15 |
| HyperThink: Text-to-Parameter Hypernetworks for Efficient Reasoning | | 84.75 | | HyperThink (SmolLM3-3B) (extra data) | 2026-10-02 |
| OVM, Outcome-supervised Value Models for Planning in Mathematical Reasoning | ✓ Link | 84.7 | 7 | OVM-Mistral-7B (verify100@1) | 2023-11-16 |
| OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset | ✓ Link | 84.7 | 70 | OpenMath-Llama2-70B (w/ code) | 2024-02-15 |
| Group Sequence Policy Optimization | | 84.7 | | AliceAI-T5-35B-A0.6B-Base | 2025-07-24 |
| HRM-Text: Efficient Pretraining Beyond Scaling | ✓ Link | 84.7 | | HRM-Text-1B | 2026-05-20 |
| OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset | ✓ Link | 84.6 | 70 | OpenMath-CodeLlama-70B (w/ code) | 2024-02-15 |
| LEVER: Learning to Verify Language-to-Code Generation with Execution | ✓ Link | 84.5 | 175 | code-davinci-002 175B (LEVER, 8-shot) | 2023-02-16 |
| Introducing Llama 3.1: Our most capable models to date | ✓ Link | 84.5 | | Llama 3.1 (8B, Instruct) (8-shot) | 2024-07-23 |
| The Llama 3 Herd of Models | ✓ Link | 84.5 | | Llama 3.1 (8B, Instruct) | 2024-07-31 |
| SeKV: Resolution-Adaptive KV Cache with Hierarchical Semantic Memory for Long-Context LLM Inference | ✓ Link | 84.5 | | SeKV (Qwen2.5-14B-Instruct) | 2026-06-30 |
| LFM2 Technical Report | ✓ Link | 84.38 | | LFM2-8B-A1B (5-shot) | 2025-11-28 |
| ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving | ✓ Link | 84.3 | 70 | ToRA 70B | 2023-09-29 |
| Dynamic Latent Routing | | 84.3 | | Dynamic Latent Routing (DLR) (Qwen3-8B) | 2026-05-14 |
| Can Large Models Teach Student Models to Solve Mathematical Problems Like Human Beings? A Reasoning Distillation Method via Multi-LoRA Interaction | ✓ Link | 84.2 | | LoRID (Mistral) | 2025-08-18 |
| Laguna M.1/XS.2 Technical Report | ✓ Link | 84.2 | | Laguna XS.2 | 2026-05-26 |
| Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations | ✓ Link | 84.1 | 7 | Shepherd + Mistral-7B (SFT on MetaMATH + PRM RL) | 2023-12-14 |
| DeepSeek LLM: Scaling Open-Source Language Models with Longtermism | ✓ Link | 84.1 | | DeepSeek LLM 67B Chat | 2024-01-05 |
| LLaDA MoE v2: Scaling Mixture-of-Experts Diffusion Language Models | | 83.93 | | LLaDA MoE v2 Base | 2026-08-04 |
| MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning | ✓ Link | 83.9 | 70 | MathCoder-L-70B | 2023-10-05 |
| Falcon-H1: A Family of Hybrid-Head Language Models Redefining Efficiency and Performance | ✓ Link | 83.62 | | Falcon-H1-34B-Instruct (5-shot) | 2025-07-30 |
| Who Should Lead Decoding Now? Tracking Reliable Trajectories for Ensembling Masked Diffusion Language Models | | 83.62 | | TIE (LLaDA + Dream; Entropy) | 2026-06-15 |
| Towards Looped Models Done Right, Part I: Topology, Input Injection, Recurrent-State Design | ✓ Link | 83.62 | | Huginn MoE (8.0B resident, 0.8B active) | 2026-10-16 |
| The Optimizer Is the Agent: Reasoning-Driven Search across Prompts, Programs, and ML Workflows | ✓ Link | 83.4 | | ReASearch (Llama3.1-8B optimized prompt) | 2026-08-07 |
| Making Large Language Models Better Reasoners with Step-Aware Verifier | | 83.2 | 175 | DIVERSE 175B (8-shot) | 2022-06-06 |
| WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct | ✓ Link | 83.2 | 7 | WizardMath-7B-V1.1 | 2023-08-18 |
| NCP-ArchPreview Technical Report: Moving towards Latent Space Language Models through Next Concept Prediction | ✓ Link | 83.02 | | NCP-ArchPreview Stage-2 | 2026-09-09 |
| Every Token Counts: Generalizing 16M Ultra-Long Context in Large Language Models | ✓ Link | 82.94 | | HSA-UltraLong-Inst | 2025-11-28 |
| The ART of LLM Refinement: Ask, Refine, and Trust | | 82.6 | | ChatGPT (Ask, Refine, Trust) | 2023-11-14 |
| OVM, Outcome-supervised Value Models for Planning in Mathematical Reasoning | ✓ Link | 82.6 | 7 | OVM-Mistral-7B (verify20@1) | 2023-11-16 |
| DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solving | ✓ Link | 82.6 | 7 | DART-Math-Mistral-7B-Uniform (0-shot CoT, w/o code) | 2024-06-18 |
| DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solving | ✓ Link | 82.5 | 8 | DART-Math-Llama3-8B-Uniform (0-shot CoT, w/o code) | 2024-06-18 |
| Attention Residuals | ✓ Link | 82.4 | | Kimi Linear (48B) with Block AttnRes | 2026-03-16 |
| MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models | ✓ Link | 82.3 | 70 | MetaMath 70B | 2023-09-21 |
| MuggleMath: Assessing the Impact of Query and Response Augmentation on Math Reasoning | ✓ Link | 82.3 | 70 | MuggleMATH 70B | 2023-10-09 |
| Large Language Models Can Self-Improve | | 82.1 | 540 | PaLM 540B (Self Improvement, Self Consistency) | 2022-10-20 |
| Linear Ensembles Wash Away Watermarks: On the Fragility of Distributional Perturbations in LLMs | ✓ Link | 82.1 | | WASH (N=4) on Ministral3-8B | 2026-05-28 |
| Token-Level LLM Collaboration via FusionRoute | | 82.0 | | FusionRoute (Llama-3.1-8B) | 2026-01-08 |
| C^2DLM: Causal Concept-Guided Diffusion Large Language Models | ✓ Link | 81.96 | | C2DLM | 2025-11-27 |
| Improved Large Language Diffusion Models | ✓ Link | 81.9 | | iLLaDA 8B Base | 2026-06-24 |
| MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning | ✓ Link | 81.7 | 34 | MathCoder-CL-34B | 2023-10-05 |
| Mellum2 Technical Report | ✓ Link | 81.7 | | Mellum 2 (base) | 2026-05-29 |
| WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct | ✓ Link | 81.6 | 70 | WizardMath-70B-V1.0 | 2023-08-18 |
| TinyGSM: achieving >80% on GSM8k with small language models | | 81.5 | 2.6 | Phi-GSM+V 1.3B+1.3B (verify48@1) | 2023-12-14 |
| Fine-tuning Multi-modal LLMs with ART: Art-based Reinforcement Training | | 81.2 | | ART (Optimized Image) — Qwen3.5-2B | 2026-06-10 |
| DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solving | ✓ Link | 81.1 | 7 | DART-Math-Mistral-7B-Prop2Diff (0-shot CoT, w/o code) | 2024-06-18 |
| DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solving | ✓ Link | 81.1 | 8 | DART-Math-Llama3-8B-Prop2Diff (0-shot CoT, w/o code) | 2024-06-18 |
| Model Card and Evaluations for Claude Models | | 80.9 | | Claude Instant 1.1 (0-shot chain-of-thought) | 2023-07-11 |
| Beyond Hard Masks: Progressive Token Evolution for Diffusion Language Models | ✓ Link | 80.74 | | EvoToken-DLM (LLaDA-Instruct-8B, 10k FT) | 2026-01-12 |
| PaLM 2 Technical Report | ✓ Link | 80.7 | | PaLM 2 (few-shot, k=8, CoT) | 2023-05-17 |
| ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving | ✓ Link | 80.7 | 34 | ToRA-Code 34B | 2023-09-29 |
| OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset | ✓ Link | 80.7 | 34 | OpenMath-CodeLlama-34B (w/ code) | 2024-02-15 |
| dLLM: Simple Diffusion Language Modeling | ✓ Link | 80.59 | | LLaDA-Instruct (reasoning SFT) | 2026-02-26 |
| An Empirical Study of Data Ability Boundary in LLMs' Math Reasoning | ✓ Link | 80.5 | 7 | MMOS-DeepSeekMath-7B(0-shot) | 2024-02-23 |
| MatryoshkaLoRA: Learning Accurate Hierarchical Low-Rank Representations for LLM Fine-Tuning | ✓ Link | 80.5 | | MatryoshkaLora (Llama-3.1-8B-Instruct) | 2026-05-08 |
| An Empirical Study of Data Ability Boundary in LLMs' Math Reasoning | ✓ Link | 80.4 | 34 | MMOS-CODE-34B(0-shot) | 2024-02-23 |
| OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset | ✓ Link | 80.2 | 7 | OpenMath-Mistral-7B (w/ code) | 2024-02-15 |
| | 80.2 | | Self-Evaluation Guided Decoding (Codex, PAL, single reasoning chain, 9-shot gen, 5-shot eval) | |
| Knowledge Distillation During Mid-Training Favors Reasoning over Factual Recall | ✓ Link | 79.8 | | Switch Distillation (1B; 7B teacher, full post-training) | 2026-09-01 |
| Pivot-SD: Efficient Self-Distillation for Masked Diffusion Language Models | ✓ Link | 79.51 | | Pivot-SD | 2026-10-02 |
| Stop the Flip-Flop: Context-Preserving Verification for Fast Revocable Diffusion Decoding | | 79.3 | | COVER (Dream-7B-Instruct) | 2026-02-05 |
| OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset | ✓ Link | 78.8 | 13 | OpenMath-CodeLlama-13B (w/ code) | 2024-02-15 |
| What Makes a Good Unmasking Policy? A Controlled Study of Learned Unmasking in Diffusion Language Models | | 78.8 | | OnTrack (Ours) | 2026-10-22 |
| Solving Quantitative Reasoning Problems with Language Models | ✓ Link | 78.5 | 540 | Minerva 540B (CoT) | 2022-06-29 |
| Can Large Models Teach Student Models to Solve Mathematical Problems Like Human Beings? A Reasoning Distillation Method via Multi-LoRA Interaction | ✓ Link | 78.5 | | LoRID (LLaMA-2) | 2025-08-18 |
| Parameter-Efficient Sparsity Crafting from Dense to Mixture-of-Experts for Instruction Tuning on General Tasks | ✓ Link | 78.3 | | Camelidae-8×34B (5-shot) | 2024-01-05 |
| dVoting: Fast Voting for dLLMs | ✓ Link | 78.24 | | dVoting (LLaDA-8B-Instruct) | 2026-02-12 |
| Parameter-Efficient Sparsity Crafting from Dense to Mixture-of-Experts for Instruction Tuning on General Tasks | ✓ Link | 77.8 | | Qwen2idae-16x14B (5-shot) | 2024-01-05 |
| Fast-Decoding Diffusion Language Models via Progress-Aware Confidence Schedules | ✓ Link | 77.71 | | Dream (Base) with SchED (Exp k=2, τhigh=7.5, τlow=2.5) | 2025-12-02 |
| MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models | ✓ Link | 77.7 | 7 | MetaMath-Mistral-7B | 2023-09-21 |
| Llama 3.2: Revolutionizing edge AI and vision with open, customizable models | ✓ Link | 77.7 | | Llama 3.2 3B Instruct | 2024-09-25 |
| SLPO: Scaling Latent Reasoning via a Surrogate Policy | ✓ Link | 77.63 | | SLPO (Llama-3.2-3B-Instruct) | 2026-07-22 |
| Apertus: Democratizing Open and Compliant LLMs for Global Language Environments | ✓ Link | 77.6 | | Apertus-70B-Instruct | 2025-09-17 |
| Youtu-LLM: Unlocking the Native Agentic Potential for Lightweight Large Language Models | ✓ Link | 77.6 | | Youtu-LLM (2B, base) | 2025-12-31 |
| Beyond Length Scaling: Synergizing Breadth and Depth for Generative Reward Models | ✓ Link | 77.6 | | Mix-GRM (used in DPO) | 2026-03-02 |
| OpenChat: Advancing Open-source Language Models with Mixed-Quality Data | ✓ Link | 77.3 | 7 | OpenChat-3.5 7B | 2023-09-20 |
| POP: Prefill-Only Pruning for Efficient Large Model Inference | | 77.26 | | POP (Llama-3.1-8B-Instruct) | 2026-02-03 |
| When Confidence Misleads: Suffix Anchoring and Anchor-Proximity Confidence Modulation for Diffusion Language Models | | 76.88 | | Suffix-Anchored Confidence Modulation (LLaDA 8B-Instruct, top-probability decoding) | 2026-05-27 |
| Solving math word problems with process- and outcome-based feedback | | 76.5 | 70 | DeepMind 70B Model (STaR, maj1@96) | 2022-11-25 |
| | 76.4 | 7 | Arithmo2-Mistral-7B | |
| AdaPreLoRA: Adafactor Preconditioned Low-Rank Adaptation | | 76.4 | | Qwen2-7B + AdaPreLoRA AdamW | 2026-05-09 |
| Liquid Foundation Models: Our First Series of Generative AI Models | | 76.04 | | LFM-40B Preview (5-shot) | 2024-09-30 |
| OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset | ✓ Link | 75.9 | 7 | OpenMath-CodeLlama-7B (w/ code) | 2024-02-15 |
| LastOPD: Taming Collapse in Latent On-Policy Distillation | ✓ Link | 75.82 | | LastOPD (Qwen3-4B -> Qwen3-1.7B-Base) | 2026-09-23 |
| ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving | ✓ Link | 75.8 | 13 | ToRA-Code 13B | 2023-09-29 |
| River-LLM: Large Language Model Seamless Exit Based on KV Share | | 75.6 | | River-LLM (Llama3.1 8B, τ=0.7) | 2026-04-20 |
| Agon: Competitive Cross-Model RL with Implicit Rival Grading of Reasoning | | 75.0 | | Agon | 2026-07-08 |
| | 74.7 | 7 | Arithmo-Mistral-7B | |
| Self-Consistency Improves Chain of Thought Reasoning in Language Models | ✓ Link | 74.4 | 540 | PaLM 540B maj1@40 (8-shot) | 2022-03-21 |
| Large Language Models Can Self-Improve | | 74.4 | 540 | PaLM 540B (Self Consistency) | 2022-10-20 |
| Mixtral of Experts | ✓ Link | 74.4 | | Mixtral 8x7B (8-shot) | 2024-01-08 |
| The Master Key Hypothesis: Unlocking Cross-Model Capability Transfer via Linear Subspace Alignment | ✓ Link | 74.4 | | Qwen1.5-14B + Unlock | 2026-04-07 |
| TinyGSM: achieving >80% on GSM8k with small language models | | 74.3 | 2.7 | Phi-GSM 2.7B (fine-tuned) | 2023-12-14 |
| MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning | ✓ Link | 74.1 | 7 | MathCoder-CL-13B | 2023-10-05 |
| MuggleMath: Assessing the Impact of Query and Response Augmentation on Math Reasoning | ✓ Link | 74 | 13 | MuggleMATH 13B | 2023-10-09 |
| Gemma 2: Improving Open Language Models at a Practical Size | ✓ Link | 74.0 | | Gemma 2 27B (5-shot) | 2024-07-31 |
| An Empirical Study of Data Ability Boundary in LLMs' Math Reasoning | ✓ Link | 73.9 | 7 | MMOS-CODE-7B(0-shot) | 2024-02-23 |
| CodeT5+: Open Code Large Language Models for Code Understanding and Generation | ✓ Link | 73.8 | 0.77 | CodeT5+ | 2023-05-13 |
| CAPO: Cost-Aware Prompt Optimization | ✓ Link | 73.73 | | Llama-3.3-70B + CAPO | 2025-04-22 |
| OVM, Outcome-supervised Value Models for Planning in Mathematical Reasoning | ✓ Link | 73.7 | 7 | OVM-Llama2-7B (verify100@1) | 2023-11-16 |
| Large Language Models Can Self-Improve | | 73.5 | 540 | PaLM 540B (Self Improvement, CoT Prompting) | 2022-10-20 |
| KwaiYiiMath: Technical Report | | 73.3 | 13 | KwaiYiiMath 13B | 2023-10-11 |
| daVinci-LLM:Towards the Science of Pretraining | ✓ Link | 72.86 | | daVinci-LLM-3B | 2026-03-28 |
| Encoder-Decoder Gemma: Improving the Quality-Efficiency Trade-Off via Adaptation | ✓ Link | 72.8 | | T5Gemma 9B-9B PrefixLM | 2025-04-08 |
| ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving | ✓ Link | 72.6 | 7 | ToRA-Code 7B | 2023-09-29 |
| MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning | ✓ Link | 72.6 | 13 | MathCoder-L-13B | 2023-10-05 |
| | 72.3 | | DBRX Base 132B | |
| CADENCE: Closing the Reasoning Gap via Coverage-Adaptive On-Policy Distillation | | 72.1 | | CADENCE (3B teacher) | 2026-07-18 |
| | 71.9 | | Self-Evaluation Guided Decoding (Codex, CoT, single reasoning chain, 9-shot gen, 5-shot eval) | |
| Triplet-Block Diffusion RWKV | ✓ Link | 71.5 | | B3D-RWKV-7.2B | 2026-05-25 |
| Rethinking Selective Knowledge Distillation | ✓ Link | 71.2 | | SE-KD + TopSmp (on-policy) | 2026-02-01 |
| MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models | ✓ Link | 71.0 | 13 | MetaMath 13B | 2023-09-21 |
| Trust-Region Behavior Blending for On-Policy Distillation | | 70.1 | | Trust-Region behavior Blending (Qwen3-0.6B-Base ← Qwen3-4B) | 2026-05-29 |
| MuggleMath: Assessing the Impact of Query and Response Augmentation on Math Reasoning | ✓ Link | 69.8 | 7 | MuggleMATH 7B | 2023-10-09 |
| LLaMA: Open and Efficient Foundation Language Models | ✓ Link | 69.7 | 65 | LLaMA 65B-maj1@k | 2023-02-27 |
| Solving Quantitative Reasoning Problems with Language Models | ✓ Link | 68.5 | 62 | Minerva 62B (maj1@100) | 2022-06-29 |
| SkillFactory: Self-Distillation For Learning Cognitive Behaviors | ✓ Link | 68.2 | | Qwen2.5-1.5B-Instruct (SkillFactory → GRPO) | 2025-12-03 |
| Least-to-Most Prompting Enables Complex Reasoning in Large Language Models | ✓ Link | 68.01 | 175 | code-davinci-002 (Least-to-Most Prompting) | 2022-05-21 |
| MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning | ✓ Link | 67.8 | 7 | MathCoder-CL-7B | 2023-10-05 |
| SmolLM3: smol, multilingual, long-context reasoner | ✓ Link | 67.63 | | SmolLM3 3B Base (5-shot) | 2025-07-08 |
| Follow the Gradient: Tracing Safety Degradation Back to Training Data | | 67.5 | | Llama 3.1 8B (SafeTulu) | 2026-10-27 |
| Beyond the Assistant Turn: User Turn Generation as a Probe of Interaction Awareness in Language Models | | 67.4 | | Qwen3.5-2B (RL) | 2026-04-03 |
| | 66.9 | | DBRX Instruct 132B | |
| MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models | ✓ Link | 66.4 | 7 | MetaMath 7B | 2023-09-21 |
| Pion: A Spectrum-Preserving Optimizer via Orthogonal Equivalence Transformation | ✓ Link | 65.76 | | Pion (Qwen2.5-1.5B finetuning) | 2026-05-12 |
| ReMix: Reinforcement routing for mixtures of LoRAs in LLM finetuning | | 65.66 | | ReMix (Llama 3 8B) | 2026-03-10 |
| What Matters for Latent Reasoning with Flow Matching | | 65.2 | | FLaRe, stage 2 (Llama-3.2-3B-Instruct) (extra data) | 2026-10-05 |
| CAPO: Cost-Aware Prompt Optimization | ✓ Link | 65.07 | | Mistral-Small-24B + CAPO | 2025-04-22 |
| Scaling Relationship on Learning Mathematical Reasoning with Large Language Models | ✓ Link | 64.8 | 79 | RFT 70B | 2023-08-03 |
| MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning | ✓ Link | 64.2 | 7 | MathCoder-L-7B | 2023-10-05 |
| WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct | ✓ Link | 63.9 | 13 | WizardMath-13B-V1.0 | 2023-08-18 |
| Solving Math Word Problems via Cooperative Reasoning induced Language Models | ✓ Link | 63.2 | 12 | GPT-J (CoRe) | 2022-10-28 |
| Cross-Tokenizer LLM Distillation through a Byte-Level Interface | | 62.55 | | Gemma2 (2B) with BLD (distilled from OpenMath2-Llama3.1-8B) | 2026-04-13 |
| Data Science and Technology Towards AGI Part I: Tiered Data Management | ✓ Link | 61.79 | | MiniCPM-1.2B (Math-L3) | 2026-02-09 |
| Chameleon: Mixed-Modal Early-Fusion Foundation Models | ✓ Link | 61.4 | | Chameleon 34B | 2024-05-16 |
| The Unreasonable Effectiveness of Eccentric Automatic Prompts | | 61 | 70 | Llama-2 70B (on 100 first questions, 4-shot, auto-optimized prompting) | 2024-02-09 |
| CAPO: Cost-Aware Prompt Optimization | ✓ Link | 60.2 | | Qwen2.5-32B + CAPO | 2025-04-22 |
| Jamba: A Hybrid Transformer-Mamba Language Model | ✓ Link | 59.9 | | Jamba (3-shot CoT) | 2024-03-28 |
| Fewer is More: Boosting LLM Reasoning with Reinforced Context Pruning | | 59.59 | 70 | LLaMA 2 70B (CoT-Influx) | 2023-12-14 |
| Orca 2: Teaching Small Language Models How to Reason | | 59.14 | 13 | Orca 2 13B | 2023-11-18 |
| Transcending Scaling Laws with 0.1% Extra Compute | | 58.5 | 540 | U-PaLM | 2022-10-20 |
| BitNet b1.58 2B4T Technical Report | ✓ Link | 58.38 | | BitNet b1.58 2B4T | 2025-04-16 |
| Large Language Models are Zero-Shot Reasoners | ✓ Link | 58.1 | 540 | PaLM-540B (few-Shot-cot) | 2022-05-24 |
| GPT-4 Technical Report | ✓ Link | 57.1 | | GPT-3.5 (few-shot, k=5) | 2023-03-15 |
| Refuse without Refusal: A Structural Analysis of Safety-Tuning Responses for Reducing False Refusals in Language Models | ✓ Link | 57.01 | | Llama-3.1-8B + Rationale-Only | 2026-09-04 |
| Solving Quantitative Reasoning Problems with Language Models | ✓ Link | 56.8 | 8 | Minerva 8B (maj5@100) | 2022-06-29 |
| Llama 2: Open Foundation and Fine-Tuned Chat Models | ✓ Link | 56.8 | 70 | LLaMA 2 70B (on-shot) | 2023-07-18 |
| Solving Quantitative Reasoning Problems with Language Models | ✓ Link | 56.5 | 540 | PaLM 540B (8-shot) | 2022-06-29 |
| Large Language Models Can Self-Improve | | 56.5 | 540 | PaLM 540B (CoT Prompting) | 2022-10-20 |
| Training Verifiers to Solve Math Word Problems | ✓ Link | 55.9 | | GPT-3 175B + Verifier | 2021-10-27 |
| Scaling Relationship on Learning Mathematical Reasoning with Large Language Models | ✓ Link | 55.3 | 13 | RFT 13B | 2023-08-03 |
| Large Language Models are Zero-Shot Reasoners | ✓ Link | 55.0 | 175 | Finetuned GPT-3 175B + verifier | 2022-05-24 |
| DeepSeek-VL: Towards Real-World Vision-Language Understanding | ✓ Link | 55.0 | | DeepSeek-VL 7B Chat | 2024-03-08 |
| WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct | ✓ Link | 54.9 | 7 | WizardMath-7B-V1.0 | 2023-08-18 |
| Falcon2-11B Technical Report | | 53.83 | | Falcon2-11B (5-shot) | 2024-07-20 |
| mHC: Manifold-Constrained Hyper-Connections | | 53.8 | | mHC (27B) | 2025-12-31 |
| LLaMA: Open and Efficient Foundation Language Models | ✓ Link | 53.1 | 33 | LLaMA 33B-maj1@k | 2023-02-27 |
| Falcon Mamba: The First Competitive Attention-free 7B Language Model | ✓ Link | 52.54 | | Falcon Mamba 7B (5-shot) | 2024-10-07 |
| Solving Quantitative Reasoning Problems with Language Models | ✓ Link | 52.4 | 62 | Minerva 62B (8-shot) | 2022-06-29 |
| Mistral 7B | ✓ Link | 52.2 | 7 | Mistral 7B (maj@8) | 2023-10-10 |
| Large Language Models are Zero-Shot Reasoners | ✓ Link | 51.5 | 175 | Text-davinci-002-175B (zero-plus-few-Shot-cot (8 samples)) | 2022-05-24 |
| Llemma: An Open Language Model For Mathematics | ✓ Link | 51.5 | 34 | Llemma 34B | 2023-10-16 |
| Compact Language Models via Pruning and Knowledge Distillation | ✓ Link | 51.3 | | Minitron 8B | 2024-07-19 |
| Scaling Relationship on Learning Mathematical Reasoning with Large Language Models | ✓ Link | 51.2 | 7 | RFT 7B | 2023-08-03 |
| LLaMA: Open and Efficient Foundation Language Models | ✓ Link | 50.9 | 65 | LLaMA 65B | 2023-02-27 |
| Learnable Multipliers: Freeing the Scale of Language Model Matrix Layers | ✓ Link | 50.63 | | Falcon-H1-0.5B with Learnable Multipliers (Muon) | 2026-01-08 |
| Universal YOCO for Efficient Depth Scaling | | 50.57 | | YOCO-U (Equal Steps) | 2026-04-01 |
| Power Scheduler: A Batch Size and Token Number Agnostic Learning Rate Scheduler | ✓ Link | 49.7 | | PowerLM-3B (8-shot CoT) | 2024-08-23 |
| SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model | ✓ Link | 48.8 | | SmolLM2 1.7B Instruct | 2025-02-04 |
| Expert Upcycling: Shifting the Compute-Efficient Frontier of Mixture-of-Experts | ✓ Link | 48.3 | | Upcycled 32→64 (100% CPT) | 2026-04-21 |
| Orca 2: Teaching Small Language Models How to Reason | | 47.23 | 7 | Orca 2 7B | 2023-11-18 |
| Flux Attention: Context-Aware Hybrid Attention for Efficient LLMs Inference | ✓ Link | 46.9 | | FluxAttn (FA-TA) (Llama-3.1-8B-Instruct) | 2026-04-08 |
| Gemma: Open Models Based on Gemini Research and Technology | ✓ Link | 46.4 | | Gemma 7B | 2024-03-13 |
| Nemotron-4 15B Technical Report | ✓ Link | 46.0 | | Nemotron-4 15B (8-shot) | 2024-02-26 |
| Coupling Experts and Routers in Mixture-of-Experts via an Auxiliary Loss | | 45.8 | | MoE + ERC | 2025-12-29 |
| OLMoE: Open Mixture-of-Experts Language Models | ✓ Link | 45.5 | | OLMoE-1B-7B-Instruct (8-shot) | 2024-09-03 |
| Scaling Inherently Interpretable Language Models | ✓ Link | 44.4 | | Steerling-8B | 2026-08-06 |
| The Unreasonable Effectiveness of Eccentric Automatic Prompts | | 43 | 13 | Llama-2 13B (on 100 first questions, 4-shot, auto-optimized prompting) | 2024-02-09 |
| RecurrentGemma: Moving Past Transformers for Efficient Open Language Models | ✓ Link | 42.6 | | RecurrentGemma-9B | 2024-04-11 |
| How Lossless Is Lossless Speculative Decoding? The Role of Numerical Precision in Orthrus | | 41.47 | | Orthrus-1.7B-final | 2026-09-14 |
| Large Language Models are Zero-Shot Reasoners | ✓ Link | 41.3 | 175 | text-davinci-002 175B (2-shot, CoT) | 2022-05-24 |
| The Unreasonable Effectiveness of Eccentric Automatic Prompts | | 41 | 7 | Mistral 7B (on 100 first questions, 4-shot, auto-optimized prompting) | 2024-02-09 |
| Large Language Models are Zero-Shot Reasoners | ✓ Link | 40.7 | 175 | text-davinci-002 175B (0-shot, CoT) | 2022-05-24 |
| Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM | ✓ Link | 37.1 | | Branch-Train-MiX 4x7B (sampling top-2 experts) | 2024-03-12 |
| From Reweighting to Rewriting: Unlocking the Intervention Effects of Influential Samples in Training Data Attribution | | 37.07 | | OLMo2-1B + influence rewriting (Opposed Helpful) | 2026-09-02 |
| Llemma: An Open Language Model For Mathematics | ✓ Link | 36.4 | 7 | Llemma 7B | 2023-10-16 |
| LLaMA: Open and Efficient Foundation Language Models | ✓ Link | 35.6 | 33 | LLaMA 33B | 2023-02-27 |
| Frugal LMs Trained to Invoke Symbolic Solvers Achieve Parameter-Efficient Arithmetic Reasoning | ✓ Link | 35.2 | 13 | Vicuna (SYRELM) | 2023-12-09 |
| Code Llama: Open Foundation Models for Code | ✓ Link | 34.420 | | Code Llama - Python 34B | 2023-08-24 |
| Copy-Paste Language Models | ✓ Link | 33.1 | | Copy-Paste Language Model (1.7B, Full) | 2026-10-13 |
| Solving Quantitative Reasoning Problems with Language Models | ✓ Link | 33.0 | 62 | PaLM 62B (8-shot) | 2022-06-29 |
| Sumi: Open Uniform Diffusion Language Model from Scratch | ✓ Link | 32.8 | | Sumi-7B (4-shot) | 2026-06-17 |
| Queryable LoRA: Instruction-Regularized Routing Over Shared Low-Rank Update Atoms | | 32.4 | | Instruction-Queryable LoRA | 2026-05-08 |
| Large Language Models Can Self-Improve | | 32.2 | 540 | PaLM 540B (Self Improvement, Standard-Prompting) | 2022-10-20 |
| SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model | ✓ Link | 31.1 | | SmolLM2 1.7B | 2025-02-04 |
| LLaMA: Open and Efficient Foundation Language Models | ✓ Link | 29.3 | 13 | LLaMA 13B-maj1@k | 2023-02-27 |
| Latent Chain-of-Thought as Planning: Decoupling Reasoning from Verbalization | | 28.66 | | PLaT | 2026-01-29 |
| Solving Quantitative Reasoning Problems with Language Models | ✓ Link | 28.4 | 8 | Minerva 8B-maj1@k (8-shot) | 2022-06-29 |
| Library of Layers (LoL): Adaptively Ordering Transformer Layers at Runtime | ✓ Link | 27.9 | | LoL (3-shot) | 2026-11-02 |
| Denoising Hierarchical Representations: Joint Continuous Diffusion for Language Modeling | ✓ Link | 27.4 | | H-CoBit (n=16) | 2026-10-06 |
| Compact Language Models via Pruning and Knowledge Distillation | ✓ Link | 24.1 | | Minitron 4B | 2024-07-19 |
| How Does Reasoning Flow? Tracing Attention-Induced Information Flow for Targeted RL in LLMs | | 21.1 | | FlowTracer (Llama-3.1-8B) | 2026-06-09 |
| Decouple Searching from Training: Scaling Data Mixing via Model Merging for Large Language Model Pre-training | ✓ Link | 20.98 | | Qwen3 (1.7B) with DeMix (224 proxies) | 2026-01-31 |
| Composing Ensembles of Pre-trained Models via Iterative Consensus | | 20.8 | 0.355 | GPT-2-Medium 355M + question-solution classifier (BS=5) | 2022-10-20 |
| Learning Math Reasoning from Self-Sampled Correct and Partially-Correct Solutions | ✓ Link | 19.5 | 2.7 | GPT-Neo-2.7B + Self-Sampling | 2022-05-28 |
| Composing Ensembles of Pre-trained Models via Iterative Consensus | | 18.3 | 0.355 | GPT-2-Medium 355M (fine-tuned, BS=5) | 2022-10-20 |
| LLaMA: Open and Efficient Foundation Language Models | ✓ Link | 18.1 | 7 | LLaMA 7B (maj1@k) | 2023-02-27 |
| Large Language Models are Zero-Shot Reasoners | ✓ Link | 17.9 | 540 | PaLM 540B (few-shot) | 2022-05-24 |
| Large Language Models Can Self-Improve | | 17.9 | 540 | PaLM 540B (Standard-Prompting) | 2022-10-20 |
| LLaMA: Open and Efficient Foundation Language Models | ✓ Link | 17.8 | 13 | LLaMA 13B | 2023-02-27 |
| Composing Ensembles of Pre-trained Models via Iterative Consensus | | 16.8 | 0.355 | GPT-2-Medium 355M + question-solution classifier (BS=1) | 2022-10-20 |
| Solving Quantitative Reasoning Problems with Language Models | ✓ Link | 16.2 | 8 | Minerva 8B (8-shot) | 2022-06-29 |
| RecurrentGemma: Moving Past Transformers for Efficient Open Language Models | ✓ Link | 13.4 | | RecurrentGemma-2B | 2024-04-11 |
| Composing Ensembles of Pre-trained Models via Iterative Consensus | | 12.2 | 0.355 | GPT-2-Medium 355M (BS=5) | 2022-10-20 |
| LLaMA: Open and Efficient Foundation Language Models | ✓ Link | 11.0 | 7 | LLaMA 7B | 2023-02-27 |
| Large Language Models are Zero-Shot Reasoners | ✓ Link | 10.4 | 175 | Text-davinci-002-175B (0-shot) | 2022-05-24 |
| OLMo: Accelerating the Science of Language Models | ✓ Link | 8.5 | | OLMo 7B (8-shot) | 2024-02-01 |
| Learning Math Reasoning from Self-Sampled Correct and Partially-Correct Solutions | ✓ Link | 7.5 | 0.125 | GPT-Neo 125M + Self-Sampling | 2022-05-28 |
| UL2: Unifying Language Learning Paradigms | ✓ Link | 4.4 | 20 | UL2 20B (chain-of-thought) | 2022-05-10 |
| Training Language Models via Neural Cellular Automata | ✓ Link | 4.4 | | NCA-pretrained 1.6B | 2026-03-09 |
| UL2: Unifying Language Learning Paradigms | ✓ Link | 4.1 | 20 | UL2 20B (0-shot) | 2022-05-10 |
| Solving Quantitative Reasoning Problems with Language Models | ✓ Link | 4.1 | 8 | PaLM 8B (8-shot) | 2022-06-29 |