| PaLM: Scaling Language Modeling with Pathways | ✓ Link | 88.4 | PaLM 540B | 2022-04-05 |
| Dream 7B: Diffusion Large Language Models | | 83.9 | Dream 7B | 2025-08-21 |
| PaLM: Scaling Language Modeling with Pathways | ✓ Link | 80.6 | PaLM 62B | 2022-04-05 |
| Mamba-3: Improved Sequence Modeling using State Space Principles | ✓ Link | 76.5 | Mamba-3-MIMO-1.5B | 2026-03-16 |
| Mamba-3: Improved Sequence Modeling using State Space Principles | ✓ Link | 75.9 | Mamba-3-SISO-1.5B | 2026-03-16 |
| Mamba-3: Improved Sequence Modeling using State Space Principles | ✓ Link | 74.7 | Mamba-3-MIMO-880M | 2026-03-16 |
| Modular TTT: Rethinking Test-Time Training as Composable Modules | ✓ Link | 73.74 | Modular TTT 1.45B (inner-product M-SiLU) | 2026-08-07 |
| Mamba-3: Improved Sequence Modeling using State Space Principles | ✓ Link | 72.7 | Mamba-3-SISO-880M | 2026-03-16 |
| Distributionally Robust Mixture-of-Experts Training | ✓ Link | 72.1 | DRMoET (1.7B active, 10.3B total, eta=0.01) | 2026-10-05 |
| PaLM: Scaling Language Modeling with Pathways | ✓ Link | 71.3 | PaLM 8B | 2022-04-05 |
| Language Models are Few-Shot Learners | ✓ Link | 70.1 | GPT-3 175B (few-shot) | 2020-05-28 |
| Maglev: Sliding Recurrent Memory | | 70.0 | Maglev (lambda=0.1) | 2026-08-05 |
| Language Models are Few-Shot Learners | ✓ Link | 69.1 | GPT-3 13B (few-shot) | 2020-05-28 |
| Mamba-3: Improved Sequence Modeling using State Space Principles | ✓ Link | 68.9 | Mamba-3-SISO-440M | 2026-03-16 |
| Language Models are Few-Shot Learners | ✓ Link | 68.8 | GPT-3 175B (zero-shot) | 2020-05-28 |
| Copy-Paste Language Models | ✓ Link | 68.1 | Copy-Paste Language Model (1.7B, Full) | 2026-10-13 |
| Language Models are Few-Shot Learners | ✓ Link | 65.8 | GPT-3 6.7B (few-shot) | 2020-05-28 |
| Scaling Laws for Looped Mixture of Experts | | 64.3 | A0.3B-1.3B LoopMoE (R=5) | 2026-09-30 |
| Grouped Value Attention: Efficient KV Caching via On-Demand Key Reconstruction | | 63.81 | GVA + DRoPE d_r=16 | 2026-09-08 |
| Language Models are Few-Shot Learners | ✓ Link | 63.8 | GPT-3 13B (zero-shot) | 2020-05-28 |
| Mamba-3: Improved Sequence Modeling using State Space Principles | ✓ Link | 60.6 | Mamba-3-MIMO-180M | 2026-03-16 |
| Language Models are Few-Shot Learners | ✓ Link | 60.2 | GPT-3 6.7B (zero-shot) | 2020-05-28 |
| Language Models are Few-Shot Learners | ✓ Link | 59.1 | GPT-3 XL (few-shot) | 2020-05-28 |
| Circuit Hypernetworks for Quantum-Augmented Diffusion Language Models | | 55.5 | HyperQ, 64 q | 2026-09-21 |
| Language Models are Few-Shot Learners | ✓ Link | 53.8 | GPT-3 XL (zero-shot) | 2020-05-28 |
| Daedalus-150M: A Convolution-Attention Hybrid Designed for CPU Inference | ✓ Link | 50.42 | Daedalus-150M | 2026-08-20 |
| ParameterNet: Parameters Are All You Need for Large-scale Visual Pretraining of Mobile Networks | ✓ Link | 49.58 | ParameterNet-MoE-up-proj-8 | 2024-06-17 |
| WhiteMatter: All-to-All Cross-Layer Connections via KV Source Mixing | ✓ Link | 46.21 | WhiteMatter k=16 | 2026-09-27 |
| Embeddings Are Overpriced? A Frozen 16 MB Vocabulary for Small and Ternary Language Models | | 45.9 | A learned table, float | 2026-10-15 |
| Gated Recurrent Transformers: Expressive Depth through Recurrent Modulation | ✓ Link | 45.54 | GRT Large (isoParams, zero-shot) | 2026-08-15 |
| Language Models are Few-Shot Learners | ✓ Link | 43.6 | GPT-3 Small (zero-shot) | 2020-05-28 |
| Language Models are Few-Shot Learners | ✓ Link | 42.7 | GPT-3 Small (few-shot) | 2020-05-28 |