| The Falcon 3 Family of Open Models | | 95.3 | Falcon3-7B-Base (lm-evaluation-harness) | 2024-12-17 |
| Decoding Looped Transformers Better for (Almost) Free | | 95.3 | LoopCD-Ouro-1.4B (fixed) (lm-evaluation-harness) | 2026-10-01 |
| OLMo: Accelerating the Science of Language Models | ✓ Link | 93.8 | OLMo 7B (Catwalk) | 2024-02-01 |
| GPT-NeoX-20B: An Open-Source Autoregressive Language Model | ✓ Link | 92.8 | GPT-NeoX-20B (LM Evaluation Harness) | 2022-04-14 |
| Distributionally Robust Mixture-of-Experts Training | ✓ Link | 91.1 | DRMoET (1.7B active, 10.3B total, eta=0.001) | 2026-10-05 |
| Copy-Paste Language Models | ✓ Link | 89.2 | Copy-Paste Language Model (1.7B, Full) | 2026-10-13 |
| Parallax: Parameterized Local Linear Attention for Language Modeling | ✓ Link | 88.7 | Parallax (1.7B) (LM Evaluation Harness) | 2026-05-27 |
| The State-Prediction Separation Hypothesis | ✓ Link | 87.5 | SPS XL (LM Evaluation Harness) | 2026-07-01 |
| MeKi: Memory-based Expert Knowledge Injection for Efficient LLM Scaling | ✓ Link | 87.2 | MeKi (4B) | 2026-02-03 |
| Error-Free Linear Attention is a Free Lunch: Exact Solution from Continuous-Time Dynamics | ✓ Link | 84.2 | EFLA | 2025-12-14 |
| MDM-Prime-v2: Binary Encoding and Index Shuffling Enable Scaling of Diffusion Language Models | ✓ Link | 83.3 | MDM-Prime-v2 (1.1B) (lm-evaluation-harness) | 2026-03-17 |