| Nemotron-4 15B Technical Report | ✓ Link | 60.9 | Nemotron-4 15B | 2024-02-26 |
| Gemma 2: Improving Open Language Models at a Practical Size | ✓ Link | 53.7 | Gemma 2 27B | 2024-07-31 |
| MobileMoE: Scaling On-Device Mixture of Experts | | 53.4 | MobileMoE-L (922M) | 2026-05-26 |
| LLaMA: Open and Efficient Foundation Language Models | ✓ Link | 52.3 | LLaMA-65B | 2023-02-27 |
| RecurrentGemma: Moving Past Transformers for Efficient Open Language Models | ✓ Link | 52.3 | RecurrentGemma-9B | 2024-04-11 |
| Gemma: Open Models Based on Gemini Research and Technology | ✓ Link | 51.8 | Gemma 7B | 2024-03-13 |
| RecurrentGemma: Moving Past Transformers for Efficient Open Language Models | ✓ Link | 51.8 | RecurrentGemma-2B | 2024-04-11 |
| Encoder-Decoder Gemma: Improving the Quality-Efficiency Trade-Off via Adaptation | ✓ Link | 50.5 | T5Gemma 9B-9B PrefixLM | 2025-04-08 |
| Data Science and Technology Towards AGI Part I: Tiered Data Management | ✓ Link | 43.35 | MiniCPM-1.2B (Math-L3) | 2026-02-09 |
| PLDR-LLMs Reason At Self-Organized Criticality | ✓ Link | 43.09 | PLDRv51-SOC-110M-5 | 2026-03-12 |
| Maglev: Sliding Recurrent Memory | | 42.7 | Maglev (separate parameters, lambda=0.1) | 2026-08-05 |
| Memory Caching: RNNs with Growing Memory | | 42.6 | Titans + GRM | 2026-02-27 |
| Modular TTT: Rethinking Test-Time Training as Composable Modules | ✓ Link | 42.32 | Modular TTT 1.45B (inner-product M-SiLU) | 2026-08-07 |
| SonicMoE: Accelerating MoE with IO and Tile-aware Optimizations | ✓ Link | 42.1 | Token Rounding (1.8B MoE) | 2025-12-16 |
| MDM-Prime-v2: Binary Encoding and Index Shuffling Enable Scaling of Diffusion Language Models | ✓ Link | 42.02 | MDM-Prime-v2 (1.1B) | 2026-03-17 |
| Gated Delta Networks: Improving Mamba2 with Delta Rule | ✓ Link | 41.91 | Gated DeltaNet-H2 | 2024-12-09 |
| Gated DeltaNet-2: Decoupling Erase and Write in Linear Attention | ✓ Link | 41.5 | Gated DeltaNet-2 (1.3B) | 2026-05-21 |
| Looping Beyond Twice: A Scalable Recipe for Looped Mixture-of-Experts | ✓ Link | 40.0 | LOOM (1.7B, 12 loops) | 2026-10-01 |