OpenCodePapers

language-modelling-on-openwebtext

Language Modelling
Dataset Link
Results over time
Click legend items to toggle metrics. Hover points for model names.
Leaderboard
Click a sortable column header to sort.
PaperCodeeval_perplexityeval_lossparametersModelNameReleaseDate
Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Masking15.36131MMDLM-Prime2025-05-24
Simple and Effective Masked Diffusion Language Models✓ Link17.54131MARM2024-06-11
Energy-Based Diffusion Language Models for Text Generation17.58131MEDLM-coAR2024-10-28
Polynomial, trigonometric, and tropical activations✓ Link18.392.91124MGPT2-Hermite2025-02-03
Polynomial, trigonometric, and tropical activations✓ Link18.642.92124MGPT2-Tropical2025-02-03
Polynomial, trigonometric, and tropical activations✓ Link18.722.93124MGPT2-Fourier2025-02-03
Polynomial, trigonometric, and tropical activations✓ Link19.242.95124MGPT2-GELU2025-02-03
Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models✓ Link20.73131MBD3-LMs2025-03-12
Energy-Based Diffusion Language Models for Text Generation21.52131MEDLM-NCE2024-10-28
Simplified and Generalized Masked Diffusion for Discrete Data✓ Link21.80131MGenMD42024-06-06
Simple and Effective Masked Diffusion Language Models✓ Link22.98131MMDLM2024-06-11
Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution✓ Link24.10131MSEDD2023-10-25
Check out how to contribute new results.