Scaling Embeddings Outperforms Scaling Experts in Language Models

Benchmark Model Rank Results
code-generation-on-swe-bench-multilingualLongCat-Flash-Lite (68.5B)–Accuracy: 38.1
mathematical-reasoning-on-aime-2025LongCat-Flash-Lite (68.5B)–Accuracy: 63.23
mathematical-reasoning-on-aime24LongCat-Flash-Lite (68.5B)–Acc: 72.19
mathematical-reasoning-on-math-500LongCat-Flash-Lite (68.5B)–Accuracy: 96.8
multi-task-language-understanding-on-c-evalLongCat-Flash-Lite (68.5B)–Accuracy: 86.55
multi-task-language-understanding-on-mmlu-proLongCat-Flash-Lite (68.5B)–Accuracy: 78.29
multiple-choice-question-answering-mcqa-on-gpqa-diamondLongCat-Flash-Lite (68.5B)–Accuracy: 66.78