Every Token Counts: Generalizing 16M Ultra-Long Context in Large Language Models

Benchmark Model Rank Results
math-word-problem-solving-on-mathHSA-UltraLong-Inst#46Accuracy: 61.56
mathematical-reasoning-on-olympiadbenchHSA-UltraLong-Inst#47Accuracy: 27.85
question-answering-on-piqaHSA-UltraLong-MoE#51Accuracy: 80.69
code-generation-on-humanevalHSA-UltraLong-Inst#84Pass@1: 71.95
code-generation-on-mbppHSA-UltraLong-Inst#94Accuracy: 57
sentence-completion-on-hellaswagHSA-UltraLong-MoE#121Accuracy: 67.43
arithmetic-reasoning-on-gsm8kHSA-UltraLong-Inst#133Accuracy: 82.94