Lila: A Unified Benchmark for Mathematical Reasoning

Benchmark Model Rank Results
mathematical-reasoning-on-lila-iidCodex (Few-Shot, 175B)#1Accuracy: 0.604
mathematical-reasoning-on-lila-iidBhāskara-P (Fine-tuned, 2.7B)#2Accuracy: 0.48
mathematical-reasoning-on-lila-iidNeo-P (Fine-tuned, 2.7B)#3Accuracy: 0.394
mathematical-reasoning-on-lila-iidGPT-3 (Few-Shot, 175B)#4Accuracy: 0.384
mathematical-reasoning-on-lila-iidBhāskara-A (Fine-tuned, 2.7B)#5Accuracy: 0.252
mathematical-reasoning-on-lila-iidNeo-A (Fine-tuned, 2.7B)#6Accuracy: 0.204
mathematical-reasoning-on-lila-oodCodex (Few-Shot, 175B)#1Accuracy: 0.586
mathematical-reasoning-on-lila-oodBhāskara-P (Fine-tuned, 2.7B)#2Accuracy: 0.448
mathematical-reasoning-on-lila-oodGPT-3 (Few-Shot, 175B)#3Accuracy: 0.384
mathematical-reasoning-on-lila-oodBhāskara-A (Fine-tuned, 2.7B)#4Accuracy: 0.268
mathematical-reasoning-on-lila-oodNeo-P (Fine-tuned, 2.7B)#5Accuracy: 0.238
mathematical-reasoning-on-lila-oodNeo-A (Fine-tuned, 2.7B)#6Accuracy: 0.177