RAFT: A Real-World Few-Shot Text Classification Benchmark

Benchmark Model Rank Results
few-shot-text-classification-on-raftHuman (crowdsourced)#2Avg: 0.735ADE: 0.830B77: 0.607NIS: 0.857OSE: 0.646…
few-shot-text-classification-on-raftGPT-3#3Avg: 0.627ADE: 0.686B77: 0.299NIS: 0.679OSE: 0.431…
few-shot-text-classification-on-raftAdaBoost#4Avg: 0.514ADE: 0.543B77: 0.023NIS: 0.626OSE: 0.475…
few-shot-text-classification-on-raftGPT-Neo#5Avg: 0.481ADE: 0.452B77: 0.149NIS: 0.408OSE: 0.343…
few-shot-text-classification-on-raftGPT-2#6Avg: 0.458ADE: 0.600B77: 0.121NIS: 0.561OSE: 0.245…
few-shot-text-classification-on-raftBART MNLI zero-shot#7Avg: 0.382ADE: 0.234B77: 0.332NIS: 0.615OSE: 0.360…
few-shot-text-classification-on-raftPlurality-class#8Avg: 0.331ADE: 0.446B77: 0.000NIS: 0.353OSE: 0.164…
few-shot-text-classification-on-raftGPT-3 zero-shot#9Avg: 0.292ADE: 0.163B77: 0.000NIS: 0.572OSE: 0.323…