OPT: Open Pre-trained Transformer Language Models

Benchmark Model Rank Results
question-answering-on-copa-zero-shotOPT-175B#1Accuracy: 87
stereotypical-bias-analysis-on-crows-pairsGPT-3#2Gender: 62.6Religion: 62.6Race/Color: 64.7Sexual Orientation: 76.2…
stereotypical-bias-analysis-on-crows-pairsOPT-175B#3Gender: 65.7Religion: 65.7Race/Color: 68.6Sexual Orientation: 78.6…
zero-shot-question-answering-on-openbookqaOPT-175B#2Accuracy: 58.6
hate-speech-detection-on-ethos-binaryOPT-175B (few-shot)#3F1-score: 0.759
hate-speech-detection-on-ethos-binaryOPT-175B (one-shot)#5F1-score: 0.713
hate-speech-detection-on-ethos-binaryOPT-175B (zero-shot)#6F1-score: 0.667
hate-speech-detection-on-ethos-binaryDavinci (zero-shot)#9F1-score: 0.628
hate-speech-detection-on-ethos-binaryDavinci (one-shot)#10F1-score: 0.616
hate-speech-detection-on-ethos-binaryDavinci (few-shot)#11F1-score: 0.354
question-answering-on-piqaOPT-175B#59Accuracy: 80.1
common-sense-reasoning-on-arc-easyOPT-175B#71Accuracy: 67
common-sense-reasoning-on-winograndeOPT-175B#72Accuracy: 71.4
question-answering-on-boolqOPT-175B#75Accuracy: 65
sentence-completion-on-hellaswagOPT-175B#83Accuracy: 78.5