TAPE: Assessing Few-shot Russian Language Understanding

Benchmark Model Rank Results
ethics-on-ethicsRuGPT-3 Large#1Accuracy: 68.6
ethics-on-ethicsRuGPT-3 Meduim#2Accuracy: 68.3
ethics-on-ethicsRuGPT-3 Small#3Accuracy: 55.5
ethics-on-ethicsHuman benchmark#4Accuracy: 52.9
ethics-on-ethics-2Human benchmark#1Accuracy: 67.6
ethics-on-ethics-2RuGPT-3 Small#2Accuracy: 60.9
ethics-on-ethics-2RuGPT-3 Large#3Accuracy: 44.9
ethics-on-ethics-2RuGPT-3 Medium#4Accuracy: 44.1
logical-reasoning-on-ruworldtreeHuman benchmark#1Accuracy: 83.7
logical-reasoning-on-ruworldtreeRuGPT-3 Large#2Accuracy: 40.7
logical-reasoning-on-ruworldtreeRuGPT-3 Medium#3Accuracy: 38.0
logical-reasoning-on-ruworldtreeRuGPT-3 Small#4Accuracy: 34.0
logical-reasoning-on-winograd-automaticHuman benchmark#1Accuracy: 87.0
logical-reasoning-on-winograd-automaticRuGPT-3 Small#2Accuracy: 57.9
logical-reasoning-on-winograd-automaticRuGPT-3 Medium#3Accuracy: 57.2
logical-reasoning-on-winograd-automaticRuGPT-3 Large#4Accuracy: 55.5
question-answering-on-chegekaHuman benchmark#1Accuracy: 64.5
question-answering-on-chegekaRuGPT-3 Large#2Accuracy: 00
question-answering-on-chegekaRuGPT-3 Medium#3Accuracy: 00
question-answering-on-chegekaRuGPT-3 Small#4Accuracy: 00
question-answering-on-multiqHuman benchmark#1Accuracy: 91.0
question-answering-on-multiqRuGPT-3 Large#2Accuracy: 00
question-answering-on-multiqRuGPT-3 Medium#3Accuracy: 00
question-answering-on-multiqRuGPT-3 Small#4Accuracy: 00
question-answering-on-ruopenbookqaHuman benchmark#1Accuracy: 86.5
question-answering-on-ruopenbookqaRuGPT-3 Small#2Accuracy: 57.9
question-answering-on-ruopenbookqaRuGPT-3 Medium#3Accuracy: 57.2
question-answering-on-ruopenbookqaRuGPT-3 Large#4Accuracy: 55.5