| Paper | Code | Average F1 | Accuracy | ModelName | ReleaseDate |
|---|---|---|---|---|---|
| RussianSuperGLUE: A Russian Language Understanding Evaluation Benchmark | ✓ Link | 0.68 | 0.702 | Human Benchmark | 2020-10-29 |
| []() | 0.452 | 0.484 | RuBERT conversational | ||
| []() | 0.417 | 0.484 | RuGPT3Large | ||
| []() | 0.408 | 0.447 | YaLM 1.0B few-shot | ||
| []() | 0.406 | 0.546 | Golden Transformer | ||
| Unreasonable Effectiveness of Rule-Based Heuristics in Solving Russian SuperGLUE Tasks | 0.4 | 0.438 | heuristic majority | 2021-05-03 | |
| []() | 0.372 | 0.461 | RuGPT3Medium | ||
| []() | 0.371 | 0.452 | SBERT_Large | ||
| []() | 0.367 | 0.463 | RuBERT plain | ||
| []() | 0.367 | 0.445 | Multilingual Bert | ||
| mT5: A massively multilingual pre-trained text-to-text transformer | ✓ Link | 0.366 | 0.454 | MT5 Large | 2020-10-22 |
| []() | 0.357 | 0.518 | ruRoberta-large finetune | ||
| []() | 0.356 | 0.5 | ruBert-large finetune | ||
| []() | 0.356 | 0.473 | RuGPT3Small | ||
| []() | 0.351 | 0.486 | SBERT_Large_mt_ru_finetuning | ||
| []() | 0.333 | 0.509 | ruBert-base finetune | ||
| Unreasonable Effectiveness of Rule-Based Heuristics in Solving Russian SuperGLUE Tasks | 0.319 | 0.374 | Random weighted | 2021-05-03 | |
| []() | 0.307 | 0.468 | ruT5-base-finetune | ||
| []() | 0.306 | 0.498 | ruT5-large-finetune | ||
| []() | 0.302 | 0.418 | RuGPT3XL few-shot | ||
| RussianSuperGLUE: A Russian Language Understanding Evaluation Benchmark | ✓ Link | 0.301 | 0.441 | Baseline TF-IDF1.1 | 2020-10-29 |
| Unreasonable Effectiveness of Rule-Based Heuristics in Solving Russian SuperGLUE Tasks | 0.217 | 0.484 | majority_class | 2021-05-03 |