| linguistic-acceptability-on-cola | ELC-BERT-base 98M | – | Accuracy: 82.6 |
| linguistic-acceptability-on-cola | ELC-BERT-small 24M | – | Accuracy: 76.1 |
| linguistic-acceptability-on-cola | LTG-BERT-base 98M | – | Accuracy: 82.7 |
| linguistic-acceptability-on-cola | LTG-BERT-small 24M | – | Accuracy: 77.6 |
| natural-language-inference-on-multinli | ELC-BERT-base 98M (zero init) | – | Matched: 84.4Mismatched: 84.5 |
| natural-language-inference-on-multinli | ELC-BERT-small 24M | – | Matched: 79.2Mismatched: 79.9 |
| natural-language-inference-on-multinli | LTG-BERT-base 98M | – | Matched: 83Mismatched: 83.4 |
| natural-language-inference-on-multinli | LTG-BERT-small 24M | – | Matched: 78Mismatched: 78.8 |
| natural-language-inference-on-rte | ELC-BERT-base 98M (zero init) | – | Accuracy: 63 |
| natural-language-inference-on-rte | ELC-BERT-small 24M | – | Accuracy: 55.4 |
| natural-language-inference-on-rte | LTG-BERT-base 98M | – | Accuracy: 54.7 |
| natural-language-inference-on-rte | LTG-BERT-small 24M | – | Accuracy: 53.7 |