N-Grammer: Augmenting Transformers with latent n-grams

Benchmark Model Rank Results
common-sense-reasoning-on-recordN-Grammer 343M#13EM: 28.9F1: 29.9
coreference-resolution-on-winograd-schemaN-Grammer 343M#32Accuracy: 68.3
language-modelling-on-c4N-Grammer 343M#7Perplexity: 14.79
language-modelling-on-c4N-Grammer 288M#8Perplexity: 15.01
natural-language-inference-on-commitmentbankN-Grammer 343M#12Accuracy: 67.9F1: 59.7
natural-language-inference-on-rteN-Grammer 343M#64Accuracy: 59.2%
question-answering-on-boolqN-Grammer 343M#45Accuracy: 65
question-answering-on-copaN-Grammer 343M#45Accuracy: 60.0
question-answering-on-multircN-Grammer 343M#17F1: 62EM: 11.3
word-sense-disambiguation-on-words-in-contextN-Grammer 343M#13Accuracy: 56.1