FNet: Mixing Tokens with Fourier Transforms

Benchmark Model Rank Results
linguistic-acceptability-on-colaFNet-Large#7Accuracy: 78%
natural-language-inference-on-multinliBERT-Large#13Matched: 88Mismatched: 88
natural-language-inference-on-multinliFNet-Large#36Matched: 78Mismatched: 76
natural-language-inference-on-qnliFNet-Large#34Accuracy: 85%
natural-language-inference-on-rteFNet-Large#51Accuracy: 69%
paraphrase-identification-on-quora-questionFNet-Large#5F1: 85
semantic-textual-similarity-on-mrpcFNet-Large#17Accuracy: 88%
semantic-textual-similarity-on-sts-benchmarkFNet-Large#44Spearman Correlation: 0.84
sentiment-analysis-on-sst-2-binaryFNet-Large#34Accuracy: 94