Pay Less Attention with Lightweight and Dynamic Convolutions

Benchmark Model Rank Results
abstractive-text-summarization-on-cnn-dailyDynamic Conv#35ROUGE-1: 39.84ROUGE-2: 16.25ROUGE-L: 36.73
document-summarization-on-cnn-daily-mailDynamicConv#19ROUGE-1: 39.84ROUGE-2: 16.25ROUGE-L: 36.73
document-summarization-on-cnn-daily-mailLightConv#20ROUGE-1: 39.52ROUGE-2: 15.97ROUGE-L: 36.51
language-modelling-on-one-billion-wordDynamicConv#14PPL: 26.67Number of params: 0.34B
machine-translation-on-iwslt2014-germanDynamicConv#22BLEU score: 35.2
machine-translation-on-iwslt2014-germanLightConv#23BLEU score: 34.8
machine-translation-on-wmt2014-english-frenchDynamicConv#13BLEU score: 43.2
machine-translation-on-wmt2014-english-frenchLightConv#15BLEU score: 43.1
machine-translation-on-wmt2014-english-germanDynamicConv#18BLEU score: 29.7Number of Params: 213M
machine-translation-on-wmt2014-english-germanLightConv#33BLEU score: 28.9Number of Params: 202M