Sparsifying Transformer Models with Trainable Representation Pooling

Benchmark Model Rank Results
text-summarization-on-arxivBlockwise(baseline)–ROUGE-1: 46.85ROUGE-2: 19.39
text-summarization-on-arxivDeepPyramidion–ROUGE-1: 47.15ROUGE-2: 19.99
text-summarization-on-arxiv-summarizationDeepPyramidion#2ROUGE-1: 47.15ROUGE-2: 19.99
text-summarization-on-arxiv-summarizationBlockwise (baseline)#3ROUGE-1: 46.85ROUGE-2: 19.39
text-summarization-on-pubmedDeepPyramidion#8ROUGE-1: 47.81ROUGE-2: 21.14