Reducing Transformer Depth on Demand with Structured Dropout

Open paper
Benchmark Model Rank Results
open-domain-question-answering-on-eli5Transformer Multitask + LayerDrop#5Rouge-L: 23.4Rouge-1: 29.4Rouge-2: 5.5

Rank counts only results with a code link.