The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics

Benchmark Model Rank Results
data-to-text-generation-on-cleaned-e2e-nlg-1BARTMETEOR (Validation set): 0.373
data-to-text-generation-on-cleaned-e2e-nlg-1LSTMMETEOR (Validation set): 0.394
data-to-text-generation-on-cleaned-e2e-nlg-1T5METEOR (Validation set): 0.369
data-to-text-generation-on-cleaned-e2e-nlg-1TGenMETEOR (Validation set): 0.391
data-to-text-generation-on-tottoT5METEOR: 0.363
extreme-summarization-on-gem-xsumPEGASUSROUGE-2: 23.2Parameters: 568 M
text-generation-on-commongen-1BARTMETEOR: 0.301
text-generation-on-commongen-1T5METEOR: 0.291
text-generation-on-dartBARTMETEOR: 0.107
text-generation-on-dartT5METEOR: 0.115
text-simplification-on-assetBARTMETEOR: 0.560
text-simplification-on-assetT5METEOR: 0.581
text-simplification-on-turkcorpusBARTMETEOR: 0.556
text-simplification-on-turkcorpusT5METEOR: 0.649