The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics

Benchmark Model Rank Results
data-to-text-generation-on-cleaned-e2e-nlg-challengeBART–METEOR (Validation set): 0.373
data-to-text-generation-on-cleaned-e2e-nlg-challengeLSTM–METEOR (Validation set): 0.394
data-to-text-generation-on-cleaned-e2e-nlg-challengeT5–METEOR (Validation set): 0.369
data-to-text-generation-on-cleaned-e2e-nlg-challengeTGen–METEOR (Validation set): 0.391
data-to-text-generation-on-tottoT5–METEOR: 0.363
extreme-summarization-on-gem-xsumPEGASUS–ROUGE-2: 23.2Parameters: 568 M
text-generation-on-commongenBART–METEOR: 0.301
text-generation-on-commongenT5–METEOR: 0.291
text-generation-on-dartBART–METEOR: 0.107
text-generation-on-dartT5–METEOR: 0.115
text-simplification-on-assetBART–METEOR: 0.560
text-simplification-on-assetT5–METEOR: 0.581
text-simplification-on-turkcorpusBART–METEOR: 0.556
text-simplification-on-turkcorpusT5–METEOR: 0.649