FactSpotter: Evaluating the Factual Faithfulness of Graph-to-Text Generation

Benchmark Model Rank Results
data-to-text-generation-on-dartT5B Baseline#1BLEU: 48.47METEOR: 40.74FactSpotter: 96.65
data-to-text-generation-on-dartFactT5B#2BLEU: 48.37METEOR: 40.72FactSpotter: 97.60
data-to-text-generation-on-dartJointGT Baseline#4BLEU: 47.51METEOR: 40.43FactSpotter: 95.86
data-to-text-generation-on-dartFactJointGT#5BLEU: 47.39METEOR: 40.32FactSpotter: 97.25
data-to-text-generation-on-webnlgJointGT Baseline#3BLEU: 67.08METEOR: 48.34FactSpotter: 99.09
data-to-text-generation-on-webnlgT5B Baseline#4BLEU: 67.04METEOR: 48.35FactSpotter: 99.44
data-to-text-generation-on-webnlgFactT5B#5BLEU: 67.04METEOR: 48.22FactSpotter: 99.71
data-to-text-generation-on-webnlgFactJointGT#6BLEU: 66.89METEOR: 48.19FactSpotter: 99.67
kg-to-text-generation-on-webnlg-2-0-constrainedFactT5B#1BLEU: 67.08METEOR: 48.34FactSpotter: 99.71
kg-to-text-generation-on-webnlg-2-0-constrainedJointGT Baseline#2BLEU: 67.08METEOR: 48.34FactSpotter: 99.44
kg-to-text-generation-on-webnlg-2-0-constrainedT5B Baseline#3BLEU: 67.04METEOR: 48.35FactSpotter: 99.05
kg-to-text-generation-on-webnlg-2-0-constrainedFactJointGT#4BLEU: 66.89METEOR: 48.19FactSpotter: 99.67
question-generation-on-grailqa-compositionalFactJointGT#1METEOR: 36.21BLEU: 31.25FactSpotter: 97.10
question-generation-on-grailqa-compositionalFactT5B#3METEOR: 35.72BLEU: 31.66FactSpotter: 96.53
question-generation-on-grailqa-iidFactT5B#1BLEU: 46.10METEOR: 42.67FactSpotter: 99.50
question-generation-on-grailqa-iidFactJointGT#4BLEU: 43.61METEOR: 41.65FactSpotter: 99.12
question-generation-on-grailqa-zero-shotFactJointGT#2METEOR: 37.55bleu: 32.46FactSpotter: 94.95
question-generation-on-grailqa-zero-shotFactT5B#3METEOR: 37.39bleu: 32.39FactSpotter: 95.61
table-to-text-generation-on-dartT5B Baseline#1METEOR: 0.4074BLEU: 48.47BERT: 0.9505BLEURT: 0.6749…
table-to-text-generation-on-dartFactT5B#2METEOR: 0.4072BLEU: 48.37BERT: 0.9505BLEURT: 0.6743…
table-to-text-generation-on-dartJointGT Baseline#3METEOR: 0.4043BLEU: 47.51BERT: 0.9492BLEURT: 0.6733…
table-to-text-generation-on-dartFactJointGT#4METEOR: 0.4032BLEU: 47.39BERT: 0.9492BLEURT: 0.6726…
text-generation-on-dartT5B Baseline#1BLEU: 48.74METEOR: 0.4074FactSpotter: 96.65
text-generation-on-dartFactT5B#2BLEU: 48.37METEOR: 0.4072FactSpotter: 97.60
text-generation-on-dartJointGT Baseline#3BLEU: 47.51METEOR: 0.4043FactSpotter: 0.9586
text-generation-on-dartFactJointGT#4BLEU: 47.39METEOR: 0.4032FactSpotter: 97.25