GLM: General Language Model Pretraining with Autoregressive Blank Infilling

Benchmark Model Rank Results
abstractive-text-summarization-on-cnn-dailyGLM-XXLarge#8ROUGE-1: 44.7ROUGE-2: 21.4ROUGE-L: 41.4
document-summarization-on-cnn-daily-mailGLM-XXLarge#4ROUGE-1: 44.7ROUGE-2: 21.4ROUGE-L: 41.4
language-modelling-on-lambadaGLM-XXLarge (bidirectional)#16Accuracy: 72.35
language-modelling-on-lambadaGLM-XXLarge (unidirectional)#21Accuracy: 67.18
language-modelling-on-wikitext-103GLM-XXLarge (bidirectional)#4Test perplexity: 11.33Number of params: 10000M
language-modelling-on-wikitext-103GLM-XXLarge (unidirectional)#5Test perplexity: 12.22Number of params: 10000M