GLM: General Language Model Pretraining with Autoregressive Blank Infilling

Benchmark Model Rank Results
document-summarization-on-cnn-daily-mailGLM-XXLarge#4ROUGE-1: 44.7ROUGE-2: 21.4ROUGE-L: 41.4
language-modelling-on-wikitext-103GLM-XXLarge (bidirectional)#4Test perplexity: 11.33Number of params: 10000M
language-modelling-on-wikitext-103GLM-XXLarge (unidirectional)#6Test perplexity: 12.22Number of params: 10000M
abstractive-text-summarization-on-cnn-daily-mailGLM-XXLarge#8ROUGE-1: 44.7ROUGE-2: 21.4ROUGE-L: 41.4
language-modelling-on-lambadaGLM-XXLarge (bidirectional)#19Accuracy: 72.35
language-modelling-on-lambadaGLM-XXLarge (unidirectional)#30Accuracy: 67.18