Encoder-Decoder Models Can Benefit from Pre-trained Masked Language Models in Grammatical Error Correction

Open paper

Benchmark	Model	Rank	Results
grammatical-error-correction-on-bea-2019-test	Transformer + Pre-train with Pseudo Data (+BERT)	#15	F0.5: 69.8
grammatical-error-correction-on-conll-2014-shared-task	Transformer + Pre-train with Pseudo Data (+BERT)	#13	F0.5: 65.2
grammatical-error-correction-on-jfleg	Transformer + Pre-train with Pseudo Data + BERT	#2	GLEU: 62.0