GOLD: Generalized Knowledge Distillation via Out-of-Distribution-Guided Language Data Generation

Open paper

Benchmark	Model	Rank	Results
data-free-knowledge-distillation-on-qnli	GOLD (T5-base)	–	Accuracy: 91.7
data-free-knowledge-distillation-on-squad	GOLD (T5-base)	–	Exact Match: 75.2