Taylor-Calibrate: Principled Initialization for Hybrid Linear Attention Distillation

Benchmark Model Rank Results
question-answering-on-copa-zero-shotTaylor-Calibrate qwen3_8b 100M_ga_s2_taylor#2Accuracy: 87.0
common-sense-reasoning-on-arc-easyTaylor-Calibrate qwen3_8b 700M_ga_s2_taylor#19Accuracy: 80.72
question-answering-on-boolqTaylor-Calibrate qwen3_8b 700M_uniform_taylor#28Accuracy: 84.56
question-answering-on-piqaTaylor-Calibrate qwen2_3b 100M_ga_s2_taylor#70Accuracy: 78.4
sentence-completion-on-hellaswagTaylor-Calibrate qwen3_8b 700M_uniform_taylor#97Accuracy: 74.37