Acceptability Judgements via Examining the Topology of Attention Maps

Benchmark Model Rank Results
linguistic-acceptability-on-colaEn-BERT + TDA + PCA#1Accuracy: 88.6%
linguistic-acceptability-on-colaEn-BERT + TDA#6Accuracy: 82.1%MCC: 0.565
linguistic-acceptability-on-cola-devEn-BERT + TDA#1Accuracy: 88.6MCC: 0.725
linguistic-acceptability-on-cola-devXLM-R (pre-trained) + TDA#2Accuracy: 73
linguistic-acceptability-on-cola-devEn-BERT (pre-trained) + TDA#6MCC: 0.420
linguistic-acceptability-on-itacolaXLM-R + TDA#1MCC: 0.683Accuracy: 92.8
linguistic-acceptability-on-itacolaIt-BERT (pre-trained) + TDA#3MCC: 0.478Accuracy: 89.2