OpenCodePapers

question-answering-on-squad20

Question Answering
Dataset Link
Results over time
Click legend items to toggle metrics. Hover points for model names.
Leaderboard
Click a sortable column header to sort.
PaperCodeEMF1ModelNameReleaseDate
[]()90.93993.214IE-Net (ensemble)
[]()90.87193.183FPNet (ensemble)
[]()90.86093.100IE-NetV2 (ensemble)
[]()90.72493.011SA-Net on Albert (ensemble)
[]()90.67992.948SA-Net-V2 (ensemble)
[]()90.60092.899FPNet (ensemble)
Retrospective Reader for Machine Reading Comprehension✓ Link90.57892.978Retro-Reader (ensemble)2020-01-27
[]()90.52192.824EntitySpanFocusV2 (ensemble)
[]()90.48792.894TransNets + SFVerifier + SFEnsembler (ensemble)
[]()90.45492.748EntitySpanFocus+AT (ensemble)
[]()90.44292.877ATRLP+PV (ensemble)
[]()90.42092.807LANetV2 (ensemble)
[]()90.42092.799ELECTRA+ALBERT+EntitySpanFocus (ensemble)
[]()90.38692.777ALBERT + DAAF + Verifier
[]()90.28492.691electra+nlayers+adv+ds
[]()90.19492.594MixEnsemble (ensemble)
[]()90.12692.535electra+nlayers (ensemble)
[]()90.11592.580Retro-Reader on ALBERT (ensemble)
Retrospective Reader for Machine Reading Comprehension✓ Link90.11592.580Retro-Reader on ALBERT (ensemble)2020-01-27
[]()90.08192.457ANet
[]()90.05992.517Answer Dependent Classify (single model)
[]()90.00292.497electra+nlayers+kdav (ensemble)
[]()90.00292.425ALBERT + DAAF + Verifier (ensemble)
[]()89.92392.425LANet (ensemble)
[]()89.77792.312Electra-nlayers (ensemble)
[]()89.74392.180Albert_Verifier_AA_Net (ensemble)
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations✓ Link89.73192.215ALBERT (ensemble model)2019-09-26
[]()89.56292.226Span Extract + Classify (single model)
Retrospective Reader for Machine Reading Comprehension✓ Link89.56292.052Retro-Reader on ELECTRA (single model)2020-01-27
[]()89.55192.366ELECTRA+ATRLP+PV (single model)
[]()89.52892.059albert+transform+verify (ensemble)
[]()89.46192.134albert+KD+transfer (ensemble)
[]()89.44992.118ROaD-Electra (single model)
[]()89.44992.118ROaD-Electra
[]()89.40491.964Albert-nlayers (ensemble)
[]()89.34891.985ELECTRA + E-Verifier (ensemble)
[]()89.32591.994ELECTRA + ROBERTA + ALBERT (ensemble)
[]()89.32591.9392task (single model)
[]()89.23591.900Deberta
[]()89.23591.739ALBERT + MTDA + SFVerifier (ensemble model)
[]()89.22491.853ALBert-LSTM (ensemble)
[]()89.13391.666ALBERT + SFVerifier (ensemble model)
[]()89.02191.765ELECTRA+RL+EV (single model)
[]()88.99891.635electra & albert (ensemble)
[]()88.99891.635AE-TEST
[]()88.87491.546ELECTRA+EntitySpanFocus (Single model)
[]()88.85191.486SA-Net on Electra (single model)
[]()88.76191.745ALBERT+Entailment DA (ensemble)
[]()88.71691.365ELECTRA (single model)
[]()88.63791.230Tuned ALBERT (ensemble model)
[]()88.61491.303ELECTRA_ATT (single model)
[]()88.60391.299Deberta+prefix
[]()88.59291.286ALBERT (Single model)
[]()88.59290.859XLNet + DAAF + Verifier (ensemble)
[]()88.56991.287ALBERT + IG + NE (single model)
[]()88.52491.256ALBERT + IG (single model)
[]()88.43490.918aanet_v2.0 (single model)
[]()88.35591.019albert+verifier (single model)
[]()88.29891.078albert+KD+transfer (single)
[]()88.23190.713UPM (ensemble)
[]()88.19790.830ALBERT + SFVerifier (single model)
[]()88.18690.939{alber_m_transfor} (single model)
SG-Net: Syntax-Guided Machine Reading Comprehension✓ Link88.17490.702XLNet + SG-Net Verifier (ensemble)2019-08-14
[]()88.10791.419Retro-Reader on ALBERT (single model)
Retrospective Reader for Machine Reading Comprehension✓ Link88.10791.419Retro-Reader on ALBERT (single model)2020-01-27
[]()88.10790.902ALBERT (single model)
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations✓ Link88.10790.902ALBERT (single model)2019-09-26
[]()88.10790.902{MTL} (single model)
[]()88.10790.902MTL (single model)
[]()88.07391.179albert_with_tricks (single)
[]()88.05091.036ALBert (single-model)
[]()88.05090.645XLNet + SG-Net Verifier (ensemble)
DeBERTa: Decoding-enhanced BERT with Disentangled Attention✓ Link88.090.7DeBERTalarge2020-06-05
[]()87.99490.944ALBERT 1.1 Th (single model)
[]()87.99490.944SkERT-Large (single model)
[]()87.94990.818albert+KD+transfer+twopass (single)
XLNet: Generalized Autoregressive Pretraining for Language Understanding✓ Link87.92690.689XLNet (single model)2019-06-19
[]()87.87090.823ALBERT+RL (single model)
[]()87.84791.265ALBERT+Entailment DA Verifier (single model)
[]()87.84790.532Tuned ALBERT (single model)
[]()87.80290.872albert_xxlarge (single model)
[]()87.70090.588ALBERT 1.1 (single model)
[]()87.42990.163LUKE (single model)
LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention✓ Link87.42990.163LUKE (single model)2020-10-02
SG-Net: Syntax-Guided Machine Reading Comprehension✓ Link87.23890.071XLNet + SG-Net Verifier++ (single model)2019-08-14
[]()87.19389.934UPM (single model)
[]()87.14789.474BERT + DAE + AoA (ensemble)
[]()87.04689.899XLNet + SG-Net Verifier (single model)
[]()86.93390.037RoBERTa+Verify (ensemble)
RoBERTa: A Robustly Optimized BERT Pretraining Approach✓ Link86.82089.795RoBERTa (single model)2019-07-26
[]()86.82089.795RoBERTa (single model)
[]()86.73089.286BERT + ConvLSTM + MTL + Verifier (ensemble)
[]()86.67389.147BERT + N-Gram Masking + Synthetic Self-Training (ensemble)
[]()86.65189.595RoBERTa+Span (ensemble)
[]()86.59489.082Xlnet+Verifier
[]()86.57289.063Xlnet+Verifier(single model)
[]()86.57289.063Xlnet+Verifier (single model)
[]()86.44889.586RoBERTa+Verify (single model)
[]()86.43689.086XLNET-123 (single model)
[]()86.40389.148XLNET-V2-123+ (single model)
[]()86.34689.133XLNet (single model)
[]()86.21188.848SG-Net (ensemble)
SG-Net: Syntax-Guided Machine Reading Comprehension✓ Link86.21188.848SG-Net (ensemble)2019-08-14
[]()86.16688.886BERT++(ensemble)
[]()86.16688.886SemBERT(ensemble)
Semantics-aware BERT for Language Understanding✓ Link86.16688.886SemBERT(ensemble)2019-09-05
Semantics-aware BERT for Language Understanding✓ Link86.16688.886SemBERT (ensemble)2019-09-05
[]()86.09889.634Enhanced Albert+Verifier (ensemble)
[]()85.88488.621BERT + DAE + AoA (single model)
[]()85.87288.989RoBERTa+Span (single model)
[]()85.87288.793RoBERTa-Large (ensemble model)
[]()85.85088.449BNDVnet (ensemble model)
[]()85.83888.921Unnamed submission by guo
[]()85.83888.921BERTSP(single model)
[]()85.83888.921BERTSP (single model)
[]()85.82789.778Enhanced Albert+Verifier3 (ensemble)
[]()85.82788.699SENSEFORTH + XLNet (single model)
[]()85.74888.709SpanBERT (single model)
[]()85.70388.400SemBERT (ensemble model)
SpanBERT: Improving Pre-training by Representing and Predicting Spans✓ Link85.788.7SpanBERT2019-07-24
[]()85.24087.901BertCNN (single model)
[]()85.22987.926SG-Net (single model)
SG-Net: Syntax-Guided Machine Reading Comprehension✓ Link85.22987.926SG-Net (single model)2019-08-14
[]()85.17388.425RoBERTa-Large (single model)
[]()85.15087.715BERT + N-Gram Masking + Synthetic Self-Training (single model)
[]()85.08287.615BERT + MMFT + ADA (ensemble)
[]()85.00387.833BNDVnet (single model)
[]()84.92488.204BERT + ConvLSTM + MTL + Verifier (single model)
[]()84.83487.644Insight-baseline-BERT (single model)
[]()84.82387.489BertSpan (ensemble)
[]()84.80087.864SemBERT (single model)
Semantics-aware BERT for Language Understanding✓ Link84.80087.864SemBERT (single model)2019-09-05
[]()84.72187.117Hanvon_model(single model)
[]()84.72187.117Hanvon_model (single model)
[]()84.64288.000xlnet
[]()84.64288.000xlnet(single model)
[]()84.64288.000xlnet (single model)
[]()84.62087.625BERT++(single model)
[]()84.29286.967BERT + Synthetic Self-Training (ensemble)
[]()84.20286.767BERT + Multiple-CNN (ensemble)(Kyonggi University
[]()84.20286.767BERT + Multiple-CNN (ensemble)
[]()84.12387.013RoberTa Parallel Adapters Singl
[]()84.12387.013RoberTa Parallel Adapters Single
[]()84.12387.013RoberTa+Parallel+Adapters (single model)
[]()83.81986.669SemNet (single model)
[]()83.75186.594Tuned BERT-1seq Large Cased (single model)
[]()83.53686.096BERT finetune baseline (ensemble)
[]()83.52586.222SynNet (single model)
[]()83.46986.043Lunet + Verifier + BERT (ensemble)
[]()83.45786.122PAML+BERT (ensemble model)
[]()83.14285.873Unnamed submission by Senseforth_AI
[]()83.14285.873SENSEFORTH + BERT
[]()83.11985.510Bert-raw (ensemble)
[]()83.05185.737BERT with Something (ensemble)
[]()83.04085.892BERT + MMFT + ADA (single model)
[]()82.99586.035Lunet + Verifier + BERT (single model)
[]()82.97285.810BERT + Synthetic Self-Training (single model)
[]()82.96186.075mgrc_l (single model)
[]()82.88286.002ATB (single model)
[]()82.80385.863Tuned BERT Large Cased (single model)
[]()82.72485.491BERT-Base + QA Pre-training (single model)
[]()82.71385.584BERT + NeurQuRI (ensemble)
[]()82.57785.603PAML+BERT (single model)
[]()82.43185.178Unnamed submission by cooelf
[]()82.37485.310AoA + DA + BERT (ensemble)
[]()82.30685.670BART + Adapters + Lohfink-Rossi-Leaveout (single-model)
[]()82.12684.820BERT finetune baseline (single model)
[]()82.12684.624Candi-Net+BERT (ensemble)
[]()82.02484.854BERT-Base PMI-Masking Additional Data (single model)
[]()81.97984.846BERT_s (single model)
[]()81.73184.862Unnamed submission by zw10
[]()81.57384.535BERT-large+UBFT (single model)
[]()81.17884.251AoA + DA + BERT (single model)
[]()81.11084.386BERT with Something (single model)
[]()80.89683.604BERT-Base PMI-Masking (single model)
[]()80.74983.851BERT + UnAnsQ (single model)
[]()80.71583.827BERT + AL (single model)
[]()80.59183.391BERT + NeurQuRI (single model)
[]()80.45683.509BERTlarge (ensemble)
[]()80.42283.118Bert
[]()80.41183.457Bert-raw (single)
[]()80.38882.908Candi-Net+BERT (single model)
[]()80.37783.262PMI-Masking Additional Data Random Baseline (single model)
[]()80.35483.329Unnamed submission by zw4
[]()80.35483.329{Bert-Span} (single model)
[]()80.34383.243Bert-raw (single model)
[]()80.24183.175PMI-Masking Pure-PMI (single model)
[]()80.20883.149BISAN-CC (single model)
[]()80.14082.962ST_bl
[]()80.11783.189PwP+BERT (single model)
[]()80.03882.796PMI-Masking Random Baseline (single model)
[]()80.00583.208BERT + UDA (single model)
[]()79.99383.039PMI-Masking Additional Data Pure-PMI (single model)
[]()79.97183.266Original BERT Large Cased (single model)
[]()79.97183.184bert (single model)
[]()79.94883.023BERT + Sparse-Transformer
[]()79.77983.099Insight-baseline (single model)
[]()79.77982.912NEXYS_BASE (single model)
[]()79.74583.020BERT uncased (single model)
[]()79.63282.852{bert-finetuning} (single model)
[]()79.18182.259L6Net + BERT (single model)
[]()78.93381.863Fusion Adapters TriviaQA NQ Singl
[]()78.93381.863RoberTa Fusion Adapters Single
[]()78.93381.863RoberTa+Fusion+Adapters (single model)
[]()78.87682.524{Anonymous} (single model)
[]()78.65081.497BERT + WIAN (ensemble)
[]()78.65081.474BERTlarge (single model)
[]()78.59481.445AMBERT (single model)
[]()78.48181.531BISAN (single model)
[]()78.35781.500BERT-Large-Cased (single model)
[]()78.35781.500BERT-Large-Cased
[]()78.05281.174BERT+AC(single model)
[]()78.05281.174BERT+AC (single model)
[]()77.31980.310BERT (single model)
[]()77.26280.258TriviaQA Adapter Single Tune
[]()77.26280.258RoberTa Adapter Single
[]()77.26280.258RoberTa+Adapter (single model)
[]()77.00380.209SLQA+BERT (single model)
[]()76.71079.659AMBERT-H (single model)
[]()76.56379.776AMBERT-S (single model)
[]()76.05579.329synss (single model )
[]()76.05579.329synss (single model)
[]()75.45778.232BERT-Base-L (single model)
[]()75.34478.381mgrc
[]()75.07377.805BERT-Base-V (single model)
[]()74.79177.988MIR-MRC(F-Net) (single model)
[]()74.79177.988MIR-MRC (F-Net)
[]()74.76977.706BERT-Base-DT (single model)
[]()74.74678.227ARSG-BERT (single model)
[]()74.65677.404BERT-Base-V2
[]()74.57777.464BERT-Base-DP (single model)
[]()74.38577.308{BERTcw} (single model)
[]()74.32977.396BERT-Base-Add (single model)
[]()74.27277.052nlnet (single model)
[]()73.74276.858batch2 (single model)
[]()73.50576.424MMIPN
[]()73.30276.284BERT-Base-Baseline (single model)
[]()73.09976.236BERT-Base (single model)
[]()72.88476.217ICL_MODEL(ensemble)
[]()72.88476.217ICL_MODEL (ensemble)
[]()72.67075.507YARCS (ensemble)
[]()72.07275.513BERT-base
[]()72.07275.513BERTBase (single model)
Read + Verify: Machine Reading Comprehension with Unanswerable Questions71.76774.295Reinforced Mnemonic Reader + Answer Verifier (single model)2018-08-17
[]()71.69974.430BERT-Base (single)
[]()71.66675.457BERT+Answer Verifier (single model)
[]()71.46274.434SLQA+ (single model)
U-Net: Machine Reading Comprehension with Unanswerable Questions✓ Link71.41774.869Unet (ensemble)2018-10-12
Stochastic Answer Networks for Machine Reading Comprehension✓ Link71.31673.704SAN (ensemble model)2017-12-10
[]()71.29374.578HYDRA_BERT (single model)
[]()70.76374.449{BERT-base} (single-model)
FusionNet: Fusing via Fully-Aware Attention with Application to Machine Comprehension✓ Link70.30072.484FusionNet++ (ensemble)2017-11-16
[]()69.47672.857Multi-Level Attention Fusion(MLAF) (single model)
[]()69.47672.857Multi-Level Attention Fusion (MLAF)
[]()69.26272.642Unet (single model)
[]()68.76671.662DocQA + NeurQuRI (single model)
Stochastic Answer Networks for Machine Reading Comprehension✓ Link68.65371.439SAN (single model)2017-12-10
[]()68.21370.878KACTEIL-MRC(GFN-Net) (single model)
[]()68.21370.878KACTEIL-MRC (GFN-Net)
[]()68.02171.583BiDAF++ with pair2vec (single model)
[]()67.89770.884VS^3-NET (single model)
[]()66.61070.303EBB-Net (single model)
[]()65.71969.381KakaoNet2 (single model)
[]()65.65168.866BiDAF++ (single model)
[]()65.25669.206abcNet (single model)
Deep contextualized word representations✓ Link63.37266.251BiDAF + Self Attention + ELMo (single model)2018-02-15
[]()63.37266.251BiDAF + Self Attention + ELMo (single model)
[]()63.33867.422BSAE AddText (single model)
[]()63.32766.633eeAttNet (single model)
[]()59.33262.305BiDAF + Self Attention (single model)
[]()59.17462.093BiDAF-No-Answer (single model)
[]()58.50862.045BNA + SoftDrop (single model)
[]()57.70762.341Tree-LSTM + BiDAF + ELMo (single model)
[]()56.54559.546BNA + HardDrop (single model)
[]()56.54559.546Unnamed submission by Simon
[]()49.69549.701Anonymous (single model)
[]()48.88348.883{FOO} (single model)
[]()48.80448.815Bert Large Sentence (Single Model)
[]()44.94547.994XLNet + DAAF + BERTverifier (ensemble)
[]()40.39743.213TSAN
[]()27.21729.597
[]()4.8305.920CHECK SYSTEM (single model)
[]()0.0683.9715cls_squad1_fai
LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention✓ Link90.2LUKE 483M2020-10-02
Ensemble ALBERT on SQuAD 2.0✓ Link90.123Ensemble ALBERT2021-10-19
Pay Attention to MLPs✓ Link78.3gMLP-large2021-05-17
Check out how to contribute new results.