question-answering-on-squad20

Question Answering

Results over time

Click legend items to toggle metrics. Hover points for model names.

Leaderboard

Paper	Code	EM	F1	ModelName	ReleaseDate
		90.939	93.214	IE-Net (ensemble)
		90.871	93.183	FPNet (ensemble)
		90.860	93.100	IE-NetV2 (ensemble)
		90.724	93.011	SA-Net on Albert (ensemble)
		90.679	92.948	SA-Net-V2 (ensemble)
		90.600	92.899	FPNet (ensemble)
Retrospective Reader for Machine Reading Comprehension	✓ Link	90.578	92.978	Retro-Reader (ensemble)	2020-01-27
		90.521	92.824	EntitySpanFocusV2 (ensemble)
		90.487	92.894	TransNets + SFVerifier + SFEnsembler (ensemble)
		90.454	92.748	EntitySpanFocus+AT (ensemble)
		90.442	92.877	ATRLP+PV (ensemble)
		90.420	92.807	LANetV2 (ensemble)
		90.420	92.799	ELECTRA+ALBERT+EntitySpanFocus (ensemble)
		90.386	92.777	ALBERT + DAAF + Verifier
		90.284	92.691	electra+nlayers+adv+ds
		90.194	92.594	MixEnsemble (ensemble)
		90.126	92.535	electra+nlayers (ensemble)
		90.115	92.580	Retro-Reader on ALBERT (ensemble)
Retrospective Reader for Machine Reading Comprehension	✓ Link	90.115	92.580	Retro-Reader on ALBERT (ensemble)	2020-01-27
		90.081	92.457	ANet
		90.059	92.517	Answer Dependent Classify (single model)
		90.002	92.497	electra+nlayers+kdav (ensemble)
		90.002	92.425	ALBERT + DAAF + Verifier (ensemble)
		89.923	92.425	LANet (ensemble)
		89.777	92.312	Electra-nlayers (ensemble)
		89.743	92.180	Albert_Verifier_AA_Net (ensemble)
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations	✓ Link	89.731	92.215	ALBERT (ensemble model)	2019-09-26
		89.562	92.226	Span Extract + Classify (single model)
Retrospective Reader for Machine Reading Comprehension	✓ Link	89.562	92.052	Retro-Reader on ELECTRA (single model)	2020-01-27
		89.551	92.366	ELECTRA+ATRLP+PV (single model)
		89.528	92.059	albert+transform+verify (ensemble)
		89.461	92.134	albert+KD+transfer (ensemble)
		89.449	92.118	ROaD-Electra (single model)
		89.449	92.118	ROaD-Electra
		89.404	91.964	Albert-nlayers (ensemble)
		89.348	91.985	ELECTRA + E-Verifier (ensemble)
		89.325	91.994	ELECTRA + ROBERTA + ALBERT (ensemble)
		89.325	91.939	2task (single model)
		89.235	91.900	Deberta
		89.235	91.739	ALBERT + MTDA + SFVerifier (ensemble model)
		89.224	91.853	ALBert-LSTM (ensemble)
		89.133	91.666	ALBERT + SFVerifier (ensemble model)
		89.021	91.765	ELECTRA+RL+EV (single model)
		88.998	91.635	electra & albert (ensemble)
		88.998	91.635	AE-TEST
		88.874	91.546	ELECTRA+EntitySpanFocus (Single model)
		88.851	91.486	SA-Net on Electra (single model)
		88.761	91.745	ALBERT+Entailment DA (ensemble)
		88.716	91.365	ELECTRA (single model)
		88.637	91.230	Tuned ALBERT (ensemble model)
		88.614	91.303	ELECTRA_ATT (single model)
		88.603	91.299	Deberta+prefix
		88.592	91.286	ALBERT (Single model)
		88.592	90.859	XLNet + DAAF + Verifier (ensemble)
		88.569	91.287	ALBERT + IG + NE (single model)
		88.524	91.256	ALBERT + IG (single model)
		88.434	90.918	aanet_v2.0 (single model)
		88.355	91.019	albert+verifier (single model)
		88.298	91.078	albert+KD+transfer (single)
		88.231	90.713	UPM (ensemble)
		88.197	90.830	ALBERT + SFVerifier (single model)
		88.186	90.939	{alber_m_transfor} (single model)
SG-Net: Syntax-Guided Machine Reading Comprehension	✓ Link	88.174	90.702	XLNet + SG-Net Verifier (ensemble)	2019-08-14
		88.107	91.419	Retro-Reader on ALBERT (single model)
Retrospective Reader for Machine Reading Comprehension	✓ Link	88.107	91.419	Retro-Reader on ALBERT (single model)	2020-01-27
		88.107	90.902	ALBERT (single model)
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations	✓ Link	88.107	90.902	ALBERT (single model)	2019-09-26
		88.107	90.902	{MTL} (single model)
		88.107	90.902	MTL (single model)
		88.073	91.179	albert_with_tricks (single)
		88.050	91.036	ALBert (single-model)
		88.050	90.645	XLNet + SG-Net Verifier (ensemble)
DeBERTa: Decoding-enhanced BERT with Disentangled Attention	✓ Link	88.0	90.7	DeBERTalarge	2020-06-05
		87.994	90.944	ALBERT 1.1 Th (single model)
		87.994	90.944	SkERT-Large (single model)
		87.949	90.818	albert+KD+transfer+twopass (single)
XLNet: Generalized Autoregressive Pretraining for Language Understanding	✓ Link	87.926	90.689	XLNet (single model)	2019-06-19
		87.870	90.823	ALBERT+RL (single model)
		87.847	91.265	ALBERT+Entailment DA Verifier (single model)
		87.847	90.532	Tuned ALBERT (single model)
		87.802	90.872	albert_xxlarge (single model)
		87.700	90.588	ALBERT 1.1 (single model)
		87.429	90.163	LUKE (single model)
LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention	✓ Link	87.429	90.163	LUKE (single model)	2020-10-02
SG-Net: Syntax-Guided Machine Reading Comprehension	✓ Link	87.238	90.071	XLNet + SG-Net Verifier++ (single model)	2019-08-14
		87.193	89.934	UPM (single model)
		87.147	89.474	BERT + DAE + AoA (ensemble)
		87.046	89.899	XLNet + SG-Net Verifier (single model)
		86.933	90.037	RoBERTa+Verify (ensemble)
RoBERTa: A Robustly Optimized BERT Pretraining Approach	✓ Link	86.820	89.795	RoBERTa (single model)	2019-07-26
		86.820	89.795	RoBERTa (single model)
		86.730	89.286	BERT + ConvLSTM + MTL + Verifier (ensemble)
		86.673	89.147	BERT + N-Gram Masking + Synthetic Self-Training (ensemble)
		86.651	89.595	RoBERTa+Span (ensemble)
		86.594	89.082	Xlnet+Verifier
		86.572	89.063	Xlnet+Verifier(single model)
		86.572	89.063	Xlnet+Verifier (single model)
		86.448	89.586	RoBERTa+Verify (single model)
		86.436	89.086	XLNET-123 (single model)
		86.403	89.148	XLNET-V2-123+ (single model)
		86.346	89.133	XLNet (single model)
		86.211	88.848	SG-Net (ensemble)
SG-Net: Syntax-Guided Machine Reading Comprehension	✓ Link	86.211	88.848	SG-Net (ensemble)	2019-08-14
		86.166	88.886	BERT++(ensemble)
		86.166	88.886	SemBERT(ensemble)
Semantics-aware BERT for Language Understanding	✓ Link	86.166	88.886	SemBERT(ensemble)	2019-09-05
Semantics-aware BERT for Language Understanding	✓ Link	86.166	88.886	SemBERT (ensemble)	2019-09-05
		86.098	89.634	Enhanced Albert+Verifier (ensemble)
		85.884	88.621	BERT + DAE + AoA (single model)
		85.872	88.989	RoBERTa+Span (single model)
		85.872	88.793	RoBERTa-Large (ensemble model)
		85.850	88.449	BNDVnet (ensemble model)
		85.838	88.921	Unnamed submission by guo
		85.838	88.921	BERTSP(single model)
		85.838	88.921	BERTSP (single model)
		85.827	89.778	Enhanced Albert+Verifier3 (ensemble)
		85.827	88.699	SENSEFORTH + XLNet (single model)
		85.748	88.709	SpanBERT (single model)
		85.703	88.400	SemBERT (ensemble model)
SpanBERT: Improving Pre-training by Representing and Predicting Spans	✓ Link	85.7	88.7	SpanBERT	2019-07-24
		85.240	87.901	BertCNN (single model)
		85.229	87.926	SG-Net (single model)
SG-Net: Syntax-Guided Machine Reading Comprehension	✓ Link	85.229	87.926	SG-Net (single model)	2019-08-14
		85.173	88.425	RoBERTa-Large (single model)
		85.150	87.715	BERT + N-Gram Masking + Synthetic Self-Training (single model)
		85.082	87.615	BERT + MMFT + ADA (ensemble)
		85.003	87.833	BNDVnet (single model)
		84.924	88.204	BERT + ConvLSTM + MTL + Verifier (single model)
		84.834	87.644	Insight-baseline-BERT (single model)
		84.823	87.489	BertSpan (ensemble)
		84.800	87.864	SemBERT (single model)
Semantics-aware BERT for Language Understanding	✓ Link	84.800	87.864	SemBERT (single model)	2019-09-05
		84.721	87.117	Hanvon_model(single model)
		84.721	87.117	Hanvon_model (single model)
		84.642	88.000	xlnet
		84.642	88.000	xlnet(single model)
		84.642	88.000	xlnet (single model)
		84.620	87.625	BERT++(single model)
		84.292	86.967	BERT + Synthetic Self-Training (ensemble)
		84.202	86.767	BERT + Multiple-CNN (ensemble)(Kyonggi University
		84.202	86.767	BERT + Multiple-CNN (ensemble)
		84.123	87.013	RoberTa Parallel Adapters Singl
		84.123	87.013	RoberTa Parallel Adapters Single
		84.123	87.013	RoberTa+Parallel+Adapters (single model)
		83.819	86.669	SemNet (single model)
		83.751	86.594	Tuned BERT-1seq Large Cased (single model)
		83.536	86.096	BERT finetune baseline (ensemble)
		83.525	86.222	SynNet (single model)
		83.469	86.043	Lunet + Verifier + BERT (ensemble)
		83.457	86.122	PAML+BERT (ensemble model)
		83.142	85.873	Unnamed submission by Senseforth_AI
		83.142	85.873	SENSEFORTH + BERT
		83.119	85.510	Bert-raw (ensemble)
		83.051	85.737	BERT with Something (ensemble)
		83.040	85.892	BERT + MMFT + ADA (single model)
		82.995	86.035	Lunet + Verifier + BERT (single model)
		82.972	85.810	BERT + Synthetic Self-Training (single model)
		82.961	86.075	mgrc_l (single model)
		82.882	86.002	ATB (single model)
		82.803	85.863	Tuned BERT Large Cased (single model)
		82.724	85.491	BERT-Base + QA Pre-training (single model)
		82.713	85.584	BERT + NeurQuRI (ensemble)
		82.577	85.603	PAML+BERT (single model)
		82.431	85.178	Unnamed submission by cooelf
		82.374	85.310	AoA + DA + BERT (ensemble)
		82.306	85.670	BART + Adapters + Lohfink-Rossi-Leaveout (single-model)
		82.126	84.820	BERT finetune baseline (single model)
		82.126	84.624	Candi-Net+BERT (ensemble)
		82.024	84.854	BERT-Base PMI-Masking Additional Data (single model)
		81.979	84.846	BERT_s (single model)
		81.731	84.862	Unnamed submission by zw10
		81.573	84.535	BERT-large+UBFT (single model)
		81.178	84.251	AoA + DA + BERT (single model)
		81.110	84.386	BERT with Something (single model)
		80.896	83.604	BERT-Base PMI-Masking (single model)
		80.749	83.851	BERT + UnAnsQ (single model)
		80.715	83.827	BERT + AL (single model)
		80.591	83.391	BERT + NeurQuRI (single model)
		80.456	83.509	BERTlarge (ensemble)
		80.422	83.118	Bert
		80.411	83.457	Bert-raw (single)
		80.388	82.908	Candi-Net+BERT (single model)
		80.377	83.262	PMI-Masking Additional Data Random Baseline (single model)
		80.354	83.329	Unnamed submission by zw4
		80.354	83.329	{Bert-Span} (single model)
		80.343	83.243	Bert-raw (single model)
		80.241	83.175	PMI-Masking Pure-PMI (single model)
		80.208	83.149	BISAN-CC (single model)
		80.140	82.962	ST_bl
		80.117	83.189	PwP+BERT (single model)
		80.038	82.796	PMI-Masking Random Baseline (single model)
		80.005	83.208	BERT + UDA (single model)
		79.993	83.039	PMI-Masking Additional Data Pure-PMI (single model)
		79.971	83.266	Original BERT Large Cased (single model)
		79.971	83.184	bert (single model)
		79.948	83.023	BERT + Sparse-Transformer
		79.779	83.099	Insight-baseline (single model)
		79.779	82.912	NEXYS_BASE (single model)
		79.745	83.020	BERT uncased (single model)
		79.632	82.852	{bert-finetuning} (single model)
		79.181	82.259	L6Net + BERT (single model)
		78.933	81.863	Fusion Adapters TriviaQA NQ Singl
		78.933	81.863	RoberTa Fusion Adapters Single
		78.933	81.863	RoberTa+Fusion+Adapters (single model)
		78.876	82.524	{Anonymous} (single model)
		78.650	81.497	BERT + WIAN (ensemble)
		78.650	81.474	BERTlarge (single model)
		78.594	81.445	AMBERT (single model)
		78.481	81.531	BISAN (single model)
		78.357	81.500	BERT-Large-Cased (single model)
		78.357	81.500	BERT-Large-Cased
		78.052	81.174	BERT+AC(single model)
		78.052	81.174	BERT+AC (single model)
		77.319	80.310	BERT (single model)
		77.262	80.258	TriviaQA Adapter Single Tune
		77.262	80.258	RoberTa Adapter Single
		77.262	80.258	RoberTa+Adapter (single model)
		77.003	80.209	SLQA+BERT (single model)
		76.710	79.659	AMBERT-H (single model)
		76.563	79.776	AMBERT-S (single model)
		76.055	79.329	synss (single model )
		76.055	79.329	synss (single model)
		75.457	78.232	BERT-Base-L (single model)
		75.344	78.381	mgrc
		75.073	77.805	BERT-Base-V (single model)
		74.791	77.988	MIR-MRC(F-Net) (single model)
		74.791	77.988	MIR-MRC (F-Net)
		74.769	77.706	BERT-Base-DT (single model)
		74.746	78.227	ARSG-BERT (single model)
		74.656	77.404	BERT-Base-V2
		74.577	77.464	BERT-Base-DP (single model)
		74.385	77.308	{BERTcw} (single model)
		74.329	77.396	BERT-Base-Add (single model)
		74.272	77.052	nlnet (single model)
		73.742	76.858	batch2 (single model)
		73.505	76.424	MMIPN
		73.302	76.284	BERT-Base-Baseline (single model)
		73.099	76.236	BERT-Base (single model)
		72.884	76.217	ICL_MODEL(ensemble)
		72.884	76.217	ICL_MODEL (ensemble)
		72.670	75.507	YARCS (ensemble)
		72.072	75.513	BERT-base
		72.072	75.513	BERTBase (single model)
Read + Verify: Machine Reading Comprehension with Unanswerable Questions		71.767	74.295	Reinforced Mnemonic Reader + Answer Verifier (single model)	2018-08-17
		71.699	74.430	BERT-Base (single)
		71.666	75.457	BERT+Answer Verifier (single model)
		71.462	74.434	SLQA+ (single model)
U-Net: Machine Reading Comprehension with Unanswerable Questions	✓ Link	71.417	74.869	Unet (ensemble)	2018-10-12
Stochastic Answer Networks for Machine Reading Comprehension	✓ Link	71.316	73.704	SAN (ensemble model)	2017-12-10
		71.293	74.578	HYDRA_BERT (single model)
		70.763	74.449	{BERT-base} (single-model)
FusionNet: Fusing via Fully-Aware Attention with Application to Machine Comprehension	✓ Link	70.300	72.484	FusionNet++ (ensemble)	2017-11-16
		69.476	72.857	Multi-Level Attention Fusion(MLAF) (single model)
		69.476	72.857	Multi-Level Attention Fusion (MLAF)
		69.262	72.642	Unet (single model)
		68.766	71.662	DocQA + NeurQuRI (single model)
Stochastic Answer Networks for Machine Reading Comprehension	✓ Link	68.653	71.439	SAN (single model)	2017-12-10
		68.213	70.878	KACTEIL-MRC(GFN-Net) (single model)
		68.213	70.878	KACTEIL-MRC (GFN-Net)
		68.021	71.583	BiDAF++ with pair2vec (single model)
		67.897	70.884	VS^3-NET (single model)
		66.610	70.303	EBB-Net (single model)
		65.719	69.381	KakaoNet2 (single model)
		65.651	68.866	BiDAF++ (single model)
		65.256	69.206	abcNet (single model)
Deep contextualized word representations	✓ Link	63.372	66.251	BiDAF + Self Attention + ELMo (single model)	2018-02-15
		63.372	66.251	BiDAF + Self Attention + ELMo (single model)
		63.338	67.422	BSAE AddText (single model)
		63.327	66.633	eeAttNet (single model)
		59.332	62.305	BiDAF + Self Attention (single model)
		59.174	62.093	BiDAF-No-Answer (single model)
		58.508	62.045	BNA + SoftDrop (single model)
		57.707	62.341	Tree-LSTM + BiDAF + ELMo (single model)
		56.545	59.546	BNA + HardDrop (single model)
		56.545	59.546	Unnamed submission by Simon
		49.695	49.701	Anonymous (single model)
		48.883	48.883	{FOO} (single model)
		48.804	48.815	Bert Large Sentence (Single Model)
		44.945	47.994	XLNet + DAAF + BERTverifier (ensemble)
		40.397	43.213	TSAN
		27.217	29.597
		4.830	5.920	CHECK SYSTEM (single model)
		0.068	3.971	5cls_squad1_fai
LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention	✓ Link		90.2	LUKE 483M	2020-10-02
Ensemble ALBERT on SQuAD 2.0	✓ Link		90.123	Ensemble ALBERT	2021-10-19
Pay Attention to MLPs	✓ Link		78.3	gMLP-large	2021-05-17

OpenCodePapers

question-answering-on-squad20