BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models

Benchmark Model Rank Results
biomedical-information-retrieval-on-bioasq-1BM25+CE#4nDCG@10: 0.523
biomedical-information-retrieval-on-bioasq-1BM25#5nDCG@10: 0.514
biomedical-information-retrieval-on-nfcorpus-1BM25+CE#4nDCG@10: 0.350
biomedical-information-retrieval-on-nfcorpus-1ColBERT#7nDCG@10: 0.305
biomedical-information-retrieval-on-trec-1BM25+CE#5nDCG@10: 0.757
biomedical-information-retrieval-on-trec-1ColBERT#6nDCG@10: 0.677
fact-checking-on-climate-fever-beirBM25+CE#3nDCG@10: 0.253
fact-checking-on-fever-beirBM25+CE#2nDCG@10: 0.819
fact-checking-on-scifact-beirBM25+CE#3nDCG@10: 0.688
fact-checking-on-scifact-beirColBERT#5nDCG@10: 0.671
passage-retrieval-on-msmarco-beirBM25+CE#1nDCG@10: 0.413
passage-retrieval-on-msmarco-beirTAS-b#2nDCG@10: 0.408
passage-retrieval-on-msmarco-beirColBERT#3nDCG@10: 0.401
passage-retrieval-on-msmarco-beirANCE#5nDCG@10: 0.388
passage-retrieval-on-msmarco-beirSPARTA#6nDCG@10: 0.351
passage-retrieval-on-msmarco-beirdocT5query#7nDCG@10: 0.338
passage-retrieval-on-msmarco-beirDeepCT#8nDCG@10: 0.296
passage-retrieval-on-msmarco-beirBM25#11nDCG@10: 0.228
passage-retrieval-on-msmarco-beirDPR#12nDCG@10: 0.177
question-answering-on-fiqa-2018-beirBM25+CE#4nDCG@10: 0.347
question-answering-on-hotpotqa-beirBM25+CE#2nDCG@10: 0.707
question-answering-on-nq-beirBM25+CE#3nDCG@10: 0.533
question-answering-on-nq-beirColBERT#4nDCG@10: 0.524