ChatQA: Surpassing GPT-4 on Conversational QA and RAG

Benchmark Model Rank Results
question-answering-on-natural-questionsChatQA-1.5-llama3-70b (Zero-Shot, KILT)–EM: 47.0
question-answering-on-natural-questionsChatQA-1.5-llama3-8b (Zero-Shot, KILT)–EM: 42.7
question-answering-on-triviaqaChatQA-1.5-llama3-70b (Zero-Shot, DPR)–EM: 69.0
question-answering-on-triviaqaChatQA-1.5-llama3-70b (Zero-Shot, KILT)–EM: 85.6
question-answering-on-triviaqaChatQA-1.5-llama3-8B (Zero-Shot, KILT)–EM: 81.0