ChatQA: Surpassing GPT-4 on Conversational QA and RAG

Benchmark Model Rank Results
question-answering-on-natural-questionsChatQA-1.5-llama3-70b (Zero-Shot, KILT)EM: 47.0
question-answering-on-natural-questionsChatQA-1.5-llama3-8b (Zero-Shot, KILT)EM: 42.7
question-answering-on-triviaqaChatQA-1.5-llama3-70b (Zero-Shot, DPR)EM: 69.0
question-answering-on-triviaqaChatQA-1.5-llama3-70b (Zero-Shot, KILT)EM: 85.6
question-answering-on-triviaqaChatQA-1.5-llama3-8B (Zero-Shot, KILT)EM: 81.0