Can large language models reason about medical questions?

Benchmark Model Rank Results
question-answering-on-pubmedqaCodex 5-shot CoT#8Accuracy: 78.2
multiple-choice-question-answering-mcqa-on-medmcqaCodex 5-shot CoT#9Test Set (Acc-%): 0.627Dev Set (Acc-%): 0.597
question-answering-on-medqa-usmleCodex 5-shot CoT#11Accuracy: 60.2