Introducing OpenAI o1

Benchmark Model Rank Results
agents-on-taubench-airlineOpenAI o1–Accuracy: 50.0
agents-on-taubench-retailOpenAI o1–Accuracy: 70.8
chart-question-answering-on-charxiv-rqOpenAI o1–Accuracy: 55.1
code-generation-on-aider-polyglotOpenAI o1 (High)–Accuracy: 64.4
code-generation-on-swe-bench-verifiedOpenAI o1 (High)–Accuracy: 48.9
instruction-following-on-multichallengeOpenAI o1–Accuracy: 44.93
mathematical-reasoning-on-aime-2025OpenAI o1–Accuracy: 79.2
multimodal-reasoning-on-mathvistaOpenAI o1–Accuracy: 73.9
multimodal-reasoning-on-mmmuOpenAI o1–Accuracy: 78.2
multiple-choice-question-answering-mcqa-on-gpqa-diamondOpenAI o1–Accuracy: 77.3