A Case Study of Web App Coding with OpenAI Reasoning Models

Benchmark Model Rank Results
code-generation-on-webapp1k-duo-reactclaude-3-5-sonnet#1pass@1: 0.679
code-generation-on-webapp1k-duo-reacto1-mini#2pass@1: 0.667
code-generation-on-webapp1k-duo-reacto1-preview#3pass@1: 0.652
code-generation-on-webapp1k-duo-reactgpt-4o-2024-08-06#4pass@1: 0.531
code-generation-on-webapp1k-duo-reactdeepseek-v2.5#5pass@1: 0.49
code-generation-on-webapp1k-duo-reactmistral-large-2#6pass@1: 0.449
code-generation-on-webapp1k-reacto1-preview#1pass@1: 0.952
code-generation-on-webapp1k-reacto1-mini#2pass@1: 0.939
code-generation-on-webapp1k-reactdeepseek-v2.5#5pass@1: 0.834