Web-Bench: A LLM Code Benchmark Based on Web Standards and Frameworks

Open paper

Benchmark	Model	Rank	Results
on-web-bench	claude-3-7-sonnet-20250219-thinking	#1	Pass@1: 25.11%Pass@2: 35.33%
on-web-bench	claude-3-5-sonnet-20241022	#2	Pass@1: 23.04%Pass@2: 32.39%
on-web-bench	claude-3-7-sonnet-20250219	#3	Pass@1: 22.50%Pass@2: 30.98%
on-web-bench	claude-3-5-sonnet-20240620	#4	Pass@1: 21.96%Pass@2: 30.33%
on-web-bench	gpt-4.1	#5	Pass@1: 21.09%Pass@2: 25.11%
on-web-bench	gpt-4.1-mini	#6	Pass@1: 20.76%Pass@2: 23.70%
on-web-bench	doubao-pro-1.5-thinking	#7	Pass@1: 20.11%Pass@2: 30.22%
on-web-bench	gpt-4o	#8	Pass@1: 17.17%Pass@2: 23.80%
on-web-bench	deepseek-v3-0324	#9	Pass@1: 17.07%Pass@2: 23.59%
on-web-bench	deepseek-coder-v2	#10	Pass@1: 16.74%Pass@2: 23.15%
on-web-bench	doubao-pro-1.5-32k	#11	Pass@1: 16.63%Pass@2: 22.93%
on-web-bench	llama-4 Maverick	#12	Pass@1: 15.98%Pass@2: 20.87%
on-web-bench	qwen-max-2025-01-25	#13	Pass@1: 15.87%Pass@2: 19.02%
on-web-bench	gemini-2.5-pro	#14	Pass@1: 15.67%Pass@2: 24.02%
on-web-bench	claude-3-5-haiku-20241022	#15	Pass@1: 15.43%Pass@2: 21.74%
on-web-bench	gemini-2.0-flash	#16	Pass@1: 15.33%Pass@2: 20.87%
on-web-bench	gemini-2.0-flash-thinking	#17	Pass@1: 14.89%Pass@2: 19.24%
on-web-bench	gemini-pro-1.5	#18	Pass@1: 14.78%Pass@2: 20.87%
on-web-bench	deepseek-r1	#19	Pass@1: 14.46%Pass@2: 26.20%
on-web-bench	step-fun-2-16k	#20	Pass@1: 13.70%Pass@2: 15.87%
on-web-bench	o4-mini	#21	Pass@1: 13.26%Pass@2: 22.93%
on-web-bench	mistral-large-2411	#22	Pass@1: 13.04%Pass@2: 18.70%
on-web-bench	gemini-flash-1.5	#23	Pass@1: 12.83%Pass@2: 17.07%
on-web-bench	qwen-plus-2025-01-25	#24	Pass@1: 11.85%Pass@2: 15.11%
on-web-bench	grok-2-1212	#25	Pass@1: 11.30%Pass@2: 17.17%
on-web-bench	qwen-2.5-72b-instruct	#26	Pass@1: 10.54%Pass@2: 13.70%
on-web-bench	o1	#27	Pass@1: 10.43%Pass@2: 12.39%
on-web-bench	gemma-3-27b	#28	Pass@1: 9.89%Pass@2: 11.85%
on-web-bench	o3-mini	#29	Pass@1: 9.13%Pass@2: 14.24%
on-web-bench	gpt-4o-mini	#30	Pass@1: 8.48%Pass@2: 13.04%
on-web-bench	sense-chat-5	#31	Pass@1: 8.48%Pass@2: 12.72%
on-web-bench	minimax-text	#32	Pass@1: 8.48%Pass@2: 10.76%
on-web-bench	360-gpt2-o1	#33	Pass@1: 8.26%Pass@2: 14.46%
on-web-bench	GLM-4-0414	#34	Pass@1: 7.50%Pass@2: 9.02%
on-web-bench	gpt-4.1-nano	#35	Pass@1: 7.07%Pass@2: 12.28%
on-web-bench	llama-3.3	#36	Pass@1: 6.63%Pass@2: 9.57%
on-web-bench	moonshot-kimi-latest	#37	Pass@1: 5.22%Pass@2: 11.85%
on-web-bench	llama-4 Scout	#38	Pass@1: 5.00%Pass@2: 7.72%
on-web-bench	doubao-pro-1.5-32k-lite	#39	Pass@1: 3.48%Pass@2: 5.98%
on-web-bench	qwen-turbo-2024-11-01	#40	Pass@1: 2.61%Pass@2: 5.11%