Execution Guided Line-by-Line Code Generation

Benchmark Model Rank Results
code-generation-on-codecontestsEG-CFG (DeepSeek-V3-0324)#1Test Set pass@1: 58.18
code-generation-on-mbppEG-CFG (DeepSeek-V3-0324)#1Accuracy: 96.6
code-generation-on-mbppEG-CFG (DeepSeek Coder 1.3b Instruct)#23Accuracy: 83.2
code-generation-on-humanevalEG-CFG (DeepSeek-V3-0324)#7Pass@1: 96.95