Execution Guided Line-by-Line Code Generation

Benchmark Model Rank Results
code-generation-on-codecontestsEG-CFG (DeepSeek-V3-0324)#1Test Set pass@1: 58.18
code-generation-on-humanevalEG-CFG (DeepSeek-V3-0324)#5Pass@1: 96.95
code-generation-on-mbppEG-CFG (DeepSeek-V3-0324)#1Accuracy: 96.6
code-generation-on-mbppEG-CFG (DeepSeek Coder 1.3b Instruct)#10Accuracy: 83.2