Coder Reviewer Reranking for Code Generation

Benchmark Model Rank Results
code-generation-on-mbppcode-davinci-002 175B + Reviewer#66Accuracy: 66.9
code-generation-on-mbppcode-davinci-002 175B + Coder-Reviewer#67Accuracy: 66.4
code-generation-on-mbppcode-davinci-002 175B + MBR-Exec#77Accuracy: 63
code-generation-on-mbppcode-cushman-001 12B + MBR-Exec#113Accuracy: 48.3
code-generation-on-mbppCodeGen 16B + MBR-Exec#116Accuracy: 47.3
code-generation-on-mbppCodeGen 16B + Coder-Reviewer#121Accuracy: 46.2
code-generation-on-mbppCodeGen 16B + Reviewer#128Accuracy: 44.1
code-generation-on-mbppInCoder 6.7B + MBR-Exec#153Accuracy: 26.7
code-generation-on-mbppInCoder 6.7B + Coder-Reviewer#155Accuracy: 26.1
code-generation-on-mbppInCoder 6.7B + Reviewer#156Accuracy: 24.4