| Benchmark | Model | Rank | Results |
|---|---|---|---|
| code-generation-on-conala-ext | External Knowledge With API + Reranking | #3 | BLEU: 20.54 |
| code-generation-on-conala-ext | External Knowledge With API | #4 | BLEU: 20.37 |
| code-generation-on-conala | External Knowledge With API + Reranking | #6 | BLEU: 32.26 |
| code-generation-on-conala | External Knowledge With API | #7 | BLEU: 30.69 |