| code-generation-on-conala | RoBERTaMarian | #1 | BLEU: 35.74Exact Match Accuracy: 13.8 |
| code-generation-on-conala | BERTMarian | #5 | BLEU: 32.46Exact Match Accuracy: 12.40 |
| code-generation-on-conala | ELECTRAMarian | #9 | BLEU: 30.18Exact Match Accuracy: 10.0 |
| code-generation-on-conala | LUKEMarian | #10 | BLEU: 29.83Exact Match Accuracy: 7.6 |
| code-generation-on-django | LUKEMarian | #4 | Accuracy: 78.50BLEU Score: 89.34 |
| code-generation-on-django | RoBERTaMarian | #5 | Accuracy: 77.95BLEU Score: 88.91 |
| code-generation-on-django | BERTMarian | #6 | Accuracy: 76.68BLEU Score: 56.55 |
| code-generation-on-django | ELECTRAMarian | #8 | Accuracy: 65.32BLEU Score: 53.02 |