| code-generation-on-mbpp | Code Llama - Python 70B (3-shot) | #72 | Accuracy: 65.5 |
| code-generation-on-mbpp | Code Llama 70B (3-shot) | #79 | Accuracy: 62.4 |
| code-generation-on-mbpp | Code Llama - Instruct 70B (3-shot) | #81 | Accuracy: 62.2 |
| code-generation-on-mbpp | Unnatural Code Llama 34B (3-shot) | #85 | Accuracy: 61.2 |
| code-generation-on-mbpp | Code Llama - Instruct 34B (3-shot) | #93 | Accuracy: 57 |
| code-generation-on-mbpp | Code Llama - Python 34B (3-shot) | #95 | Accuracy: 56.2 |
| code-generation-on-mbpp | Code Llama 34B (3-shot) | #99 | Accuracy: 55 |
| code-generation-on-mbpp | GPT-3.5 Turbo | #103 | Accuracy: 52.2 |
| code-generation-on-mbpp | Code Llama - Instruct 13B (3-shot) | #107 | Accuracy: 49.4 |
| code-generation-on-mbpp | Code Llama - Python 13B (3-shot) | #110 | Accuracy: 49 |
| code-generation-on-mbpp | Code Llama - Python 7B (3-shot) | #114 | Accuracy: 47.6 |
| code-generation-on-mbpp | Code Llama 13B (3-shot) | #119 | Accuracy: 47 |
| code-generation-on-mbpp | Code Llama - Instruct 7B (3-shot) | #126 | Accuracy: 44.4 |
| code-generation-on-mbpp | Code Llama 7B (3-shot) | #133 | Accuracy: 41.4 |
| code-generation-on-humaneval | Code Llama - Instruct 70B | #89 | Pass@1: 67.8 |
| arithmetic-reasoning-on-gsm8k | Code Llama - Python 34B | #254 | Accuracy: 34.420 |