Turbulence: Systematically and Automatically Testing Instruction-Tuned Large Language Models for Code

Benchmark Model Rank Results
code-generation-on-turbulenceGPT-4#1CorrSc: 0.848
code-generation-on-turbulenceGPT-3.5-Turbo#2CorrSc: 0.617
code-generation-on-turbulenceCodeLlama:13B-4bit-quantised#3CorrSc: 0.327
code-generation-on-turbulenceCodeLlama:7B-4bit-quantised#4CorrSc: 0.289
code-generation-on-turbulenceCommand#5CorrSc: 0.063