OpenCodePapers

code-completion-on-safim

Code Completion

Results over time

Click legend items to toggle metrics. Hover points for model names.

Leaderboard

Paper	Code	Average	Algorithmic	Control	API	ModelName	ReleaseDate
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks	✓ Link	69.01	60.78	71.10	75.16	deepseek-coder-33b-base	2024-03-07
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks	✓ Link	63.40	54.74	65.79	69.68	deepseek-coder-6.7b-base	2024-03-07
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks	✓ Link	55.54	44.11	54.46	68.06	starcoderbase	2024-03-07
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks	✓ Link	53.28	42.11	55.15	62.58	gpt-4-1106-preview	2024-03-07
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks	✓ Link	52.78	41.41	57.25	59.68	CodeLlama-13b-hf	2024-03-07
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks	✓ Link	52.63	41.20	54.10	62.58	deepseek-coder-1.3b-base	2024-03-07
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks	✓ Link	49.66	38.55	53.98	56.45	CodeLlama-34b-hf	2024-03-07
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks	✓ Link	45.00	34.68	53.56	46.77	CodeLlama-7b-hf	2024-03-07
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks	✓ Link	40.86	31.24	37.48	53.87	gpt-3.5-turbo-0301	2024-03-07
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks	✓ Link	33.79	25.16	28.16	48.06	incoder-6B	2024-03-07
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks	✓ Link	30.99	25.94	35.74	31.29	codegen-16B-multi	2024-03-07
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks	✓ Link	29.55	23.49	32.89	32.26	codegen-2B-multi	2024-03-07
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks	✓ Link	29.27	21.06	22.89	43.87	incoder-1B	2024-03-07
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks	✓ Link	28.71	23.60	34.80	27.74	codegen-6B-multi	2024-03-07
Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks	✓ Link	22.94	16.30	26.06	26.45	codegen-350M-multi	2024-03-07