OpenCodePapers

code-generation-on-res-q

Code Generation
Dataset Link
Results over time
Click legend items to toggle metrics. Hover points for model names.
Leaderboard
Click a sortable column header to sort.
PaperCodepass@1ModelNameReleaseDate
RES-Q: Evaluating Code-Editing Large Language Model Systems at the Repository Scale✓ Link58.0QurrentOS-coder + Claude 3.5 Sonnet2024-06-24
RES-Q: Evaluating Code-Editing Large Language Model Systems at the Repository Scale✓ Link46.0QurrentOS-coder + GPT-4o2024-06-24
RES-Q: Evaluating Code-Editing Large Language Model Systems at the Repository Scale✓ Link37.0QurrentOS-coder + GPT-4 Turbo2024-06-24
RES-Q: Evaluating Code-Editing Large Language Model Systems at the Repository Scale✓ Link36.0QurrentOS-coder + Claude 3 Opus2024-06-24
RES-Q: Evaluating Code-Editing Large Language Model Systems at the Repository Scale✓ Link30.0QurrentOS-coder + GPT-42024-06-24
RES-Q: Evaluating Code-Editing Large Language Model Systems at the Repository Scale✓ Link30.0QurrentOS-coder + Gemini 1.5 Pro2024-06-24
RES-Q: Evaluating Code-Editing Large Language Model Systems at the Repository Scale✓ Link29.0QurrentOS-coder + DeepSeek-Coder-V22024-06-24
RES-Q: Evaluating Code-Editing Large Language Model Systems at the Repository Scale✓ Link20.0QurrentOS-coder + Llama 3 70b2024-06-24
RES-Q: Evaluating Code-Editing Large Language Model Systems at the Repository Scale✓ Link18.0QurrentOS-coder + Qwen-72B-Instruct2024-06-24
Check out how to contribute new results.