RepoZero is a new benchmark for LLM agents to generate complete repositories from API specs using automated execution-based verification, with current models reaching only 30-55% pass rates.
Code must compile with ‘rustc‘ or as a Cargo project
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.SE 1years
2026 1verdicts
UNVERDICTED 1representative citing papers
citing papers explorer
-
RepoZero: Can LLMs Generate a Code Repository from Scratch?
RepoZero is a new benchmark for LLM agents to generate complete repositories from API specs using automated execution-based verification, with current models reaching only 30-55% pass rates.