Orchid benchmark shows requirement ambiguity degrades LLM code generation performance across all models, with advanced models hit hardest, and LLMs rarely detect or resolve the ambiguity themselves.
Humanevalcomm: Benchmarking the communication competence of code generation for llms and LLM agent
3 Pith papers cite this work. Polarity classification is still indexing.
3
Pith papers citing it
fields
cs.SE 3years
2026 3representative citing papers
AI-generated C++ programs trigger confirmed runtime (sanitizer) violations at roughly twice the odds of human contest solutions, a gap static analysis does not reveal.
A-ProS uses a hybrid multi-model feedback framework with stateful refinement to improve success rates on competitive programming problems, achieving over 2x gains compared to baseline agent loops.
citing papers explorer
-
Assessing the Impact of Requirement Ambiguity on LLM-based Function-Level Code Generation
Orchid benchmark shows requirement ambiguity degrades LLM code generation performance across all models, with advanced models hit hardest, and LLMs rarely detect or resolve the ambiguity themselves.
-
The Illusion of Safety: Multi-Tier Verification of AI vs. Human C++ Code
AI-generated C++ programs trigger confirmed runtime (sanitizer) violations at roughly twice the odds of human contest solutions, a gap static analysis does not reveal.
-
A-ProS: Towards Reliable Autonomous Programming Through Multi-Model Feedback
A-ProS uses a hybrid multi-model feedback framework with stateful refinement to improve success rates on competitive programming problems, achieving over 2x gains compared to baseline agent loops.