CyberSecEval's insecure-code tests partly measure refusal to follow bad instructions and susceptibility to leading comments, not natural insecure code generation.
Our goal was to demonstrate statistical variability between our work and Meta’s previous work, we demonstrate this by highlighting percentage point residuals in Section 2
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.AI 1years
2024 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Rethinking CyberSecEval: An LLM-Aided Approach to Evaluation Critique
CyberSecEval's insecure-code tests partly measure refusal to follow bad instructions and susceptibility to leading comments, not natural insecure code generation.