A new 70-challenge benchmark measures whether language models can autonomously discover and exploit AI/ML security vulnerabilities, finding Claude-3.7-Sonnet solves 61% of challenges while the best open-source model solves 10%.
Llm4decompile: Decompilingbinarycodewithlargelanguagemodels,
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
citation-role summary
background 1
citation-polarity summary
fields
cs.CR 1years
2025 1verdicts
CONDITIONAL 1roles
background 1polarities
background 1representative citing papers
citing papers explorer
-
AIRTBench: Measuring Autonomous AI Red Teaming Capabilities in Language Models
A new 70-challenge benchmark measures whether language models can autonomously discover and exploit AI/ML security vulnerabilities, finding Claude-3.7-Sonnet solves 61% of challenges while the best open-source model solves 10%.