REVIEW 1 major objections 5 minor 31 references
AI agents make cybersecurity certificates expire the week they're signed
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · glm-5.2
2026-07-09 19:55 UTC pith:O55JTR7T
load-bearing objection The legal-regulatory analysis is the real contribution; the robot case studies are underspecified and self-sourced. the 1 major comments →
Certifying Ghosts: How Cybersecurity AI Agents Break the EU Cyber Resilience Act
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper identifies a structural distinction between two ways cybersecurity AI agents stress a process-oriented cybersecurity regime. Quantitative stress — more vulnerabilities found than humans can triage — merely bends the regime, because risk-based prioritisation was designed for scarce attention and can be reinterpreted as the compliance object. But environmental stress — where the adversary’s capability changes so that a certified-secure product becomes exploitable without any modification to the product itself — breaks the regime, because no re-run of the existing process restores validity. The certificate attests to a property of a world that no longer exists. The paper maps each CRA
What carries the argument
The bends-versus-breaks decomposition: a process-oriented regime can absorb quantitative stress (more candidates to triage) by reinterpreting its obligations, but cannot survive the withdrawal of a premise (stable posture, discrete exploitation, winnable remediation race) because the invalidating change originates in the environment, not the product, and fires no re-assessment trigger.
Load-bearing premise
The paper assumes that defensive results from controlled evaluations against a specific offensive agent generalise to real-world deployment against adversaries the authors do not control. If the defensive advantage is specific to the particular attacker used in testing, the central remedy — that defenders can track offenders because they wield the same capability — is not yet established.
What would settle it
If point-in-time conformity assessments are treated as adequate through the first CRA review despite documented agent-driven mass exploitation of conformant products, the breaks thesis is falsified.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper argues that Cybersecurity AI (CAI) agents—AI systems used as offensive or defensive security instruments against other products—invalidate four tacit premises (P1–P4) underlying the EU Cyber Resilience Act's (CRA) process-oriented regulatory model. The central decomposition distinguishes 'bends' (P1 strained by vulnerability abundance, absorbable via re-interpreted guidance) from 'breaks' (P2–P4 withdrawn by lifecycle-tempo collapse, requiring new regulatory constructs). The paper maps each CRA mechanism to a verdict (Table 1), proposes an agent-native remedy of continuous conformity, and validates it on two CRA-scope robots (Unitree G1 humanoid, Hookii lawn mower) using a defensive agent (RIS). Legal claims are anchored to specific articles of Regulation (EU) 2024/2847; capability claims cite dated primary sources. The time-to-exploit analysis (Appendices A–C) is presented with commendable caveats about selection bias, dating rules, and provenance overlap.
Significance. The premise-level bends/breaks decomposition, tied provision-by-provision to enacted CRA text, is a genuinely novel analytical contribution at the intersection of cybersecurity law and AI capability assessment. The paper ships six falsifiable predictions (§3.4), an honest critical reading of its own empirical data (Appendices A–C), and a proof-of-concept remedy validated on physical robots. The observation that the CRA's point-in-time conformity gate has no sensor for adversary-capability evolution is well-formed and policy-relevant. The cross-check against ENISA's EUVD (Appendix B–C) adds robustness to the tempo-collapse claim. These strengths make the paper a serious contribution to the emerging literature on AI-era cybersecurity regulation.
major comments (1)
- §3.2, Table 1 (P3 row): The P3 'break' conflates automated probing with successful exploitation. The paper argues that 'when offensive tooling is automated and continuous, exploitation becomes background rather than event,' making the Article 14 'actively exploited' trigger (Art. 3(42)) uninformative. But Art. 3(42) requires 'reliable evidence that a malicious actor has exploited' a vulnerability—i.e., successful exploitation of a specific flaw in a specific product, not ambient scanning. Automated probing of exposed products does not constitute exploitation evidence under this definition. The cited sources ([11] on automated exploit generation capability, [15] on synthetic APTs collapsing TTP attribution) document that agents can generate exploits and emulate APTs, but neither shows that successful exploitation events have become so frequent as to be uninformative or so undetectable as
minor comments (5)
- Figure 3: The caption states curves are 'illustrative of the argued dynamic, not fitted measurements,' but the figure appears in a section (§3.4) making falsifiable predictions. A note clarifying that the schematic motivates rather than tests the predictions would reduce confusion.
- Appendix C, Finding 1: The text acknowledges that 99.8% of EUVD records also appear in CISA KEV, meaning the two datasets are nearly identical. This is handled honestly, but the earlier Appendix B statement calling EUVD a 'wholly separate corpus' should be corrected to match the more careful Appendix C statement.
- §2.3: The paper states the CRA 'mentions artificial intelligence only to cross-reference the AI Act' and contains 'no occurrence of machine learning.' A citation to the specific recital or article would strengthen this claim, which is rhetorically important for the 'loud silence' argument.
- Table 1: The 'Support-period lifecycle obligation' row is scored 'Bends→Breaks' but the text in §3.2 does not develop this intermediate verdict as thoroughly as the others. A sentence explaining why this mechanism is mid-transition rather than clearly bending or breaking would help readers follow the argument.
- §4.1, Recommendation 5: 'Introduce an adversary-capability baseline' is proposed but the operational form is underspecified. A brief note on whether this would be a static reference model, a living document maintained by ENISA, or a manufacturer-declared assumption would make the recommendation more actionable.
Circularity Check
Central legal-regulatory argument is self-contained against external evidence; remedy validation relies on author's own offensive and defensive tools but is not circular by construction
full rationale
The paper's central claim — the bends/breaks decomposition of the CRA — rests on the enacted Regulation text (external primary source), published CAI capability results from Google [9], DARPA [10], and independent academic groups [7, 8], and time-to-exploit data from the Zero Day Clock [18] and ENISA EUVD [29]. None of these are the author's own fitted inputs. The six predictions (Section 3.4) are genuinely falsifiable and not forced by construction. The remedy validation in Section 4.3 does rely heavily on the author's own work: offensive results on the Unitree G1 [20, 27] and Hookii mower [14] are the author's own, and the defensive Robot Immune System [28] is the author's own company's product, with the 79%→14% and 75%→8% figures drawn from 'controlled RIS evaluations' that are not independently verified. This is a legitimate self-evaluation concern (methodology risk), but it is not circularity in the logical sense: the claim 'defenders can track offenders because they draw on the same capability' is an empirical proposition that could be tested with independent offensive and defensive agents, and the supporting Attack/Defense CTF data [13] is co-authored but externally reproducible. No prediction or first-principles result reduces to its inputs by definition or by fit. The self-citation in the remedy section is not load-bearing for the central bends/breaks argument, which stands independently.
Axiom & Free-Parameter Ledger
free parameters (1)
- None (analytical paper)
axioms (5)
- domain assumption P1: Finding exploitable vulnerabilities takes costly, skilled human effort, so what is found is a small, slowly growing subset of what is latent.
- domain assumption P2: A product's set of known exploitable vulnerabilities can be enumerated at placement, making point-in-time assessment meaningful.
- domain assumption P3: Active exploitation is rare and distinct enough that 'reliable evidence' of it is an informative signal worth a 24/72-hour reporting duty.
- domain assumption P4: The interval between a vulnerability becoming known and its being weaponised is long enough for 'remediate without delay' to be a winnable race.
- domain assumption Defenders and attackers draw on the same AI capability, so defensive capability tracks offensive capability as the frontier advances.
invented entities (1)
-
Robot Immune System (RIS)
independent evidence
read the original abstract
The EU Cyber Resilience Act (CRA) makes a smart bet. It does not demand that products be free of vulnerabilities, but only that manufacturers run a process: assess risk, handle flaws, ship updates. The bet pays off if four things about the world stay true: (P1) finding vulnerabilities is slow, skilled, human work; (P2) a product's exploitable flaws are knowable the day it ships; (P3) exploitation is rare enough to notice; and (P4) fixes keep pace with discovery. Cybersecurity AI (CAI) agents, AI put to work finding and exploiting flaws in other products, falsify all four. The regime answers in two opposite ways. Against the sheer volume of flaws that agents surface it bends (P1): built for scarce attention, it re-centres compliance on defensible, documented prioritisation, and holds. But agents also collapse the speed and economics of the vulnerability lifecycle, and here it breaks (P2, P3, P4): a product that passed every check becomes exploitable without anyone touching it, so its market-entry test, its reporting trigger, and its one-and-done certificate vouch for a security that has quietly expired. The fault is in the landscape, not the product, so running the process more diligently cannot repair it. We map each mechanism to the force that strains or snaps it, and find the cure and the disease cut from the same cloth: because defenders and attackers wield the same AI, the only conformity that survives is one that never stops running. We also carry the remedy from proposal to proof on two CRA-scope robots, a humanoid and a lawn mower, where an agentic defender holds a line their undefended selves cannot. On the evidence already in hand, the CRA reaches full force in December 2027 certifying products against a world that has already changed. Static, human-paced security is finished; what replaces it must be continuous and agent-operated, and that is no longer a matter of taste.
Figures
Reference graph
Works this paper leans on
-
[1]
European Parliament and Council of the European Union. Regulation (eu) 2024/2847 of the european parliament and of the council of 23 october 2024 on horizontal cybersecurity requirements for products with digital elements (cyber resilience act). Official Journal of the European Union, OJ L, 2024/2847, 20 November 2024, 2024. CELEX 32024R2847
work page 2024
-
[2]
European Commission. Proposal for a regulation on horizontal cybersecurity requirements for products with digital elements (cyber resilience act). COM(2022) 454 final, 15 September 2022, 2022
work page 2022
-
[3]
Cybersecurity AI: The Dangerous Gap Between Automation and Autonomy
Víctor Mayoral-Vilches. Cybersecurity AI: The dan- gerous gap between automation and autonomy.arXiv preprint arXiv:2506.23592, 2025. Submitted 30 June 2025; six-level autonomy taxonomy for Cybersecurity AI
work page internal anchor Pith review Pith/arXiv arXiv 2025
-
[4]
Fabian Teichmann and Bruno S. Sergi. The EU cyber resilience act: Hybrid governance, compliance, and cybersecurity regulation in the digital ecosystem. Computer Law & Security Review, 59:106209, 2025. DOI 10.1016/j.clsr.2025.106209; CRA vulnerability gover- nance as enforced self-regulation / meta-regulation
-
[5]
Jukka Ruohonen and Paul Timmers. Vulnerability coordination under the cyber resilience act.Ap- plied Cybersecurity & Internet Governance, 2025. DOI 10.60097/acig/213350; arXiv:2412.06261; analyses actively-exploited-vulnerability reporting vs the US KEV concept
-
[6]
The Linux Foundation. Understanding the cyber resilience act: What everyone involved in open source development should know. Linux Foundation Research,
-
[7]
CRA’s manufacturer assumptions do not hold for OSS maintainers, who may not know downstream users
-
[8]
LLM Agents can Autonomously Exploit One-day Vulnerabilities
Richard Fang, Rohan Bindu, Akul Gupta, and Daniel Kang. LLM agents can autonomously exploit one-day vulnerabilities.arXiv preprint arXiv:2404.08144, 2024. Submitted 11 April 2024
work page internal anchor Pith review Pith/arXiv arXiv 2024
-
[9]
Teams of LLM Agents can Exploit Zero-Day Vulnerabilities
Richard Fang, Rohan Bindu, Akul Gupta, and Daniel Kang. Teams of LLM agents can exploit zero-day vulnerabilities.arXiv preprint arXiv:2406.01637, 2024. Submitted 2 June 2024
work page internal anchor Pith review Pith/arXiv arXiv 2024
-
[10]
From naptime to big sleep: Using large language models to catch vulnerabilities in real-world code
Google Project Zero and DeepMind. From naptime to big sleep: Using large language models to catch vulnerabilities in real-world code. Google Project Zero, November 2024, 2024. First AI-discovered exploitable flaw (stack buffer underflow) in production SQLite
work page 2024
-
[11]
Ai cyber challenge (aixcc): Final competition results
DARPA. Ai cyber challenge (aixcc): Final competition results. U.S. Defense Advanced Research Projects Agency, DEF CON 33, 8 August 2025, 2025
work page 2025
-
[12]
Automated exploit generation: How LLMs cross the threshold from research to operational tooling
Cloud Security Alliance. Automated exploit generation: How LLMs cross the threshold from research to operational tooling. Cloud Security Alliance whitepaper, April 2026, 2026
work page 2026
-
[13]
CAI: An Open, Bug Bounty-Ready Cybersecurity AI
Víctor Mayoral-Vilches, Luis Javier Navarrete-Lozano, María Sanz-Gómez, Lidia Salas Espejo, et al. CAI: An open, bug bounty-ready cybersecurity AI.arXiv preprint arXiv:2504.06017, 2025. Submitted 8 April 2025. The CRA under Cybersecurity AI Agents 12
work page internal anchor Pith review Pith/arXiv arXiv 2025
-
[14]
Cybersecurity AI: Eval- uating agentic cybersecurity in attack/defense CTFs
Francesco Balassone, Víctor Mayoral-Vilches, Stefan Rass, Martin Pinzger, et al. Cybersecurity AI: Eval- uating agentic cybersecurity in attack/defense CTFs. arXiv preprint arXiv:2510.17521, 2025. Submitted 20 October 2025
-
[15]
Cybersecurity AI: Hacking consumer robots in the AI era.arXiv preprint arXiv:2603.08665, 2026
Víctor Mayoral-Vilches, Unai Ayucar-Carbajo, Olivier Laflamme, Ruikai Peng, et al. Cybersecurity AI: Hacking consumer robots in the AI era.arXiv preprint arXiv:2603.08665, 2026. Submitted 9 March 2026
-
[16]
Synthetic APTs: the Collapse of TTP-Based Attribution
Francesco Balassone, Víctor Mayoral-Vilches, María Sanz-Gómez, Paul Zabalegui-Landa, et al. Synthetic APTs: the collapse of TTP-based attribution.arXiv preprint arXiv:2606.07158, 2026. Submitted 5 June 2026
work page internal anchor Pith review Pith/arXiv arXiv 2026
-
[17]
George A. Akerlof. The market for “lemons”: Quality uncertainty and the market mechanism.The Quarterly Journal of Economics, 84(3):488–500, 1970
work page 1970
-
[18]
The economics of information security.Science, 314(5799):610–613, 2006
Ross Anderson and Tyler Moore. The economics of information security.Science, 314(5799):610–613, 2006
work page 2006
-
[19]
Zero day clock: Tracking time-to- exploit across 83,000+ cves
Zero Day Clock. Zero day clock: Tracking time-to- exploit across 83,000+ cves. https://zerodayclock. com/, 2026. Aggregates 3,549 CVE-exploit pairs from 10 sources incl. CISA KEV, ExploitDB, Metasploit; snapshot accessed 1 July 2026. Median TTE fell from 771 days (2018) to 0 days (2025); exponential-decay fit R2=0.98
work page 2026
-
[20]
PentestGPT: An LLM-empowered Automatic Penetration Testing Tool
Gelei Deng, Yi Liu, Víctor Mayoral-Vilches, Peng Liu, Yuekang Li, Yuan Xu, Tianwei Zhang, Yang Liu, Martin Pinzger, and Stefan Rass. PentestGPT: Evaluating and harnessing large language models for automated penetration testing. In33rd USENIX Security Symposium (USENIX Security 24), pages 847– 864, 2024. arXiv:2308.06782; LLM-guided penetration testing, an...
work page internal anchor Pith review Pith/arXiv arXiv 2024
-
[21]
The cybersecurity of a humanoid robot.arXiv preprint arXiv:2509.14096,
Víctor Mayoral-Vilches. The cybersecurity of a humanoid robot.arXiv preprint arXiv:2509.14096,
-
[22]
Submitted 17 September 2025; CAI agent operationalised on the Unitree G1 humanoid
work page 2025
-
[23]
Víctor Mayoral-Vilches, María Sanz-Gómez, Francesco Balassone, et al. Dynamic cyber ranges.arXiv preprint arXiv:2604.24184, 2026. Submitted 27 April 2026; LLM- driven Defender agents reduce attacker success to 0– 55%
work page internal anchor Pith review Pith/arXiv arXiv 2026
-
[24]
CVE-Bench: A Benchmark for AI Agents' Ability to Exploit Real-World Web Application Vulnerabilities
Yuxuan Zhu, Antony Kellermann, Dylan Bowman, Philip Li, Akul Gupta, Adarsh Danda, Richard Fang, et al. CVE-Bench: A benchmark for AI agents’ ability to exploit real-world web application vulnerabilities. arXiv preprint arXiv:2503.17332, 2025. SOTA agent framework resolves up to 13% of real-world critical- severity web CVEs
work page internal anchor Pith review Pith/arXiv arXiv 2025
-
[25]
CyberGym: Evaluating AI agents’ real-world cybersecurity capabilities at scale
Zhun Wang, Tianneng Shi, Jingxuan He, Matthew Cai, Jialin Zhang, and Dawn Song. CyberGym: Evaluating AI agents’ real-world cybersecurity capabilities at scale. arXiv preprint arXiv:2506.02548, 2025. 1,507 real vulnerabilities across 188 projects; top combinations 20% success; discovered 34 zero-days and 18 incomplete patches
-
[26]
Shengye Wan, Cyrus Nikolaidis, Daniel Song, David Molnar, James Crnkovich, Jayson Grace, Manish Bhatt, et al. CYBERSECEVAL 3: Advancing the evaluation of cybersecurity risks and capabilities in large language models.arXiv preprint arXiv:2408.01605, 2024. Meta offensive-capability benchmark suite; autonomous offen- sive cyber operations
work page internal anchor Pith review Pith/arXiv arXiv 2024
-
[27]
An empirical study on vulnerability disclosure management of open source software systems
ACM Transactions on Software Engineering and Methodology. An empirical study on vulnerability disclosure management of open source software systems. ACM Trans. Softw. Eng. Methodol., 2025. DOI 10.1145/3716822; median 36 days patch-to-disclosure; 23% of exploits within a week, 50% within a month
-
[28]
Time between disclosure, patch release and vulnerability exploitation
Mandiant / Google Threat Intelligence. Time between disclosure, patch release and vulnerability exploitation. Technical report, Google Cloud, 2024. Industry- reported collapse of time-to-exploit; exploitation fre- quently precedes or coincides with disclosure
work page 2024
-
[29]
Cybersecurity AI: Humanoid robots as attack vectors.arXiv preprint arXiv:2509.14139, 2025
Víctor Mayoral-Vilches, Andreas Makris, and Kevin Finisterre. Cybersecurity AI: Humanoid robots as attack vectors.arXiv preprint arXiv:2509.14139, 2025. Submitted 17 September 2025; BLE command injection via fleet-wide hardcoded AES keys, static-key FMX telemetry, and resident-agent pivot on the Unitree G1
-
[30]
Robot immune system (RIS): a robotics endpoint protection platform.https://aliasrobotics
Alias Robotics. Robot immune system (RIS): a robotics endpoint protection platform.https://aliasrobotics. com/ris.php, 2026. Defence-in-depth robotics endpoint protection (adaptive firewall, hardening, forensic log- ging, artificial immune system) paired, in current builds, with a continuously running agentic Cybersecurity AI defender; on-premise, IEC 624...
work page 2026
-
[31]
European Union Agency for Cybersecurity (ENISA). European union vulnerability database (euvd).https: //euvd.enisa.europa.eu/, 2026. Known-exploited- vulnerability dump (n=1,634) via the EUVD API, sources CISA KEV and EU-KEV; snapshot accessed 1 July 2026. The CRA under Cybersecurity AI Agents 13 A The Zero Day Clock time-to-exploit data The claim that the...
work page 2026
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.