Position: Mind the Gap-AI Security and the Limits of Current Reporting Standards
Pith reviewed 2026-05-23 07:21 UTC · model grok-4.3
The pith
Established processes are not well aligned with AI security reporting due to fundamental shortcomings for the distinctive characteristics of AI systems.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Established processes are not well aligned with AI security reporting due to fundamental shortcomings for the distinctive characteristics of AI systems. Some of these shortcomings are immediately addressable, while others remain unresolved technically or within social systems, like the treatment of IP or the ownership of a vulnerability. The advent of AI agents will further reinforce the need to advance specialized AI security incident reporting.
What carries the argument
The distinctive characteristics of AI systems, especially IP treatment and vulnerability ownership, which produce misalignments with both cybersecurity and non-security AI reporting norms.
If this is right
- Some shortcomings in AI security reporting can be addressed immediately while others require technical or social resolution.
- Current proposals for AI security incident reporting contain limitations that must be examined.
- The development of AI agents will increase the urgency for specialized reporting standards.
Where Pith is reading between the lines
- Organizations may need separate channels for reporting AI vulnerabilities that avoid exposing proprietary model details.
- Policy makers could create AI-specific incident taxonomies that differ from both general cybersecurity and ethical AI reporting.
- Industry groups might form dedicated AI security incident repositories that define new norms for vulnerability ownership.
Load-bearing premise
AI systems possess distinctive characteristics that create fundamental misalignments with both non-AI cybersecurity reporting and non-security AI reporting.
What would settle it
A demonstration that existing non-AI cybersecurity reporting standards can fully accommodate documented AI security incidents without new categories, processes, or ownership rules would falsify the misalignment claim.
Figures
read the original abstract
AI systems face a growing number of AI security threats that are increasingly exploited in the real world. Hence, shared AI incident reporting practices are emerging in industry as best practice and as mandated by regulatory requirements. Although non-AI cybersecurity and non-security AI reporting have progressed as industrial and policy norms, existing collections of practices do not meet the specific requirements posed by AI security reporting. we argue that established processes are not well aligned with AI security reporting due to fundamental shortcomings for the distinctive characteristics of AI systems. Some of these shortcomings are immediately addressable, while others remain unresolved technically or within social systems, like the treatment of IP or the ownership of a vulnerability. Based on this position, we examine the limitations of current AI security incident reporting proposals. We conclude that the advent of AI agents will further reinforce the need to advance specialized AI security incident reporting.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript is a position paper arguing that established AI incident reporting processes are not well aligned with AI security due to fundamental shortcomings arising from distinctive characteristics of AI systems (e.g., IP treatment and vulnerability ownership). It contrasts these with non-AI cybersecurity and non-security AI reporting norms, examines limitations of current proposals, and concludes that the advent of AI agents will further necessitate specialized AI security incident reporting.
Significance. If the position holds, the paper usefully synthesizes gaps between existing reporting regimes and AI-specific security needs, which could inform policy development and industry standards as AI threats increase. It provides a coherent advocacy piece that explicitly acknowledges some shortcomings are addressable while highlighting unresolved ones, offering a clear framing for future work even without new empirical data or formal derivations.
major comments (1)
- [Abstract] Abstract: The core claim that established processes have 'fundamental shortcomings' for the 'distinctive characteristics of AI systems' (with examples of IP treatment and vulnerability ownership) is load-bearing for the argument that specialized reporting is required rather than incremental fixes. The text distinguishes addressable from unresolved shortcomings but provides no explicit criterion, case analysis, or comparison to support why the latter are fundamental rather than addressable through extensions of existing frameworks.
Simulated Author's Rebuttal
We thank the referee for their detailed review and constructive feedback on our position paper. We address the major comment below and will revise the manuscript accordingly to strengthen the argument.
read point-by-point responses
-
Referee: [Abstract] Abstract: The core claim that established processes have 'fundamental shortcomings' for the 'distinctive characteristics of AI systems' (with examples of IP treatment and vulnerability ownership) is load-bearing for the argument that specialized reporting is required rather than incremental fixes. The text distinguishes addressable from unresolved shortcomings but provides no explicit criterion, case analysis, or comparison to support why the latter are fundamental rather than addressable through extensions of existing frameworks.
Authors: We agree that an explicit criterion would strengthen the distinction between addressable and fundamental shortcomings. In the revised version, we will add a dedicated paragraph in the introduction (and reference it in the abstract) defining 'fundamental' shortcomings as those that cannot be resolved by incremental extensions of existing frameworks because they stem from AI-specific properties that alter core assumptions in reporting (e.g., the non-deterministic, data-dependent, and multi-stakeholder nature of AI systems). For IP treatment, we will include a brief case analysis contrasting traditional software (where source code ownership is clear) with AI (where training data, weights, and generated outputs create overlapping IP claims not addressed by CVE-style vulnerability disclosure). Similarly for vulnerability ownership, we will compare to non-AI cases where a single vendor owns the flaw versus AI where the 'vulnerability' may arise from model behavior, user prompts, or third-party data. This supports why specialized standards are needed rather than extensions alone, while acknowledging that some gaps (e.g., basic logging) are addressable. revision: yes
Circularity Check
No significant circularity; position paper with no derivations
full rationale
The paper is a position/advocacy piece with no equations, parameters, or formal derivations. Its central claim—that AI systems have distinctive characteristics creating misalignments with existing reporting regimes—is presented explicitly as a premise rather than derived from any self-citation chain, fitted input, or self-referential definition. No load-bearing steps reduce to inputs by construction; the argument draws on domain knowledge of reporting practices without internal reduction. This matches the expected non-finding for non-empirical position papers.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption AI systems possess distinctive characteristics that create fundamental misalignments with established non-AI cybersecurity and non-security AI reporting processes.
Reference graph
Works this paper leans on
-
[1]
Key Challenges in Implementing Zero Trust Security
-
[2]
Technical Report V2.0, Cybersecurity & Infrastruc- ture Security Agency (CISA), Apr
Zero Trust Maturity Model. Technical Report V2.0, Cybersecurity & Infrastruc- ture Security Agency (CISA), Apr. 2023
work page 2023
-
[3]
Technical report, Ponemon Institute, Sponsored by Entrust, May 2024
2024 Zero Trust & Encryption Study. Technical report, Ponemon Institute, Sponsored by Entrust, May 2024
work page 2024
-
[4]
S. Abdelnabi, K. Greshake, S. Mishra, C. Endres, T. Holz, and M. Fritz. Not what you’ve signed up for: Compromising real-world llm-integrated applications with indirect prompt injection. In Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security, pages 79–90, 2023
work page 2023
-
[5]
D. Adkins, B. Alsallakh, A. Cheema, N. Kokhlikyan, E. McReynolds, P. Mishra, C. Procope, J. Sawruk, E. Wang, and P. Zvyagina. Method cards for prescriptive 16https://cloud.google.com/transform/101-real-world-generative-ai-use-cases-from- industry-leaders Conference acronym ’XX, June 03–05, 2018, Woodstock, NY Lukas Bieringer1, Sean Mc Gregor2, Nicole Nich...
work page 2018
-
[6]
H. Aldawood and G. Skinner. Educating and raising awareness on cyber security social engineering: A literature review. In 2018 IEEE international conference on teaching, assessment, and learning for engineering (TALE) , pages 62–68. IEEE, 2018
work page 2018
-
[7]
M. Andriushchenko, F. Croce, and N. Flammarion. Jailbreaking leading safety- aligned llms with simple adaptive attacks. In ICML 2024 Next Generation of AI Safety Workshop
work page 2024
-
[8]
real attackers don’t compute gradients
G. Apruzzese, H. S. Anderson, S. Dambra, D. Freeman, F. Pierazzi, and K. Roundy. “real attackers don’t compute gradients”: bridging the gap between adversarial ml research and practice. In 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pages 339–364. IEEE, 2023
work page 2023
-
[9]
M. Arnold, R. K. Bellamy, M. Hind, S. Houde, S. Mehta, A. Mojsilović, R. Nair, K. N. Ramamurthy, A. Olteanu, D. Piorkowski, et al. Factsheets: Increasing trust in ai services through supplier’s declarations of conformity. IBM Journal of Research and Development, 63(4/5):6–1, 2019
work page 2019
-
[10]
AI vulnerability database (AVID)
ARVA. AI vulnerability database (AVID). https://avidml.org/database/. Accessed: 2024-06-14
work page 2024
-
[11]
D. Baier. Künstliche intelligenz und kriminalität. SKP Info, 2024(1):5–10, 2024
work page 2024
-
[12]
D. W. Baker, S. M. Christey, W. H. Hill, and D. E. Mann. The development of a Common Vulnerability Enumeration, 1999
work page 1999
-
[13]
M. Barreno, B. Nelson, R. Sears, A. D. Joseph, and J. D. Tygar. Can ml be secure? In ACM Symposium on Inf., Computer and Communications Security, ASIACCS , pages 16–25. ACM, 2006
work page 2006
-
[14]
E. M. Bender and B. Friedman. Data statements for natural language processing: Toward mitigating system bias and enabling better science. Transactions of the Association for Computational Linguistics , 6, 2018
work page 2018
- [15]
-
[16]
L. Bieringer, K. Grosse, M. Backes, B. Biggio, and K. Krombholz. Industrial practitioners’ mental models of adversarial machine learning. In Eighteenth Symposium on Usable Privacy and Security (SOUPS 2022) , pages 97–116, 2022
work page 2022
-
[17]
B. Biggio and F. Roli. Wild patterns: Ten years after the rise of adversarial ma- chine learning. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, pages 2154–2156, 2018
work page 2018
-
[18]
C. M. Bishop and N. M. Nasrabadi. Pattern recognition and machine learning , volume 4. Springer, 2006
work page 2006
- [19]
-
[20]
Y. Cao, C. Xiao, A. Anandkumar, D. Xu, and M. Pavone. Advdo: Realistic adversarial attacks for trajectory prediction. In ECCV, pages 36–52, 2022
work page 2022
-
[21]
On Evaluating Adversarial Robustness
N. Carlini, A. Athalye, N. Papernot, W. Brendel, J. Rauber, D. Tsipras, I. Goodfel- low, A. Madry, and A. Kurakin. On evaluating adversarial robustness. arXiv preprint arXiv:1902.06705, 2019
work page internal anchor Pith review Pith/arXiv arXiv 1902
-
[22]
N. Carlini, M. Jagielski, C. A. Choquette-Choo, D. Paleka, W. Pearce, H. Anderson, A. Terzis, K. Thomas, and F. Tramèr. Poisoning web-scale training datasets is practical. In 2024 IEEE Symposium on Security and Privacy (SP) , pages 407–425. IEEE, 2024
work page 2024
-
[23]
N. Carlini, D. Paleka, K. D. Dvijotham, T. Steinke, J. Hayase, A. F. Cooper, K. Lee, M. Jagielski, M. Nasr, A. Conmy, et al. Stealing part of a production language model. In Forty-first International Conference on Machine Learning , 2024
work page 2024
-
[24]
N. Carlini, F. Tramer, E. Wallace, M. Jagielski, A. Herbert-Voss, K. Lee, A. Roberts, T. Brown, D. Song, U. Erlingsson, et al. Extracting training data from large language models. In 30th USENIX security symposium (USENIX Security 21) , pages 2633–2650, 2021
work page 2021
-
[25]
M. Carvalho, J. DeMott, R. Ford, and D. A. Wheeler. Heartbleed 101. IEEE security & privacy, 12(4):63–67, 2014
work page 2014
-
[26]
S. Cattell, A. Ghosh, and L.-A. Kaffee.Coordinated Flaw Disclosure for AI: Beyond Security Vulnerabilities, page 267–280. AAAI Press, 2025
work page 2025
-
[27]
N. C. S. R. Center. Glossary. https://csrc.nist.gov/glossary. Accessed: 2025-05-22
work page 2025
-
[28]
S. Cha. Towards an international regulatory framework for ai safety: lessons from the iaea’s nuclear safety regulations. Humanities and Social Sciences Communications, 11(1):1–13, 2024
work page 2024
-
[29]
A. E. Cinà, K. Grosse, A. Demontis, S. Vascon, W. Zellinger, B. A. Moser, A. Oprea, B. Biggio, M. Pelillo, and F. Roli. Wild patterns reloaded: A survey of machine learning security against training data poisoning. ACM Comput. Surv., 2023
work page 2023
- [30]
-
[31]
Council regulation (EU) no 2024/1689, 2024
Council of European Union. Council regulation (EU) no 2024/1689, 2024. https: //artificialintelligenceact.eu
work page 2024
- [32]
- [33]
- [34]
-
[35]
C. S. de Witt. Open challenges in multi-agent security: Towards secure systems of interacting ai agents, 2025
work page 2025
-
[36]
Z. Deng, Y. Guo, C. Han, W. Ma, J. Xiong, S. Wen, and Y. Xiang. Ai agents under threat: A survey of key security challenges and future pathways. ACM Computing Surveys, 57(7):1–36, 2025
work page 2025
-
[37]
T. Dinh, Y. Zeng, R. Zhang, Z. Lin, M. Gira, S. Rajput, J.-y. Sohn, D. Papailiopoulos, and K. Lee. Lift: Language-interfaced fine-tuning for non-language machine learning tasks. Advances in Neural Information Processing Systems , 35:11763– 11784, 2022
work page 2022
-
[38]
C. Dwork. Differential privacy. In International colloquium on automata, lan- guages, and programming, pages 1–12. Springer, 2006
work page 2006
-
[39]
S. Ekelund and Z. Iskoujina. Cybersecurity economics–balancing operational security spending. Information Technology & People, 32(5):1318–1342, 2019
work page 2019
-
[40]
M. Fazelnia, S. Moshtari, and M. Mirakhorli. Establishing minimum ele- ments for effective vulnerability management in ai software. arXiv preprint arXiv:2411.11317, 2024
-
[41]
K. Foss-Solbrekk. Three routes to protecting AI systems and their algorithms under IP law: The good, the bad and the ugly. Journal of Intellectual Property Law & Practice, 16(3):247–258, 2021
work page 2021
-
[42]
A. Gnanasambandam, A. M. Sherman, and S. H. Chan. Optical adversarial attack. In ICCV, pages 92–101, 2021
work page 2021
-
[43]
D. Golpayegani, I. Hupont, C. Panigutti, H. J. Pandit, S. Schade, D. O’Sullivan, and D. Lewis. Ai cards: Towards an applied framework for machine-readable ai and risk documentation inspired by the eu ai act. In Annual Privacy Forum, pages 48–72. Springer, 2024
work page 2024
- [44]
- [45]
- [46]
- [47]
- [48]
-
[49]
K. Grosse, N. Papernot, P. Manoharan, M. Backes, and P. McDaniel. Adversarial examples for malware detection. In Computer Security–ESORICS 2017: 22nd European Symposium on Research in Computer Security, Oslo, Norway, September 11-15, 2017, Proceedings, Part II 22 , pages 62–79. Springer, 2017
work page 2017
- [50]
- [51]
-
[52]
T. Gu, K. Liu, B. Dolan-Gavitt, and S. Garg. Badnets: Evaluating backdooring attacks on deep neural networks. IEEE Access, 7:47230–47244, 2019
work page 2019
-
[53]
W. Guo, Y. Potter, T. Shi, Z. Wang, A. Zhang, and D. Song. Frontier ai’s impact on the cybersecurity landscape, 2025
work page 2025
-
[54]
P. Hacker. The european ai liability directives–critique of a half-hearted ap- proach and lessons for the future. Computer Law & Security Review , 51:105871, 2023
work page 2023
- [55]
-
[56]
Distilling the Knowledge in a Neural Network
G. Hinton, O. Vinyals, and J. Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015
work page internal anchor Pith review Pith/arXiv arXiv 2015
-
[57]
A. Householder, V. Sarvepalli, J. Havrilla, M. Churilla, L. Pons, S. hon Lau, N. Vanhoudnos, A. Kompanek, and L. McIlvenny. Lessons learned in coordinated disclosure for artificial intelligence and machine learning systems, 8 2024
work page 2024
- [58]
- [59]
-
[60]
ISO. Information Technology; Security Techniques; Information Security Manage- ment Systems Based on ISO/IEC 27000 . International Organization for Standard- ization, 2018
work page 2018
-
[61]
ISO. ISO/IEC 27035-1 Information technology — Information security incident management — Part 1: Principles and process . International Organization for Standardization, 2023
work page 2023
-
[62]
M. Jagielski, M. Nasr, K. Lee, C. A. Choquette-Choo, N. Carlini, and F. Tramer. Students parrot their teachers: Membership inference on model distillation. Advances in Neural Information Processing Systems , 36:44382–44397, 2023. Position: Mind the Gap—the Growing Disconnect Between Established Vulnerability Disclosure and AI Security Conference acronym ’...
work page 2023
-
[63]
M. Jagielski, G. Severi, N. Pousette Harger, and A. Oprea. Subpopulation data poi- soning attacks. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, pages 3104–3122, 2021
work page 2021
-
[64]
C. Joynt. What is AI TRiSM and Why It’s Essential in the Era of GenAI, May 2025
work page 2025
-
[65]
F. Koh, K. Grosse, and G. Apruzzese. Voices from the frontline: Revealing the ai practitioners’ viewpoint on the european ai act. In HICSS, pages 1870–1879, 2024
work page 2024
-
[66]
A. Kumar and I. Sharma. Identifying patterns in common vulnerabilities and exposures databases with exploratory data analysis. In2022 International Confer- ence on Automation, Computing and Renewable Systems (ICACRS), pages 919–924. IEEE, 2022
work page 2022
-
[67]
A. Kurakin, I. J. Goodfellow, and S. Bengio. Adversarial examples in the physical world. In Artificial intelligence safety and security , pages 99–112. Chapman and Hall/CRC, 2018
work page 2018
-
[68]
Y. Lei, D. Ye, S. Shen, Y. Sui, T. Zhu, and W. Zhou. New challenges in reinforce- ment learning: a survey of security and privacy. Artificial Intelligence Review, 56(7):7195–7236, 2023
work page 2023
-
[69]
X. Li, S. Wang, S. Zeng, Y. Wu, and Y. Yang. A survey on llm-based multi-agent systems: workflow, infrastructure, and challenges. Vicinagearth, 1(1):9, 2024
work page 2024
-
[70]
T. Liu, X. Xie, and Y. Zhang. zkcnn: Zero knowledge proofs for convolutional neural network predictions and accuracy. InProceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security , pages 2968–2985, 2021
work page 2021
-
[71]
S. Longpre, K. Klyman, R. E. Appel, S. Kapoor, R. Bommasani, M. Sahar, S. McGre- gor, A. Ghosh, B. Blili-Hamelin, N. Butters, A. Nelson, A. Elazari, A. Sellars, C. J. Ellis, D. Sherrets, D. Song, H. Geiger, I. Cohen, L. McIlvenny, M. Srikumar, M. M. Jaycox, M. Anderljung, N. F. Johnson, N. Carlini, N. Miailhe, N. Marda, P. Hen- derson, R. S. Portnoff, R. ...
-
[72]
D. Lowd and C. Meek. Adversarial learning. In Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining , pages 641–647, 2005
work page 2005
-
[73]
Language model vulnerabilities and exposures
LVEs. Language model vulnerabilities and exposures. https://lve-project.org/se curity/. Accessed: 2024-06-14
work page 2024
-
[74]
R. Marcilly, J. Schiro, M. C. Beuscart-Zéphir, and F. Magrabi. Building usability knowledge for health information technology: a usability-oriented analysis of incident reports. Applied clinical informatics, 10(03):395–408, 2019
work page 2019
-
[75]
A. Mazaheri, J. Montewka, J. Nisula, and P. Kujala. Usability of accident and in- cident reports for evidence-based risk modeling–a case study on ship grounding reports. Safety science, 76:202–214, 2015
work page 2015
- [76]
-
[77]
S. McGregor, A. Ettinger, N. Judd, P. Albee, L. Jiang, K. Rao, W. Smith, S. Longpre, A. Ghosh, C. Fiorelli, M. Hoang, S. Cattell, and N. Dziri. To err is ai : A case study informing llm flaw reporting practices, 2024
work page 2024
-
[78]
M. H. Meng, G. Bai, S. G. Teo, Z. Hou, Y. Xiao, Y. Lin, and J. S. Dong. Adver- sarial robustness of deep neural networks: A survey from a formal verification perspective. IEEE Transactions on Dependable and Secure Computing , 2022
work page 2022
-
[79]
A. A. Mir. Transparency in ai supply chains: Addressing ethical dilemmas in data collection and usage. MZ Journal of Artificial Intelligence, 1(2), 2024
work page 2024
-
[80]
M. Mitchell, S. Wu, A. Zaldivar, P. Barnes, L. Vasserman, B. Hutchinson, E. Spitzer, I. D. Raji, and T. Gebru. Model cards for model reporting. In Proceedings of the conference on fairness, accountability, and transparency , pages 220–229, 2019
work page 2019
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.