Pith. sign in

REVIEW 4 major objections 5 minor 71 references

Safe deployment of LLM-based systems requires enforceable, pre-execution supply-chain guarantees, including cryptographic attestation of model artifacts and verification of agentic dependencies, rather than post-deployment monitoring or pol

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 21:45 UTC pith:GAS26FYR

load-bearing objection Useful survey with an honest prototype, but the central index is missing its equation and the abstract overclaims what attestation proves. the 4 major comments →

arxiv 2602.19021 v2 pith:GAS26FYR submitted 2026-02-22 cs.CR

Trustworthy AI LLM Scalability Risk Index (LSRI): A Cybersecurity Framework Assessing Agentic-AI Security & Software Model Supply Chain Safety Boosting AI-Generated Malware Defense & Explainability Mitigating Emerging Risks of Generative AI

Kiarash Ahi , Vaibhav Agrawal , Saeed Valizadeh This is my paper
classification cs.CR
keywords LLM Scalability Risk Indexsupply chain securitycryptographic attestationagentic AIprompt injectionmodel provenanceAI governancezero-day detection
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that LLM-based and agentic systems cannot be secured by monitoring and policy after the fact; the guarantees have to be enforced before the model executes. It introduces the LLM Scalability Risk Index (LSRI), a weighted parametric score that folds operational metrics—latency, throughput, cost, update frequency, model size, regulatory compliance—through nonlinear risk functions and multiplies the result by an integrity multiplier built from measured violation magnitudes such as prompt-injection success or data leakage. It then makes the stronger claim that enforceable supply-chain guarantees, specifically cryptographic attestation of model checkpoints and signed provenance metadata, are a necessary condition for scalable, safe deployment and for governance that can be verified rather than self-attested. A sympathetic reader would care because this reframes LLM governance as an engineering problem with machine-checkable controls, and offers a concrete baseline that security teams and regulators could adopt.

Core claim

The paper's central claim: security and governance of large language models cannot be achieved through post-deployment monitoring or policy alone; enforceable, pre-execution supply-chain guarantees spanning build-time artifacts and run-time agentic dependencies are a necessary condition for scalable, safe, governable deployment. It introduces the LLM Scalability Risk Index (LSRI), a parametric framework that normalizes raw operational metrics through nonlinear risk-mapping functions, weights them contextually, and multiplies by an integrity multiplier Φ representing the system's aggregate security health—so a model that fails fundamental security or compliance boundaries cannot earn a 'safe'

What carries the argument

The central object is the LLM Scalability Risk Index (LSRI), a weighted parametric score that maps raw metrics (latency, throughput, regulatory compliance, model size, update frequency, cost) through nonlinear risk functions and scales the weighted sum by an integrity multiplier Φ built from measured violation magnitudes such as prompt-injection success rate or PII leakage. The load-bearing mechanism is the verifiable root-of-trust architecture: cryptographic attestation—hashing the model checkpoint, signing the hash with an ephemeral build identity, recording it in an append-only transparency log—followed by mandatory verification gates before inference. This transforms supply-chain assuran

Load-bearing premise

The load-bearing premise is that a cryptographically valid signature on a model checkpoint is a meaningful proxy for the trustworthiness of that model's behavior; an attestation proves where a file came from, not that its training data was clean or that its runtime behavior under prompt injection is safe.

What would settle it

A concrete falsifier: take a checkpoint that passes the paper's verification gates—valid signature, anchored in the transparency log, policy-compliant metadata—and show that a malicious adapter or indirect prompt injection causes a security-relevant action, such as data exfiltration or a policy-violating tool call, at a high success rate without breaking the signature. This would demonstrate that pre-execution cryptographic attestation is not sufficient for safe deployment, undermining the necessity claim.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • LSRI can be used as a sensitivity-analysis tool: security teams can vary a single metric, such as latency or prompt-injection success rate, and see the deployment-readiness score cross a threshold that signals warning or failure.
  • Checkpoint attestation becomes a pre-execution gate: a model whose signature or provenance metadata fails verification cannot be loaded, regardless of performance or cost.
  • The trust chain is transitive: each downstream fine-tune or redistribution inherits a cryptographically verified predecessor, so tampering anywhere in the chain breaks the attestation.
  • Agentic runtime behavior can be brought under the same umbrella by gating tool invocation, memory, and action plans against signed manifests and policy.
  • Governance frameworks gain a machine-verifiable compliance signal, which the paper argues is what makes regulation of LLMs feasible at scale.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the paper's central claim is right, the next bottleneck is semantic trust: a signed artifact can still harbor poisoned training data or a backdoor inserted before signing. The paper sketches this gap, but does not close it, so the practical value of the whole architecture will hinge on that extension.
  • The LSRI weights and thresholds are presented as illustrative baselines; with incident data they could be calibrated empirically, turning the index from a sensitivity rubric into a predictive risk score. That calibration is not in the paper.
  • The attestation argument naturally extends beyond checkpoints to training data and fine-tuning scripts: if dataset hashes are committed to the transparency log before training, poisoning resistance becomes a property of the attested lineage. This is a testable extension of the paper's proof of concept.
  • The paper's capital-markets mention suggests a further consequence: if LSRI becomes a standard, financial exposure to LLM-dependent vendors could be priced by their verifiable supply-chain posture, but the paper does not develop that link.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper is a broad survey/position piece on dual-use LLMs in cybersecurity. It proposes two contributions: (i) the LLM Scalability Risk Index (LSRI), a parametric formula intended to quantify deployment readiness under latency, throughput, regulatory, model-size, update-frequency, and cost constraints, together with an "Integrity Multiplier" Phi; and (ii) a model-supply-chain framework that uses cryptographic attestation (SHA-256 hashing and Sigstore signing of a model checkpoint) to establish a "verifiable root of trust." The authors claim that LSRI plus this root-of-trust architecture provides a deployable compliance baseline, defends against data poisoning, backdoors, malicious LoRA adapters, and weight tampering, and shifts security from zero-trust to semantic trust. The paper includes worked examples, a sensitivity table, a policy maturity matrix, and references to 70 sources.

Significance. If the claims were supported, the paper could be useful as a decision rubric: a well-defined parametric scalability/security risk score, combined with provenance attestation for model artifacts, would address a real need in LLM deployment governance. The cryptographic-attestation prototype is a sensible use of existing software supply-chain tooling (Sigstore, transparency logs), and the literature synthesis is broad. However, the significance is not yet realized: the central equation is never actually displayed; the sensitivity analysis is computed from the same hand-set thresholds it purports to test; and the root-of-trust prototype proves file provenance, not behavioral safety. The paper is best read as a position/survey and a partial engineering sketch, not as a validated framework. The authors deserve credit for stating, in the LSRI section, that the index is not an empirically calibrated estimator of real-world incident probability; that caveat needs to be prominent throughout the paper and reflected in the abstract.

major comments (4)
  1. [Section IV, "Equation 1" and Table 5] The central object of the paper, LSRI, is not actually defined. The text names variables (x_i, f_i, w_i, tau, sigma, lambda, Phi) and labels Equations (1)-(8), but no displayed formula for LSRI or Phi appears anywhere. A reader cannot compute LSRI for any scenario or check the algebra. Table 5 is called a "sensitivity analysis," but it is generated by substituting one latency value into Scenario A using the same hand-set sigmoid threshold (tau=100ms, sigma=15) from Table 3; it is an illustration of the sigmoid function, not validation. The paper's own statement that LSRI is "not an empirically calibrated estimator of real-world incident probability" should be moved to the abstract and the section should be retitled from "practical validation" to "worked example."
  2. [Section V, "Prototype implementation" and "Verifiable provenance and data integrity"] The security guarantees claimed in the abstract are not supported by the implemented PoC. Signing SHA-256(model.ckpt) with Sigstore and verifying the signature/hash/metadata establishes only that the file came from a particular build pipeline. It does not establish that the model is free of poisoned training data, backdoors, or malicious LoRA adapters; a checkpoint from a compromised or negligent pipeline passes every verification gate. The paper itself concedes that the PoC "does not yet capture why a model behaves as it does," and that semantic trust is future work. This concession is load-bearing: it contradicts the abstract's claim that LSRI plus the root-of-trust architecture defends against data poisoning, backdoors, and malicious adapters. The claims must be narrowed to provenance attestation, or a mechanism binding behavioral safety properties to the attestation must be implement
  3. [Section IV, Table 6] The quantitative performance comparison of static analyzers versus LLM-based zero-day detectors has no cited source or experimental description. The recall figures (62%, 88%, 91%), latencies (80/105/96 ms), false-positive rates (11%/13%/12%), and the resulting "~42-47% relative improvement" are presented as empirical results but no dataset, benchmark, or repository is provided. Since this is the only quantitative empirical evidence in the paper for LLM superiority in zero-day detection, it must either be removed and labeled as hypothetical, or replaced with references to actual benchmark measurements.
  4. [Section V, "Robustness against agentic LLM attacks"] The transition from "zero-trust" to "semantic trust" is asserted but not operationalized. The paper lists multi-stage plan validation, signed manifests, and constraint-aware memory management as "practical extensions" of the attestation framework, but no mechanism, policy language, or evaluation is given. The claim that the architecture secures run-time agentic dependencies is therefore unsupported. This is not a minor omission: the abstract advertises defense against agentic-AI threats, while the implemented portion addresses only static artifact integrity. The authors should either present a concrete semantic-enforcement mechanism or explicitly state that agentic robustness is a research direction.
minor comments (5)
  1. [Title/Abstract (submission header vs full text)] The submission contains two inconsistent abstracts. The header abstract emphasizes MCP, NASDAQ-100/QQQ, and capital-market analysis; the full-text abstract does not mention these. The authors must choose one abstract and align it with the actual content.
  2. [Equations (1)-(8)] All equation numbers are referenced but none of the equations are typeset. For a parametric framework, Equations (2)-(8) (mapping functions, Phi, regulatory risk, frequency) need to be displayed explicitly; otherwise the framework is not reproducible.
  3. [Table 1] The Policy Maturity Matrix says it is "weighted equally across five pillars," but the table lists only four criteria and no scoring rubric or raw scores. The columns "High/Medium" and "Required/Encouraged" cannot be reproduced from the stated methodology.
  4. [Author affiliations] Author names have typographical inconsistencies: "Ahia" vs "Ahi" in the contact line, "Agrawalb," "Valizadehb," and superscript letters that do not match affiliations. These should be corrected.
  5. [References] Several references lack standard bibliographic detail or venue information (e.g., [4], [9], [28], [29], [30], [35], [36], [37], [45], [46]). If the paper is intended for journal submission, the reference list must be completed and checked.

Circularity Check

0 steps flagged

No significant circularity: LSRI is explicitly a parametric scoring rubric, and the attestation PoC is an engineering demonstration with an acknowledged semantic-trust gap.

full rationale

The paper's central quantitative contribution, LSRI, is presented as a definitional risk-scoring framework, not as an empirical estimator or a prediction derived from fitted data. The paper explicitly states: 'The LSRI provides a structured, forward-looking risk score that supports comparative assessment of scalability—security stress across deployment scenarios, rather than an empirically calibrated estimator of real-world incident probability.' The worked examples and sensitivity analysis in Tables 4 and 5 are introduced as demonstrations of the rubric ('To demonstrate the application of the LSRI, we evaluate two hypothetical model deployment scenarios'), and the Table 5 footnote says the LSRI values are computed by substituting the specified latency into the Scenario A baseline while holding other factors constant. This is a self-consistency check of a definitional index, not a fitted parameter being renamed as a prediction. The supply-chain argument is built on external primitives (SHA-256, Sigstore) and external prior work (Tramèr et al., Goldblum et al.), with no load-bearing self-citation chain. The paper also candidly concedes the key limitation of its PoC: 'it does not yet capture why a model behaves as it does—namely, which datasets, transformations, and alignment steps influenced its training.' That limitation undercuts the strength of the security claim, but it is a validity gap, not a circular reduction. No equation is shown to be equivalent to its own inputs by construction, and no fitted value is relabeled as an independent prediction. Therefore no circularity is present.

Axiom & Free-Parameter Ledger

8 free parameters · 5 axioms · 1 invented entities

The central contribution rests on hand-selected thresholds, a definitional formula, and standard cryptographic primitive assumptions. No external data are used to fit or validate the parameters, and the paper explicitly disclaims empirical calibration. The only invented entity, LSRI, lacks independent evidence beyond the authors' own worked examples.

free parameters (8)
  • Latency sigmoid threshold τ = 100 ms
    Hand-set in Table 3 as the critical latency threshold for the sigmoid mapping; no calibration is provided.
  • Latency sigmoid sensitivity σ = 15
    Hand-set in Table 3 to control the steepness of the sigmoid risk curve.
  • Throughput baseline λ = 10^6 requests/day
    Hand-set in Table 3 as the industrial-scale throughput baseline for exponential risk decay.
  • Model-size step threshold = 20B parameters
    Chosen in Table 3 as the binary hardware-compatibility boundary for model size.
  • Cost ceiling = $1000/day
    Chosen in Table 3 as the linear cost threshold; no cost model or market data is cited.
  • Update frequency target = 2 updates/day
    Chosen in Table 3 for the linear update-frequency mapping.
  • Equal feature weights wi = 1/n = 1/6 in worked examples
    Adopted as a baseline in the LSRI definition, with no sensitivity analysis over weight choices.
  • Integrity multiplier sensitivity coefficients α_j = unspecified
    Equation 2 defines Φ with sensitivity coefficients α_j, but no values are given and no example computes Φ from raw violations.
axioms (5)
  • domain assumption Operational risk can be represented as a weighted sum of normalized monotonic metric mappings.
    The LSRI construction assumes that latency, throughput, size, cost, update frequency, and regulation can be combined additively into a single risk score; no empirical or theoretical support is given.
  • domain assumption The Table 3 thresholds reflect industrial deployment baselines.
    τ=100ms, λ=10^6, 20B parameters, $1000/day are asserted as representative for an app-store-scale deployment without source data.
  • domain assumption A zero integrity multiplier (Φ=0) should veto all other risk factors.
    The paper states a model cannot be 'safe' if it violates fundamental security boundaries; this is a design choice, not a derived property.
  • domain assumption A SHA-256/Sigstore signature on a checkpoint establishes a verifiable root of trust for that model artifact.
    Standard cryptographic integrity is assumed sufficient for the supply-chain claim, but it does not verify training-data provenance or semantic safety.
  • ad hoc to paper The performance numbers in Table 6 are accurate.
    The recall, latency, and false-positive values for static analyzers and LLM hybrids appear without a citation, dataset, benchmark, or error bars.
invented entities (1)
  • LLM Scalability Risk Index (LSRI) score no independent evidence
    purpose: Single parametric number intended to summarize LLM deployment readiness and security risk.
    No external validation against observed incidents or independent benchmarks is provided; the paper itself states it is not an empirically calibrated estimator.

pith-pipeline@v1.3.0-alltime-deepseek · 16547 in / 13095 out tokens · 123464 ms · 2026-08-02T21:45:48.696871+00:00 · methodology

0 comments
read the original abstract

As AI shifts from human-in-the-loop interfaces to autonomous multi-agent systems capable of real-time code execution and tool integration through protocols like the Model Context Protocol (MCP), traditional SAST, DAST, and legacy AI safety methods fail to detect modern agentic-AI threats. This paper introduces the LLM Scalability Risk Index (LSRI), a parametric framework and cybersecurity standard for stress-testing autonomous orchestration pipelines. LSRI measures the operational thresholds where load, compounding hallucinations, data poisoning, and adversarial prompt injections including jailbreaking and indirect prompt injection cause security boundaries to fail. Beyond RLHF and RLAIF, we present a Verifiable Root of Trust architecture using cryptographic attestation, semantic policy enforcement, and continuous runtime verification to secure the AI software supply chain. LSRI defends against malicious LoRA adapters, weight tampering, dependency typosquatting, and unsafe model artifacts from public registries such as Hugging Face and GitHub. By replacing post-hoc alignment with verifiable runtime controls, LSRI provides scalable API defense, safer agentic orchestration under heavy cloud workloads, stronger polymorphic malware detection, automated red-teaming, and improved system explainability. Aligned with NIST AI RMF, OWASP Top 10 for LLMs, and ISO 42001, LSRI establishes a deployable compliance baseline for securing generative AI ecosystems including ChatGPT, GPT-4o, Claude 3.5 Sonnet, Copilot, LLaMA, Gemini, and Bedrock. LSRI also supports capital market risk analysis by reducing exposure across technology portfolios linked to indices such as NASDAQ-100, QQQ, and semiconductor-driven supply chains.

Figures

Figures reproduced from arXiv: 2602.19021 by Kiarash Ahi, Saeed Valizadeh, Vaibhav Agrawal.

Figure 1
Figure 1. Figure 1: Categorization of explainability tools used in LLM￾driven cybersecurity systems. 7 [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 5
Figure 5. Figure 5: End-to-end LLM model-supply chain showing potential compromise points (data collection, pre-training, fine-tuning, checkpoint distribution, inference API) and recommended defenses such as dataset provenance audits, cryptographic signing, and weight attestation.61–65. 11 [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Proposed verifiable root-of-trust architecture for LLM supply-chain security. The architecture enforces pre-execution verification across the model lifecycle, binding build-time provenance to signed metadata and verifying model artifacts and dependencies during distribution and deployment to ensure fail-closed execution. 12 K. AHI ET AL [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

71 extracted references · 7 linked inside Pith

  1. [1]

    Gpt-4 technical report

    OpenAI. Gpt-4 technical report. arXiv:2303.08774. 2023

    Pith/arXiv arXiv 2023

  2. [2]

    Gemini overview

    Google AI Blog. Gemini overview. Google LLC; 2023

    2023

  3. [3]

    Introducing security copilot

    Microsoft. Introducing security copilot. Microsoft; 2023

    2023

  4. [4]

    Emerging security risks with AI

    Gartner. Emerging security risks with AI. Gartner Report; 2023

    2023

  5. [5]

    When LLMs meet cybersecurity: a systematic literature review

    Zhang J, Bu H, Wen H, Liu Y, Fei H, Xi R, Li L, Yang Y, Zhu H, Meng D. When LLMs meet cybersecurity: a systematic literature review. Cybersecurity. 2025 ;8 (1):55. doi: 10.1186/s42400-025-00361-w

    doi:10.1186/s42400-025-00361-w 2025

  6. [6]

    Global challenge for safe and secure LLMs track 1

    Jia X, Dong Y, Liu Z, Zhu J, Chen J, Su H. Global challenge for safe and secure LLMs track 1. arXiv:2411.14502. 2024

    Pith/arXiv arXiv 2024

  7. [7]

    Policy enforce - ment in app stores

    Zhang X, Wang X, Pang W, Zhang X. Policy enforce - ment in app stores. Ieee Tse. 2020 ;46:1005–1025

    2020

  8. [8]

    Fake review detection

    Jiang L, Sheng VS, Liu S, Xu J. Fake review detection. Acm Cikm. 2019 ;28:2657–2665

    2019

  9. [9]

    Dynamic analysis of malicious apps

    Narayanan S, Santhosh Kumar SVN, Dakshinamurthy J. Dynamic analysis of malicious apps. IEEE Access. 2023 ;11:21545–21558

    2023

  10. [10]

    Automated code generation security risks

    Pearce K, Ahmad B, Tan B, Dolan-Gavitt B, Karri R. Automated code generation security risks. IEEE Secur Privacy. 2022 ;20:30–40

    2022

  11. [11]

    Static analysis using LLMs

    Chen J, Huang J, Jeng JE. Static analysis using LLMs. IEEE Security & Privacy. 2023 ;21: 92–96

    2023

  12. [12]

    Risks of LLM data leakage

    Carlini N, Tramèr F, Wallace E, Jagielski M, Herbert- Voss A, Lee K, Roberts A, Brown T, Song D, Erlingsson Ú, et al. Risks of LLM data leakage. USENIX Secur. 2021 ;30:2633–2650

    2021

  13. [13]

    A survey on large language model (LLM) security and privacy: the good, the bad, and the ugly

    Yao Y, Duan J, Xu K, Cai Y, Sun Z, Zhang Y. A survey on large language model (LLM) security and privacy: the good, the bad, and the ugly. High-Confid Comput. 2024 ;2(2):100211. doi: 10.1016/j.hcc.2024.100211

    arXiv 2024

  14. [14]

    Exploring the dual role of LLMs in cybersecur - ity: threats and defenses

    Bryce C, Kalousis A, Leroux I, Madinier H, Pasche T, Ruch P. Exploring the dual role of LLMs in cybersecur - ity: threats and defenses. In: Vorobeychik Y, Kantarcioglu M, editors. Large language models in cybersecurity. Springer; 2024 . p. 235–242. doi: 10. 1007/978-3-031-54827-7_26

    2024

  15. [15]

    Artificial intelligence act

    European Commission. Artificial intelligence act. EU; 2023

    2023

  16. [16]

    Toward trustworthy AI development

    Brundage M, Avin S, Clark J, Toner H, Eckersley P, Garfinkel B, Dafoe A, Scharre P, Zeitzoff T, Filar B, et al. Toward trustworthy AI development. arXiv:2004.07213. 2020

    Pith/arXiv arXiv 2004

  17. [17]

    Auto DevSecOps powered by AI

    GitLab. Auto DevSecOps powered by AI. GitLab Docs; 2023

    2023

  18. [18]

    Secure development lifecycle

    Microsoft Azure. Secure development lifecycle. Azure Blog; 2023

    2023

  19. [19]

    Interim measures for the management of generative artificial intelligence services

    Cyberspace Administration of China. Interim measures for the management of generative artificial intelligence services. Beijing China; 2023

    2023

  20. [20]

    Government of Japan

    Cabinet Office. Government of Japan. AI strategy 2022. Tokyo Japan; 2022

    2022

  21. [21]

    National strategy for artificial intelligence

    Ministry of Science and ICT, Republic of Korea. National strategy for artificial intelligence. Seoul Korea; 2023

    2023

  22. [22]

    Model AI governance framework

    Infocomm Media Development Authority (IMDA) Singapore. Model AI governance framework. 2023

    2023

  23. [23]

    Advances and open pro - blems in federated learning

    Kairouz P, McMahan HB. Advances and open pro - blems in federated learning. Found Trends ML. 2021 ;14(1–2):1–210. doi: 10.1561/2200000083

    doi:10.1561/2200000083 2021

  24. [24]

    Towards federated learning at scale

    Bonawitz K, Eichner H, Grieskamp W, Huba D, Ingerman A, Ivanov V, Kiddon C, Konečný J, Mazzocchi S, McMahan B, et al. Towards federated learning at scale. SysML. 2019 ;1:1–5

    2019

  25. [25]

    Why should I trust you? Explaining the predictions of any classifier

    Ribeiro MT, Singh S, Guestrin. Why should I trust you? Explaining the predictions of any classifier. KDD. 2016 ;22: 1135–1144

    2016

  26. [26]

    Explainable artificial intelligence

    Samek W, Montavon G, Lapuschkin S, Anders CJ, Müller KR. Explainable artificial intelligence. Springer; 2021

    2021

  27. [27]

    Pi-bench: evaluating the robustness of large language models to prompt injection

    Zhu H, Liu J, Yi P, Zhao X. Pi-bench: evaluating the robustness of large language models to prompt injection. arXiv:2402.00349. 2024 . Table 7. Recommended immediate actions for key stakeholder groups to mitigate dual-use risks of LLMs. Stakeholder What They Must Do Now Governments Enact dual-use-specific AI policies & register frontier models Enterprises...

    Pith/arXiv arXiv 2024

  28. [28]

    Cybersecurity almanac: 100 facts, figures, predictions and statistics

    Cybersecurity Ventures. Cybersecurity almanac: 100 facts, figures, predictions and statistics. Cybercrime Magazine. 2024 June 26

    2024

  29. [29]

    Automated vulnerability detection

    Amazon AWS. Automated vulnerability detection. AWS; 2023

    2023

  30. [30]

    Watsonx security applications

    IBM Research. Watsonx security applications. IBM; 2023

    2023

  31. [31]

    Asleep at the keyboard? Assessing the security of GitHub Copilot’s code contributions

    Pearce K, Ahmad B, Tan B, Dolan-Gavitt B, Karri R. Asleep at the keyboard? Assessing the security of GitHub Copilot’s code contributions. Proc IEEE Symp Security and Privacy. 2022;43:754–768

    2022

  32. [32]

    Security implications of AI-generated code

    Sandoval S, Pearce H, Nys T, Mansouri M, Dolan- Gavitt B. Security implications of AI-generated code. Proc ACM Int Conf Foundations of Software Engineering (FSE); 2024, Vol. 32, p. 2205–2222

    2024

  33. [33]

    Code contamination: legal and security risks of LLM code generation

    Jagielski M, Nasr M, Choquette-Choo CA, Lee K, Carlini N, Tramèr F. Code contamination: legal and security risks of LLM code generation. arXiv:2401.05078. 2024

    Pith/arXiv arXiv 2024

  34. [34]

    Cybersecurity almanac: 2024 edition

    Cybersecurity Ventures. Cybersecurity almanac: 2024 edition. Cybersecurity Ventures; 2024 Jan 2

    2024

  35. [35]

    Combating WormGPT: What you need to know

    Abnormal Security. Combating WormGPT: What you need to know. 2025

    2025

  36. [36]

    What is WormGPT? 2025

    Lookout. What is WormGPT? 2025

    2025

  37. [37]

    Cybernews

    Ferrari executive nearly fooled by AI voice clone of CEO. Cybernews. 2023 Nov

    2023

  38. [38]

    Forewarned is forearmed: a survey on large language model-based agents in autonomous cyberattacks

    Xu M, Fan J, Huang X, Zhou Z, Kang J, Niyato D, Mao S, Han Z, Shen X, Lam K. Forewarned is forearmed: a survey on large language model-based agents in autonomous cyberattacks. arXiv:2505.12786. 2025

    Pith/arXiv arXiv 2025

  39. [39]

    On the feasibility of using LLMs to autonomously execute multi-host network attacks

    Singer B, Lucas K, Adiga L, Jain M, Bauer L, Sekar V. On the feasibility of using LLMs to autonomously execute multi-host network attacks. arXiv:2501.16466. 2025

    arXiv 2025

  40. [40]

    AI risk management framework

    NIST. AI risk management framework. NIST Special Publication; 2023

    2023

  41. [41]

    Global AI security standards

    World Economic Forum. Global AI security standards. WEF; 2023

    2023

  42. [42]

    Safe framework implementation

    Google Security Blog. Safe framework implementation. Google; 2023

    2023

  43. [43]

    Operationalizing large language models for cybersecurity: Infrastructure, scalability, and perfor - mance benchmarks

    Zhang Y. Operationalizing large language models for cybersecurity: Infrastructure, scalability, and perfor - mance benchmarks. J Cyber Inf Syst (JCIS). 2025;15 (1):102–118

    2025

  44. [44]

    Fairness and safety of LLMs

    Palo Alto Networks. Fairness and safety of LLMs. 2024 June

    2024

  45. [45]

    Data privacy and compliance for large language models (LLMs)

    Mohindroo S. Data privacy and compliance for large language models (LLMs). Medium. 2024 Sep

    2024

  46. [46]

    What is large language model (LLM) security

    Qualys. What is large language model (LLM) security. 2025 Apr

    2025

  47. [47]

    A framework for trustworthy AI in cybersecurity operations

    Rajkomar A, Anthi E, Burnap P. A framework for trustworthy AI in cybersecurity operations. IEEE Commun Surv Tutorials. 2024;26:412–438

    2024

  48. [48]

    CrowdStrike vs Palo Alto: 5 key differences and pros & cons

    Cynet. CrowdStrike vs Palo Alto: 5 key differences and pros & cons. Cynet blog. 2025

    2025

  49. [49]

    Artificial intelligence act final text

    European Union. Artificial intelligence act final text. Official Journal of the EU OJ L. 2025:55–56

    2025

  50. [50]

    Owasp top 10 for large language model applications

    OWASP Foundation. Owasp top 10 for large language model applications. OWASP; 2025

    2025

  51. [51]

    General Data Protection Regulation (GDPR); 2016

    European Union. General Data Protection Regulation (GDPR); 2016. Publications Office of the European Union

    2016

  52. [52]

    California consumer privacy act (CCPA)

    State of California Department of Justice. California consumer privacy act (CCPA). 2018

    2018

  53. [53]

    Explainable AI in cybersecurity: bridging trans - parency and trust

    Silva J. Explainable AI in cybersecurity: bridging trans - parency and trust. Proc IEEE Conf Cybersecurity Innovations; Lisbon, Portugal; 2025. p. 78–83

    2025

  54. [54]

    CyberMentor: enhancing cybersecurity learning through explainable AI

    Wang F, Zhao L, Chen M. CyberMentor: enhancing cybersecurity learning through explainable AI. Proc IEEE Int Conf Emerging Trends in Cyber Training Zhengzhou, China; 2025. p. 102–107

    2025

  55. [55]

    Ethical auditing in AI: the role of model cards and the cyber kill chain

    Barrett R, Lee S, Harmon T. Ethical auditing in AI: the role of model cards and the cyber kill chain. IEEE Trans Technol Soc. 2023;10(2):123–132

    2023

  56. [56]

    A framework for ethical AI compliance under the EU AI Act

    Gupta P, Sharma N, Desai K. A framework for ethical AI compliance under the EU AI Act. Proc IEEE Workshop on AI Governance Washington, DC, USA

  57. [57]

    Cloud CISO perspectives: Our big sleep agent makes big leap

    Google Cloud. Cloud CISO perspectives: Our big sleep agent makes big leap. Google cloud blog. 2024

    2024

  58. [58]

    Benchmarking LLMs for zero-day vulnerabilities

    Lisha M, Agarwal V, Kamthania S, Vutkur P, Chari M. Benchmarking LLMs for zero-day vulnerabilities. Proc IEEE International Conference on Electronics, Computing and Communication Technologies (CONECCT); Bengaluru, India; 2024. p. 95–102

    2024

  59. [59]

    The curse of recursion: training on generated data makes models forget

    Shumailov I, Shumaylov Z, Zhao Y, Gal Y, Papernot N. The curse of recursion: training on generated data makes models forget. arXiv:2305.17493. 2023

    Pith/arXiv arXiv 2023

  60. [60]

    A watermark for large language models

    Kirchenbauer J, Geiping J, Wen Y, Katz J, Miers I, Goldstein T. A watermark for large language models

  61. [61]

    Instruction backdoor attacks against customized LLMs

    Zhang R, Li H, Wen R, Jiang W, Zhang Y, Backes M, Shen Y, Zhang Y. Instruction backdoor attacks against customized LLMs. USENIX Secur. 2024; 33:3085–3102

    2024

  62. [62]

    deberta-v3-base-prompt-injection-v2

    Protect AI. deberta-v3-base-prompt-injection-v2. Hugging Face; 2024

    2024

  63. [63]

    Red teaming network report: findings from phase I

    OpenAI. Red teaming network report: findings from phase I. OpenAi. 2024 Jul

    2024

  64. [64]

    Privacy side channels in machine learning systems

    Debenedetti E, Severi J, Carlini N, Choquette-Choo CA, Jagielski M, Nasr M, Wallace E, Tramèr F. Privacy side channels in machine learning systems. Proc USENIX Security; 2024:33:3121–3138

    2024

  65. [65]

    Dataset and model supply chain security for foundation models

    Goldblum J, Fowl L, Goldblum M, Goldstein T. Dataset and model supply chain security for foundation models. Proc NeurIPS; 2024;37:12431–12445

    2024

  66. [66]

    Sigstore: software signing for everybody

    Dodds L, Torres-Arias S, Newman Z, Moore M, Kuppusamy TK. Sigstore: software signing for everybody. Proc USENIX Security Symposium. 2022;31:1827–1844

    2022

  67. [67]

    Stealing and misusing machine learning models

    Tramèr F, Zhang F, Juels A, Reiter MK, Ristenpart T. Stealing and misusing machine learning models. Proc USENIX Security Symposium; Austin, Texas, USA; 2016:25:601–618

    2016

  68. [68]

    Dataset security for machine learning

    Goldblum M, Tsipras D, Xie C, Chen X, Schwarzschild A, Song D, Madry A, Li B, Goldstein T. Dataset security for machine learning. Proc NeurIPS. 2020:33:14781– 14792

    2020

  69. [69]

    Rethinking deep neural net - work ownership verification

    Fan L, Ali K, Atallah MJ. Rethinking deep neural net - work ownership verification. Proc NDSS; San Diego, California, USA; 2019

    2019

  70. [70]

    Guidelines for secure use of large language models

    UK National Cyber Security Centre. Guidelines for secure use of large language models. NCSC; 2024. 17

    2024

  71. [2023]

    11561–11575

    p. 11561–11575. In: Proc Int Conf on Machine Learning (ICML)