Pith. sign in

REVIEW 4 major objections 4 minor 56 references

Machine Learning Models Have a Supply Chain Problem

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that Sigstore-based model signing and a zero-knowledge set over training data give open ML a practical supply-chain fix.

desk verdict Useful model-signing work; dataset-commitment part has a backwards verification branch and a conceded gap between commitment and actual training data. read the letter →

arxiv 2505.22778 v1 pith:OOTDXRKF submitted 2025-05-28 cs.LG cs.CR

Sarah Meiklejohn , Hayden Blauzvern , Mihai Maruseac , Spencer Schrock , Laurent Simon , Ilia Shumailov This is my paper
classification cs.LGcs.CR
keywords MLsupplychainsecuritymodelsigningSigstoreRekortransparencylogzero-knowledgesetstrainingdataprovenanceMerkletreecommitmentsverifiablerandomfunctions
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's central claim is that the open ecosystem for downloadable machine-learning models carries real, already-exploited supply-chain risks—malware-laden or tampered models, poisoned training data, and impersonated publishers—and that two cryptographic mechanisms can make model publication verifiable at acceptable cost. The first is Sigstore-based signing: publishers sign model hashes with short-lived certificates tied to their identity, and the signing metadata is recorded in a public transparency log so tampering or differentiated distribution can be detected. The second is a zero-knowledge set over the training data: a trainer commits to the data once and can later prove, without revealing the rest of the set, that a specific data point was or was not included. The paper reports concrete costs—hashing a 1 TB model takes about 800 seconds, committing a 100-million-entry registry takes 96 seconds—as evidence that these steps are practical enough for hubs to adopt. A sympathetic reader would take away that transparency for open models is achievable today with standard cryptographic building blocks, provided the trainer's commitment is made honestly.

What carries the argument

The load-bearing object for integrity is the Sigstore bundle: an ephemeral public key, a certificate binding that key to an OIDC identity, and a signature plus a Merkle inclusion proof in the public Rekor transparency log, turning 'this is the publisher's intended model' into evidence any user can check and any outsider can audit. The load-bearing object for provenance is a zero-knowledge set built from a verifiable random function and a Patricia Merkle tree: the VRF pseudo-randomly permutes entries so the tree's structure reveals no ordering, and leaf values are hiding commitments, so membership and non-membership proofs release nothing beyond the queried point.

What would settle it

Build a 1 TB model file on a comparable server, run the released hashing library, and compare the 800-second figure and the roughly 19 percent overhead against saving the file; a materially larger overhead would falsify the practicality claim. For the commitment scheme, a 100-million-entry registry that takes substantially longer than 96 seconds to commit on ordinary hardware—or proofs that take more than the reported tens of microseconds—would falsify the scaling claim.

Watch

Extended reading notes

Core claim

The paper's discovery is that two existing, standardized cryptographic tools can be reassembled to cover the two weakest points of the open-model supply chain. For integrity, it adapts Sigstore's workflow to models: a publisher authenticates to an identity provider, obtains a short-lived certificate binding an ephemeral key to that identity, signs the hash of the model, and submits the signature, certificate, and hash to the Rekor transparency log, returning a Merkle inclusion proof to the consumer. For data provenance, it constructs a zero-knowledge set: entries (individual training points or whole datasets) are placed at pseudorandom positions determined by a verifiable random function, a Patricia Merkle tree is built over those positions, and leaf values are stored as hiding commitments, so a prover can prove membership or non-membership of a queried point while learning nothing else. The paper measures both mechanisms on realistic scales and reports that hashing dominates model signing—800 seconds for a 1 TB file against 4,151 seconds to save it—and that commitments to 100 million entries take 96 seconds, with individual proofs around tens of microseconds. It is explicit that these proofs are only as meaningful as the commitment itself: nothing in the construction ties the committed set to the data actually used in training.

Load-bearing premise

The whole data-provenance mechanism rests on the trainer forming the commitment over data actually used in training; a trainer who commits to a different set makes every membership proof vacuous, and the paper leaves verifying that step to future work.

Editorial extensions

If this is right

  • Model users can verify that a downloaded file is the exact version the publisher signed, and a publisher who watches the transparency log can detect tampered or targeted copies.
  • Model hubs can integrate signing into their upload API with negligible extra code, making verification available by default without changing how models are trained or stored.
  • A trainer can publish one compact commitment at release time and later prove, without revealing the dataset, whether a specific document or image was part of training.
  • The measured overhead is small relative to model production: hashing a 1 TB model takes about 800 seconds, and committing to 100 million training entries takes 96 seconds.
  • Non-inclusion proofs make it possible to answer copyright and regulatory queries about whether particular data was used, even when the full dataset is private.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If hubs adopt signing as the default upload path, the transparency log becomes a natural place to also record dataset commitments, letting users cross-check which model version was trained against which committed data registry without building new infrastructure.
  • The paper's commitment benchmarks stop at 100 million in-memory entries; a natural test is to run the same construction over a multi-billion-document web-scale corpus with persistent storage, where the paper only extrapolates.
  • The same VRF-plus-Merkle machinery could eventually back queries in agentic systems—proving which tool, document, or retrieved context drove a given action—though the paper does not pursue that direction.
  • A useful next experiment is to measure how often real consumers query non-inclusion proofs; if demand is rare, the limiting factor for transparency will be the honesty of the initial commitment, not proof cost.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper argues that the open ML model ecosystem faces significant supply-chain risks, including model tampering, malicious uploads, and unverifiable training-data claims. It proposes two interventions: first, signing model artifacts with Sigstore and storing signing metadata in the Rekor transparency log, with an implementation and benchmarks showing hashing-dominated costs (e.g., 800 s for a 1 TB model); second, adapting a zero-knowledge set construction (inspired by SEEMless and Parakeet) to let a model trainer commit to a list of data elements and later prove (non-)inclusion of specific points, with benchmarks showing commit times of 96 s for 100 M entries. The paper explicitly acknowledges in Section 6.4 that these proofs are meaningful only if the commitment was computed over the actual training data, which it leaves to future work.

Significance. The model-signing contribution is practical and clearly presented; the open-source library, integration with Sigstore, and measurements of large-model hashing costs are useful engineering results. The zero-knowledge set is a sensible adaptation of known key-transparency techniques, and the benchmark data are informative. The paper's central data-provenance claim, however, overreaches the construction: without a mechanism tying the commitment to the real training run, (non-)inclusion proofs only attest to a trainer-chosen set. The work is therefore best read as a building block plus a deployment-oriented first step, not as a complete solution to verifiable dataset provenance.

major comments (4)
  1. [Figure 3, ZKS.Verify] The conditional in ZKS.Verify is reversed. When resp=0 (non-inclusion), the algorithm calls Acc.VerIncl, and when resp=1 (inclusion) it calls Acc.VerNonIncl, whereas ZKS.Query returns resp=1 with Acc.ProveIncl and resp=0 with Acc.ProveNonIncl. As written, an honest proof will always fail verification (or a dishonest one may pass). This should be corrected to use Acc.VerIncl when resp=1 and Acc.VerNonIncl when resp=0.
  2. [Figure 3, ZKS.Query and ZKS.Verify] There is an index mismatch. Section 6.2 states that the accumulator index for an entry D is H(VRF.Eval(sk,D)), and ZKS.Commit computes di = H(VRF.Eval(sk,Di)). However, ZKS.Query sets d = VRF.Eval(sk,D) and uses d directly as the accumulator position, while ZKS.Verify passes that same d to Acc.VerIncl/VerNonIncl. The proof πVRF is over d, but it is not linked to the hashed value used to place entries in the tree. The pseudocode should separate the VRF output v from the derived index d = H(v), and ZKS.Verify should recompute d from the verified v before checking the accumulator proof.
  3. [Section 6.4 / Abstract] The abstract claims the work enables publishers to 'prove properties about the datasets they use,' but the construction only proves (non-)inclusion with respect to a commitment that the trainer chooses. Section 6.4 concedes this: 'the proofs provided against this commitment become meaningful only when we can be sure the commitment was computed correctly.' This is not a technical flaw in the zero-knowledge set itself, but the presentation should temper the data-provenance claim or explicitly frame the ZKS as a primitive that must be combined with a separate attestation mechanism (e.g., proofs of training, TEEs) before it delivers the advertised transparency benefit.
  4. [Section 6.2] The claim that the simplification of the SEEMless construction 'achiev[es] the same soundness and privacy guarantees' is not accompanied by a proof or a precise statement of the security model, even though the paper relies on external results for soundness. Since the construction modifies leaf storage and ordering compared to a standard accumulator, the privacy claim should either be formally argued or explicitly attributed to the SEEMless proof with a careful reduction.
minor comments (4)
  1. [Section 5.4] There is a typo: 'ranging from 1 Bto 1 TB' lacks a space; also, 'the time required to hash a file of a size ranging from 1 B to 1 TB' could be clearer.
  2. [Section 6.2] In the sentence 'for entry D we compute its new index j as H(VRF.Eval(sk, Di))', the subscript 'i' appears in a context where the element is D, not Di; this is confusing given that Di denotes the i-th training element.
  3. [Section 6.4] The phrase 'a non-private proof of training data to an entity who is trusted to see the data but not collude with the model trainer' is a sentence fragment; please rephrase for clarity.
  4. [References] The reference for the AI Act is incomplete (no authors or publication venue); if it is a legislative document, a stable URL and date would be helpful.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper's contributions are engineering adaptations of external, published cryptographic primitives, with empirical benchmarks and an explicitly acknowledged limitation about commitment correctness.

full rationale

The paper's two proposed mechanisms (model signing via Sigstore and a zero-knowledge set for training-data commitment) are built on externally published primitives: Sigstore (Newman et al. 2022), the Melara et al. (2015) VRF, the Chase et al. (2019) SEEMless zero-knowledge set, and the Parakeet verifiable registry (Malvai et al. 2023). The soundness of the underlying accumulator construction is attributed to Chase et al. rather than derived in this paper, so there is no equation in the paper that is equivalent to its own inputs by construction. The performance results are empirical measurements of hashing, signing, and commitment times, not parameters fitted to a predetermined conclusion. Self-citations appear only in the threat-model survey (e.g., Shumailov et al. 2021, Gao et al. 2025, Langford et al. 2025, Tan et al. 2025) and are not load-bearing for either construction. Section 6.4 explicitly acknowledges that the commitment does not by itself prove that it corresponds to the actual training data: 'the proofs provided against this commitment become meaningful only when we can be sure the commitment was computed correctly.' This is a candid limitation of the proposal's applicability, not a circular derivation; the paper correctly identifies the gap and defers solutions to future work. Thus the central claims have independent content and the analysis yields no circular step.

Assumptions & free parameters 1 free parameters · 6 assumptions · 1 invented entities

The core cryptographic claims rest on standard assumptions (DDH, discrete logarithm) and on the security of published primitives. The main unproven domain assumption is that the trainer forms the commitment correctly, which the paper itself leaves open. The only hand-chosen numeric input is the hashing chunk size, which does not affect correctness of the cryptographic guarantees.

free parameters (1)
  • chunk size for list-based hashing = 1 GB
    Default chunk size for the optimized hashing approach; chosen by hand as a performance trade-off, not fitted to a target outcome.
assumptions (6)
  • standard math Discrete logarithm assumption
    Stated in Section 5.1 as a basis for the signature and VRF schemes.
  • standard math DDH assumption
    Stated in Section 5.1 and used for the security of the Melara et al. VRF.
  • standard math Soundness of the Patricia Merkle tree accumulator
    The ZKS relies on the accumulated tree's soundness, which is proven in Chase et al. (2019) and cited in Section 6.2.
  • standard math Security of the Melara et al. VRF
    Assumed from prior work, cited in Section 6.1.2, including pseudorandomness and unique provability.
  • domain assumption The model trainer honestly commits to the actual training data
    The dataset verifiability feature only proves membership in the committed set; Section 6.4 acknowledges that ensuring the commitment matches real training data is open future work.
  • domain assumption Model hubs can act as or integrate with OIDC identity providers
    The model signing workflow in Section 5.3 assumes the publisher can obtain an OIDC token, which requires hub cooperation.
invented entities (1)
  • none
    purpose: The paper introduces no new particle, force, dimension, or other physical entity; it proposes software mechanisms built from existing cryptographic objects.
    No invented entities appear in the paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Machine Learning Models Have a Supply Chain Problem." pith.science (2026). https://pith.science/paper/OOTDXRKF

@misc{pith2026250522778,
  author       = {Pith},
  title        = {Pith review of: Machine Learning Models Have a Supply Chain Problem},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OOTDXRKF}},
  note         = {Machine review of arXiv:2505.22778}
}
read the original abstract

Powerful machine learning (ML) models are now readily available online, which creates exciting possibilities for users who lack the deep technical expertise or substantial computing resources needed to develop them. On the other hand, this type of open ecosystem comes with many risks. In this paper, we argue that the current ecosystem for open ML models contains significant supply-chain risks, some of which have been exploited already in real attacks. These include an attacker replacing a model with something malicious (e.g., malware), or a model being trained using a vulnerable version of a framework or on restricted or poisoned data. We then explore how Sigstore, a solution designed to bring transparency to open-source software supply chains, can be used to bring transparency to open ML models, in terms of enabling model publishers to sign their models and prove properties about the datasets they use.

Figures

Figures reproduced from arXiv: 2505.22778 by the authors.

Figure 1
Figure 1. The different actors and phases in the software sup￾ply chain, along with the points at which malicious actions can be taken. This image was taken from https://slsa.dev/ spec/v1.0/threats-overview. A recent line of work has sought to overcome this limita￾tion by using zero-knowledge proofs to prove the correct￾ness of training without revealing the training data. Garg et al. (2023) and Tan et al. (2025) provide proo… view at source ↗
Figure 2
Figure 2. Averaged over five runs and plotted on a log-log scale, the time, in seconds, required to hash a file of a size ranging from 1 B to 1 TB, on three different machines and using our two dif￾ferent approaches. The green dots represent the time to hash dif￾ferent large open models using the list-based approach on M3, as summarized in [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Algorithms for our zero-knowledge set, assuming an underlying accumulator Acc and VRF VRF. 10 3 10 4 10 5 10 6 10 7 10 8 Number of data points 10 5 10 4 10 3 10 2 10 1 10 0 10 1 10 2 Runtime (in seconds) Commit Prove Verify [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Averaged over ten runs and plotted on a log-log scale, the time, in seconds, to commit to and prove and verify inclusion in a data registry of a given size, ranging from 1000 to 100 million entries. but not collude with the model trainer (e.g., a regulator), or publish…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

56 extracted references · 56 canonical work pages

  1. [1]

    Zero-knowledge proofs of training for deep neural networks

    Abbaszadeh, K., Pappas, C., Papadopoulos, D., and Katz, J. Zero-knowledge proofs of training for deep neural networks. In Proceedings of ACM CCS, 2024

    work page 2024

  2. [2]

    Croissant: A metadata format for ML -ready datasets

    Akhtar, M., Benjelloun, O., Conforti, C., Foschini, L., Gijsbers, P., Giner-Miguelez, J., Goswami, S., Jain, N., Karamousadakis, M., Krishna, S., Kuchnik, M., Lesage, S., Lhoest, Q., Marcenac, P., Maskey, M., Mattson, P., Oala, L., Oderinwale, H., Ruyssen, P., Santos, T., Shinde, R., Simperl, E., Suresh, A., Thomas, G., Tykhonov, S., Vanschoren, J., Varma...

    work page 2024

  3. [3]

    and Shmatikov, V

    Bagdasaryan, E. and Shmatikov, V. Blind backdoors in deep learning models. In Proceedings of the 30th USENIX Security Symposium, 2021

    work page 2021

  4. [4]

    and Shmatikov, V

    Bagdasaryan, E. and Shmatikov, V. Spinning language models: Risks of propaganda-as-a-service and countermeasures. In Proceedings of the IEEE Symposium on Security & Privacy, 2022

    work page 2022

  5. [5]

    Unforgeability in stochastic gradient descent

    Baluta, T., Nikolić, I., Jain, R., Aggarwal, D., and Saxena, P. Unforgeability in stochastic gradient descent. In Proceedings of ACM CCS, 2023

    work page 2023

  6. [6]

    Bytedance intern fired for planting malicious code in AI models, October 2024

    Belanger, A. Bytedance intern fired for planting malicious code in AI models, October 2024. https://arstechnica.com/tech-policy/2024/10/bytedance-intern-fired-for-planting-malicious-code-in-ai-models/

    work page 2024

  7. [7]

    K., Suriyawongkul, A., and Stewart, K

    Bennet, K., Rajbahadur, G. K., Suriyawongkul, A., and Stewart, K. Implementing AI bill of materials ( AI BOM ) with SPDX 3.0, October 2024. https://www.linuxfoundation.org/hubfs/LF\

    work page 2024

  8. [8]

    Biden, J. R. Executive Order on the safe, secure, and trustworthy development and use of artificial intelligence, October 2023. https://www.whitehouse.gov/briefing-room/presidential-actions/2023/10/30/executive-order-on-the-safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence/

    work page 2023

Show all 56 references
  1. [9]

    The foundation model transparency index v1.1, 2024

    Bommasani, R., Klyman, K., Kapoor, S., Longpre, S., Xiong, B., Maslej, N., and Liang, P. The foundation model transparency index v1.1, 2024. URL https://crfm.stanford.edu/fmti/paper.pdf

    2024

  2. [10]

    A., Paleka, D., Pearce, W., Anderson, H., Terzis, A., Thomas, K., and Tramèr, F

    Carlini, N., Jagielski, M., Choquette-Choo, C. A., Paleka, D., Pearce, W., Anderson, H., Terzis, A., Thomas, K., and Tramèr, F. Poisoning web-scale training datasets is practical. In Proceedings of the IEEE Symposium on Security & Privacy, 2024

    2024

  3. [11]

    H., Hopkins, A., Ilyas, A., Madry, A., Struckman, I., and Videgaray, L

    Cen, S. H., Hopkins, A., Ilyas, A., Madry, A., Struckman, I., and Videgaray, L. AI supply chains (and why they matter), April 2023. https://aipolicy.substack.com/p/supply-chains-2

    2023

  4. [12]

    SEEMless : Secure end-to-end encrypted messaging with less trust

    Chase, M., Deshpande, A., Ghosh, E., and Malvai, H. SEEMless : Secure end-to-end encrypted messaging with less trust. In Proceedings of ACM CCS, pp.\ 1639--1656, 2019

    2019

  5. [13]

    Tools for verifying neural models' training data

    Choi, D., Shavit, Y., and Duvenaud, D. Tools for verifying neural models' training data. In Proceedings of the 37th Conference on Neural Information Processing Systems (NeurIPS 2023), 2023

    2023

  6. [14]

    Data scientists targeted by malicious Hugging Face ML models with silent backdoor, February 2024

    Cohen, D. Data scientists targeted by malicious Hugging Face ML models with silent backdoor, February 2024. https://jfrog.com/blog/data-scientists-targeted-by-malicious-hugging-face-ml-models-with-silent-backdoor/

    2024

  7. [15]

    A new bill wants to reveal what's really inside AI training data, April 2024

    David, E. A new bill wants to reveal what's really inside AI training data, April 2024. https://www.theverge.com/2024/4/10/24126382/copyright-ai-bill-congress-schiff-training-data

    2024

  8. [16]

    Verifiable and provably secure machine unlearning

    Eisenhofer, T., Riepel, D., Chandrasekaran, V., Ghosh, E., Ohrimenko, O., and Papernot, N. Verifiable and provably secure machine unlearning. In Proceedings of the 3rd IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), 2025

    2025

  9. [17]

    Supply-chain attacks in machine learning frameworks

    Gao, Y., Shumailov, I., and Fawaz, K. Supply-chain attacks in machine learning frameworks. In Proceedings of the 8th MLSys Conference, 2025

    2025

  10. [18]

    Experimenting with zero-knowledge proofs of training

    Garg, S., Goel, A., Jha, S., Mahloujifar, S., Mahmoody, M., Policharla, G.-V., and Wang, M. Experimenting with zero-knowledge proofs of training. In Proceedings of ACM CCS, 2023

    2023

  11. [19]

    W., Wallach, H., Daum \'e III , H., and Crawford, K

    Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daum \'e III , H., and Crawford, K. Datasheets for datasets. Communications of the ACM, 64: 0 86--92, 2021

    2021

  12. [20]

    and Mac, R

    Grynbaum, M. and Mac, R. The Times Sues OpenAI and Microsoft Over A.I. Use of Copyrighted Work , December 2023. https://www.nytimes.com/2023/12/27/business/media/new-york-times-open-ai-microsoft-lawsuit.html

    2023

  13. [21]

    Badnets: Identifying vulnerabilities in the machine learning model supply chain

    Gu, T., Dolet-Gavitt, B., and Garg, S. Badnets: Identifying vulnerabilities in the machine learning model supply chain. In Proceedings of the Machine Learning and Computer Security Workshop, 2017

    2017

  14. [22]

    Hijacking Safetensors conversion on Hugging Face , February 2024

    Hidden Layer . Hijacking Safetensors conversion on Hugging Face , February 2024. https://hiddenlayer.com/innovation-hub/silent-sabotage/

    2024

  15. [23]

    Ho, K. K. Database of 16,000 Artists Used to Train Midjourney AI, Including 6-Year-Old Child, Garners Criticism , January 2024. https://www.artnews.com/art-news/news/midjourney-ai-artists-database-1234691955/

    2024

  16. [24]

    and Hardouin, J

    Huynh, D. and Hardouin, J. PoisonGPT : How we hid a lobotomized LLM on Hugging Face to spread fake news, July 2023. https://blog.mithrilsecurity.io/poisongpt-how-we-hid-a-lobotomized-llm-on-hugging-face-to-spread-fake-news/

    2023

  17. [25]

    MCP security notification: Tool poisoning attacks, April 2025

    Invariant Labs . MCP security notification: Tool poisoning attacks, April 2025. https://invariantlabs.ai/blog/mcp-security-notification-tool-poisoning-attacks

    2025

  18. [26]

    R., Sethi, R., Lu, Y.-H., Thiruvathukal, G

    Jiang, W., Synovic, N., Hyatt, M., Schorlemmer, T. R., Sethi, R., Lu, Y.-H., Thiruvathukal, G. K., and Davis, J. C. An empirical study of pre-trained model reuse in the Hugging Face deep learning model registry. In Proceedings of the 45th International Conference on Software E...

    2023

  19. [27]

    Unveiling AI / ML supply chain attacks: Name squatting organizations on Hugging Face , March 2024

    Kiani, M. Unveiling AI / ML supply chain attacks: Name squatting organizations on Hugging Face , March 2024. https://protectai.com/threat-research/unveiling-ai-supply-chain-attacks-on-hugging-face

    2024

  20. [28]

    Weight poisoning attacks on pretrained models

    Kurita, K., Michel, P., and Neubig, G. Weight poisoning attacks on pretrained models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), 2020

    2020

  21. [29]

    Architectural neural backdoors from first principles

    Langford, H., Shumailov, I., Zhao, Y., Mullins, R., and Papernot, N. Architectural neural backdoors from first principles. In Proceedings of the IEEE Symposium on Security & Privacy, 2025

    2025

  22. [30]

    +1500 Hugging Face API tokens were exposed, leaving millions of Meta - Llama , Bloom , and Pythia users vulnerable, December 2023

    Lanyado, B. +1500 Hugging Face API tokens were exposed, leaving millions of Meta - Llama , Bloom , and Pythia users vulnerable, December 2023. https://www.lasso.security/blog/1500-huggingface-api-tokens-were-exposed-leaving-millions-of-meta-llama-bloom-and-pythia-users-for-sup...

    2023

  23. [31]

    and Lewi, K

    Lawlor, S. and Lewi, K. Deploying key transparency at WhatsApp , April 2023. https://engineering.fb.com/2023/04/13/security/whatsapp-key-transparency/

    2023

  24. [32]

    Data authenticity, consent, & provenance for AI are all broken: what will it take to fix them? In Proceedings of the 41st International Conference on Machine Learning (ICML), 2024

    Longpre, S., Mahari, R., Obeng-Marnu, N., Brannon, W., South, T., Gero, K., Pentland, S., and Kabbara, J. Data authenticity, consent, & provenance for AI are all broken: what will it take to fix them? In Proceedings of the 41st International Conference on Machine Learning (ICML), 2024

    2024

  25. [33]

    S., Corry, F., Sridharan, H., Ananny, M., Schultz, J., and Crawford, K

    Luccioni, A. S., Corry, F., Sridharan, H., Ananny, M., Schultz, J., and Crawford, K. A framework for deprecating datasets: Standardizing documentation, identification, and communication. In Proceedings of the ACM Conference on Fairness, Accountability, and Transparency (FAccT), 2022

    2022

  26. [34]

    Parakeet: Practical key transparency for end-to-end encrypted messaging

    Malvai, H., Kokoris-Kogias, L., Sonnino, A., Ghosh, E., Ozt \"u rk, E., Lewi, K., and Lawlor, S. Parakeet: Practical key transparency for end-to-end encrypted messaging. In Proceedings of NDSS, 2023

    2023

  27. [35]

    Model files are invisible viruses, January 2024

    McInerney, D. Model files are invisible viruses, January 2024. https://protectai.com/threat-research/model-files-are-invisible-viruses

    2024

  28. [36]

    S., Blankstein, A., Bonneau, J., Felten, E

    Melara, M. S., Blankstein, A., Bonneau, J., Felten, E. W., and Freedman, M. J. CONIKS : Bringing key transparency to end users. In Proceedings of the 24th USENIX Security Symposium, pp.\ 383--398, 2015

    2015

  29. [37]

    Exploiting ML models with pickle file attacks: Part 1, June 2024

    Milanov, B. Exploiting ML models with pickle file attacks: Part 1, June 2024. https://blog.trailofbits.com/2024/06/11/exploiting-ml-models-with-pickle-file-attacks-part-1/

    2024

  30. [38]

    D., and Gebru, T

    Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., and Gebru, T. Model cards for model reporting. In Proceedings of the ACM Conference on Fairness, Accountability, and Transparency (FAccT), 2019

    2019

  31. [39]

    and Kapoor, S

    Narayanan, A. and Kapoor, S. Generative AI companies must publish transparency reports, June 2023. https://knightcolumbia.org/blog/generative-ai-companies-must-publish-transparency-reports

    2023

  32. [40]

    Newman, L. H. GitHub moves to guard open source against supply chain attacks, August 2022. https://www.wired.com/story/github-code-signing-sigstore/

    2022

  33. [41]

    S., and Torres-Arias, S

    Newman, Z., Meyers, J. S., and Torres-Arias, S. Sigstore: Software signing for everybody. In Proceedings of ACM CCS, 2022

    2022

  34. [42]

    Data cards: Purposeful and transparent dataset documentation for responsible AI

    Pushkarna, M., Zaldivar, A., and Kjartansson, O. Data cards: Purposeful and transparent dataset documentation for responsible AI . In Proceedings of the ACM Conference on Fairness, Accountability, and Transparency (FAccT), 2022

    2022

  35. [43]

    R., Kalu, K

    Schorlemmer, T. R., Kalu, K. G., Chigges, L., Ko, K. M., Ishgair, E. A.-M. A., Bagchi, S., Torres-Arias, S., and Davis, J. C. Signing in four public software package registries: Quantity, quality, and influencing factors. In Proceedings of the IEEE Symposium on Security & Priv...

    2024

  36. [44]

    S., Tan, G., Cebere, T

    Shamsabadi, A. S., Tan, G., Cebere, T. I., Bellet, A., Haddadi, H., Papernot, N., Wang, X., and Weller, A. Confidential- DPproof : Confidential proof of differentially private training. In Proceedings of the International Conference on Learning Representations (ICLR), 2024

    2024

  37. [45]

    A., and Anderson, R

    Shumailov, I., Shumaylov, Z., Kazhdan, D., Zhao, Y., Papernot, N., Erdogdu, M. A., and Anderson, R. Manipulating SGD with data ordering attacks. In Proceedings of the 35th Conference on Neural Information Processing Systems (NeurIPS 2021), 2021

    2021

  38. [46]

    Playing with fire – how we executed a critical supply chain attack on PyTorch , January 2024

    Stawinski, J. Playing with fire – how we executed a critical supply chain attack on PyTorch , January 2024. https://johnstawinski.com/2024/01/11/playing-with-fire-how-we-executed-a-critical-supply-chain-attack-on-pytorch/

    2024

  39. [47]

    Removing PGP from PyPI , May 2023

    Stufft, D. Removing PGP from PyPI , May 2023. https://blog.pypi.org/posts/2023-05-23-removing-pgp/

    2023

  40. [48]

    Founding zero-knowledge proofs of training on optimum vicinity

    Tan, G., Gascón, A., Meiklejohn, S., Raykova, M., Wang, X., and Luo, N. Founding zero-knowledge proofs of training on optimum vicinity. In Proceedings of ACM CCS, 2025

    2025

  41. [49]

    An exploratory study of deep learning supply chain

    Tan, X., Gao, K., Zhou, M., and Zhang, L. An exploratory study of deep learning supply chain. In Proceedings of the 44th International Conference on Software Engineering (ICSE), 2022

    2022

  42. [50]

    Using TensorFlow securely

    TensorFlow. Using TensorFlow securely. https://github.com/tensorflow/models/security

  43. [51]

    AI act, 2024

    The European Parliament and the Council of the European Union . AI act, 2024. https://artificialintelligenceact.eu/

    2024

  44. [52]

    https://pytorch.org/blog/compromised-nightly-dependency/

    The PyTorch Team , December 2022. https://pytorch.org/blog/compromised-nightly-dependency/

    2022

  45. [53]

    Getty Images sues AI art generator Stable Diffusion in the US for copyright infringement , February 2023

    Vincent, J. Getty Images sues AI art generator Stable Diffusion in the US for copyright infringement , February 2023. https://www.theverge.com/2023/2/6/23587393/ai-art-copyright-lawsuit-getty-images-stable-diffusion

    2023

  46. [54]

    EvilModel : Hiding malware inside of neural network models

    Wang, Z., Liu, C., and Cui, X. EvilModel : Hiding malware inside of neural network models. In Proceedings of the 2021 IEEE Symposium on Computers and Communications (ISCC), pp.\ 1--7, 2021

    2021

  47. [55]

    How a 2011 hack you’ve never heard of changed the Internet 's infrastructure, December 2016

    Wolff, J. How a 2011 hack you’ve never heard of changed the Internet 's infrastructure, December 2016. https://slate.com/technology/2016/12/how-the-2011-hack-of-diginotar-changed-the-internets-infrastructure.html

    2011

  48. [56]

    Zuckerpunch - abusing self hosted Github runners at Facebook , August 2022

    Young, M. Zuckerpunch - abusing self hosted Github runners at Facebook , August 2022. https://markyoung.us/post/zuckerpunch/

    2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.