Pith. sign in

REVIEW 2 major objections 5 minor 37 references

Setup Complete, Now You Are Compromised: Weaponizing Setup Instructions Against AI Coding Agents

T0 review · 2 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that AI coding agents installing dependencies from project documentation are a practical supply-chain vector, and that whether an attack is caught depends on the pairing of model and tool harness rather than model intellig

desk verdict Solid measurement of a real gap; the pair-level effect holds, but the checkpoint attribution is over-sold and the --print inconsistency needs a straight answer. read the letter →

arxiv 2607.15143 v1 pith:DNU4TMEN submitted 2026-07-16 cs.CR cs.HCcs.SE

Aadesh Bagmar , Pushkar Saraf This is my paper
classification cs.CRcs.HCcs.SE
keywords AIcodingagentssupplychainattackpackageinstallationsecuritytyposquattingdependencyconfusionuntrustedregistrypre-installverificationharness-modelinteraction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that AI coding agents, when told to set up a project, will install dependencies named in documentation without verifying name, source, or version—and that whether an attack is caught depends on the combination of the model and the tool harness that mediates its commands, not on model capability alone. The authors construct twelve realistic setup scenarios spanning five attack classes and show that the same model that catches a malicious registry through one harness installs it through another; a controlled swap moves detection from 10/10 to 9/30 and reverses on a second attack. They find a sharp asymmetry: agents catch obvious typosquats almost everywhere but nearly always install from untrusted sources, hidden index redirects, and known-vulnerable version pins. A security-oriented prompt closes part of the gap, and a deterministic pre-install hook that checks names, sources, and versions before any code runs catches ten of the eleven scenarios it targets. If the paper is right, safer agent setup is mainly an architecture problem—a checkpoint before installation—not a wait-for-smarter-models problem.

What carries the argument

The central object is the harness-model pair, and the key mechanism is a pre-execution checkpoint: a pause before a shell command runs, inside which a capable model can act on its own security reasoning. The paper contrasts this with run-then-report flows and with security-oriented system prompts, each of which covers only the dimension it names. Carrying the defense is a proof-of-concept pre-install hook—a deterministic gate that intercepts install commands before the shell executes them and applies seven checks: name proximity and separator normalization, package existence and age, source trust over index URLs, hidden directives inside requirements files, configuration-file poisoning, and

What would settle it

Run the same model on the same malicious-registry repository through two harnesses that are identical in system prompt and differ only in whether they pause before executing install commands; if detection rates do not move with the pause, the paper's claim that the harness architecture drives the outcome is wrong. A complementary check: replicate the reversed result with prompts changed but no checkpoint difference; if a prompt alone reproduces the 0/10-to-10/10 flip, the conclusion reduces to a prompt effect.

Watch

Extended reading notes

Core claim

At the core is the 'install gap': the absence of any authenticity, name, or version check between a package named in documentation and its code executing during installation. The paper shows that autonomous agents remove the human pause that once made the gap survivable, so a documentation-only edit to a README, requirements file, or Makefile can redirect an agent to an untrusted registry, a vulnerable version, or a wrong-but-plausible name. Across nine harness-model configurations, name-based attacks are largely caught, but source-based attacks are missed almost everywhere; vulnerable version pins are installed by every configuration in all 270 runs, even though the models can recite the re

Load-bearing premise

The load-bearing premise is that the controlled harness swap isolates the harness as the causal factor, but swapping production harnesses changes both the pre-execution checkpoint and the bundled system prompt at once, so the experiment as run does not cleanly separate which mechanism drives the detected effect.

Editorial extensions

If this is right

  • Security evaluations of coding agents are not transferable across harnesses: a model's detection rate on a benchmark means nothing until the harness pairing is specified.
  • Attackers seeking to compromise agent-driven setups should focus on source redirection and version pins rather than typosquats, since most agents treat registry flags as ordinary configuration and never query a vulnerability database during setup.
  • A deterministic pre-install check, independent of model tier, blocks most documentation-borne install attacks, while post-install audit tools only report a compromise after install-time code has already executed.
  • Version-pin vulnerabilities stay invisible unless the harness or prompt explicitly targets versions; general security instructions that mention names and sources leave the version dimension open.
  • The same source blind spot appears on npm and Cargo, so the install gap is a cross-ecosystem property of agentic setup rather than a Python-specific artifact.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The harness swap in the paper changes both the pre-execution checkpoint and the bundled system prompt at once; a natural extension would vary each independently to attribute the effect, which the current experiment does not do.
  • Because the source-based attacks exploit a structural trust gap in documentation-driven install flows, the result likely transfers to future harnesses and package managers beyond the ones tested, even as model capabilities improve.
  • The paper's scoring choice—count install-then-flag as a non-detection—implies that real-world deployments should treat any warning that arrives after resolution as inadequate, and this reframing could change how other agent-security benchmarks grade success.
  • The hook's measured false-positive pattern (0.5% of top packages, each a genuine edit-distance-1 collision) suggests a production version needs allowlists or reputation signals; that is an engineering step the paper sketches but does not build.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. This paper evaluates whether AI coding agents can be induced to install malicious packages by editing only project documentation (README, requirements.txt, Makefile). Twelve scenarios across five attack classes (name confusion, source/registry redirection, vulnerable version pins, configuration poisoning, error-message injection) are run against nine harness-model configurations using a generic setup prompt; detection is scored deterministically from the final environment via `uv pip show`. The central empirical claims are: (i) detection is a property of the harness-model pair, demonstrated by a controlled ablation in which the same model and attack flip from 10/10 to 9/30 (R6a, Opus) and reverse (R6b, Sonnet) when the harness is changed; (ii) source-based attacks are missed almost everywhere, while blatant name typosquats are reliably caught except for separator-confusion and social-engineering variants; (iii) vulnerable version pins are installed 0/30 by all configurations, with install-then-flag behavior from one model; and (iv) a deterministic pre-install hook plus targeted prompts close most of the measured gap, with the hook covering 10/11 scenarios on which it was built.

Significance. If the empirical results hold, this is a timely and useful measurement: it provides the first systematic, deterministically scored evaluation of install-time documentation-driven supply-chain attacks across production coding-agent harnesses, and the pair-level finding (same model, swapped harness, R6a 10/10 to 9/30, p=1.1e-4) is genuinely informative for practitioners. The name/source asymmetry (typosquats nearly solved; registry redirection missed almost everywhere) is a crisp, falsifiable claim with direct defensive implications. Strengths include deterministic end-state scoring via `uv pip show`, benign controls, Wilson intervals with Fisher's exact tests on the load-bearing comparisons, an independent audit of the name-sweep verdicts, cross-ecosystem replication, and unusually candid limitation statements (construction-only hook evaluation, narrow benign surface, 'mechanistic rather than comparative'). The main risk is over-attribution of the harness effect to the gating checkpoint, which the mechanism section claims but the experiment does not isolate.

major comments (2)
  1. [6.1/Table 3; 4.1 vs. Appendix C] The ablation treats 'harness' as a single variable, but swapping Claude Code for Copilot CLI changes the checkpoint, the bundled system prompt, and other tooling at once. The paper itself attributes Row 2 (Sonnet/R6b) to 'Copilot CLI's security-oriented system prompt' (§6.1), so prompt content is a live alternative mechanism; Row 1 (Opus/R6a: 10/10 vs 9/30) is consistent with a checkpoint effect but does not exclude other harness differences. Moreover, §4.1 says Claude Code 'requires explicit user approval for tool calls via a command preview dialog,' yet Appendix C reports all runs were non-interactive `claude --print` invocations; how the credited pre-execution checkpoint operates in `--print` mode is never explained, and if the preview is bypassed the 10/10 R6a runs do not contain the mechanism §9.3's defense rests on. Please report the exact permission setting and the agent's view in
  2. [10/Table 8] The proof-of-concept hook is evaluated on the eleven scenarios it was built to catch; the paper acknowledges this measures 'construction rather than generalization' (§10). The abstract and conclusion nonetheless present the hook as closing 'most of' the gap, and an adaptive adversary controls the same README the hook reads (§10 admits the prompt channel is 'highly steerable'). Out-of-sample and adversarial robustness therefore remain untested. Recommend restricting the abstract/conclusion claims to 'the evaluated scenarios' and stating the construction-only scope there.
minor comments (5)
  1. [6.5/Table 12] The version-targeted prompt experiment is reported on 'urllib3==1.26.4,' while Table 12 lists urllib3 pinned at 1.26.17 (CVE-2023-45803). Both are vulnerable, but the mismatch is confusing; please clarify which pin was used.
  2. [4/Table 1] The paper oscillates among 'twelve scenarios,' thirteen table columns (R1–R12 plus R6b), seven attack vectors, and five attack classes; the mapping from vectors to scenarios is not stated explicitly. A one-sentence reconciliation in Section 3.3/4 would help.
  3. [Table 2/R12] For the volume attack (R12, 377 stacked typosquats), the paper does not define what counts as 'caught' — flagging any one suspicious package, or refusing the whole install. Since detection ranges from 10/10 to 0/10, the scoring rule matters.
  4. [Appendix G/Table 1] The flagship scenario names (tranformers for R1, azurecore for R2) differ from the sweep variants (nmupy, requsts, etc.). The relationship is understandable but deserves an explicit sentence.
  5. [6.5] The 'fresh-baseline 2/10' for the version-targeted prompt experiment is not clearly reconciled with the battery's 0/30 baseline for the same model; state whether this is the same protocol at a different time or a different pin.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: measured detection rates are external observations; the minor self-citation and the acknowledged construction-style hook evaluation do not reduce the central claim to its inputs.

full rationale

The paper's central empirical claim—that install-time detection depends on the harness-model pair, not the model alone—rests on measured run outcomes (uv pip show end states, transcript audits, Fisher exact tests), not on fitted parameters or on equations derived from the conclusion. The R6a/R6b ablation holds model and attack fixed, and although swapping Claude Code for Copilot CLI changes both the pre-execution checkpoint and the system prompt (a genuine confound for the mechanistic attribution, visible in the text's statement that 'Copilot CLI's security-oriented system prompt flags the untrusted domain'), this is an experimental-design limitation, not a circular reduction. The only self-citation, Bagmar et al. [1], is used as background for PyPI concentration (Sections 1 and 2.1) and is not load-bearing for any central result. The pre-install hook is evaluated against the scenarios it was built to catch; the paper explicitly flags this ('We also evaluate it on the scenarios it was built to catch, measuring construction rather than generalization,' Section 10), and its false-positive estimate is an external check, so this is an acknowledged scope limitation rather than a concealed circularity. Overall, no prediction or first-principles result reduces by construction to its inputs; the score reflects only the minor, non-load-bearing self-citation.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central empirical claims rest on external measurements, not on fitted models. The only hand-chosen parameters live in the proof-of-concept pre-install hook, and the paper explicitly evaluates that hook on the scenarios it was built to catch, so its coverage figure is in-sample. The axioms are standard domain assumptions about package managers and agent tool execution, each grounded in cited incidents or the appendix's configuration details.

free parameters (4)
  • name-similarity threshold = edit distance 1 (Levenshtein, transposition, separator-normalization)
    Hand-chosen in the pre-install hook (Appendix I) to flag typosquats; determines which names are blocked.
  • package age threshold = 30 days
    Hand-chosen in the hook (Appendix I) to flag unusually new packages; affects false-positive rate.
  • source trust allowlist = pypi.org, files.pythonhosted.org, test.pypi.org
    Hand-chosen whitelist for the hook's untrusted-source check (Appendix I); any other index is blocked.
  • popular-package reference set = top 1,000 PyPI packages (implied by the false-positive check on 'the 1,000 most-downloaded PyPI packages')
    Reference set used for name-proximity checks in the hook; its exact contents are not fully specified (Appendix I).
assumptions (5)
  • domain assumption Package installation executes arbitrary code at install time (setup.py, import hooks, post-install scripts) before any security check.
    Invoked throughout Section 1 and Section 3.1; standard in the package-supply-chain literature and grounded in documented incidents (torchtriton, Ultralytics).
  • domain assumption The attacker can register typosquat packages on PyPI and operate a package server referenced via --extra-index-url.
    Threat model Section 3.1; the authors assume these capabilities because they are observed in real incidents and public registries.
  • domain assumption AI coding agents execute shell commands with the developer's permissions and often auto-approve them.
    Section 1 and Appendix C; true for the tested harnesses invoked with auto-execution flags (--yolo, --no-ask-user, --force).
  • domain assumption The tested four harnesses and seven models are representative of production coding agents.
    Appendix C lists versions and model identifiers and calls the data a point-in-time snapshot; coverage of the broader agent ecosystem is limited.
  • domain assumption None of the tested harnesses implement automated verification hooks (source/name/CVE checks) before install.
    Section 4.1 states this as a property of the tested systems; it is the premise for the 'install gap' being open.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Setup Complete, Now You Are Compromised: Weaponizing Setup Instructions Against AI Coding Agents." pith.science (2026). https://pith.science/paper/DNU4TMEN

@misc{pith2026260715143,
  author       = {Pith},
  title        = {Pith review of: Setup Complete, Now You Are Compromised: Weaponizing Setup Instructions Against AI Coding Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DNU4TMEN}},
  note         = {Machine review of arXiv:2607.15143}
}
read the original abstract

AI coding agents set up projects by reading documentation and installing the dependencies it lists, without verifying their names, sources, or known vulnerabilities. By editing only a README, requirements file, or Makefile, an attacker can redirect the agent to an untrusted registry, a known-vulnerable version, or a wrong-but-plausible name: documentation becomes a vector for code execution. We present the first systematic evaluation of package-install-time supply-chain attacks delivered through ordinary project-setup documentation across production coding-agent harnesses, probing frontier models on twelve scenarios in five attack classes, grounded in documented incidents. The same model catches an attack through one harness and installs it through another: install-time security rests on the harness-model combination, not the model alone. Agents catch blatant typosquats reliably, but plausible separator-confusion names (azurecore for azure-core) slip through, and how often depends on the harness-model pairing. Source-based attacks like registry redirection are missed almost everywhere. The source blind spot recurs on npm and Cargo, where nearly every model installs the untrusted dependency; name detection carries over less consistently across ecosystems. Security-oriented prompts recover part of the gap but only for the dimension they name; a deterministic pre-install check that verifies names, sources, and versions before any code runs closes most of it.

Figures

Figures reproduced from arXiv: 2607.15143 by the authors.

Figure 1
Figure 1. Trust chain in agent-mediated package installation. Blue blocks show the legitimate flow; red dashed blocks show [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

37 extracted references · 5 linked inside Pith

  1. [1]

    Bagmar, B

    A. Bagmar, B. Wedgwood, E. Levin, and J. Purtilo. I Know What You Imported Last Summer: A Study of Se- curity Threats in the Python Ecosystem.arXiv preprint arXiv:2102.06301, 2021

    arXiv 2021

  2. [2]

    A. Birsan. Dependency confusion: How I hacked into Apple, Microsoft, and dozens of other companies. Medium, February 2021

    2021

  3. [3]

    Neupane, T

    S. Neupane, T. Holmes, N. Wyss, E. Davidson, and L. De Carli. Beyond typosquatting: An in-depth look at pack- age confusion. InProc. USENIX Security Symposium, 2023

    2023

  4. [4]

    Liu, C.-Y

    Y . Liu, C.-Y . Hsu, C.-Y . Huang, M. Backes, R. Wen, and C.-M. Yu. Trust me, import this: Dependency steer- ing attacks via malicious agent skills.arXiv preprint arXiv:2605.09594, 2026

    arXiv 2026

  5. [5]

    Slopsquatting: How AI hallucinations are fueling a new class of supply chain attacks

    Socket.dev. Slopsquatting: How AI hallucinations are fueling a new class of supply chain attacks. Socket Blog, 2025

    2025

  6. [6]

    D. Vu, Z. Gao, L. Williams, and M. Gadyatskaya. An empirical study of malicious code in PyPI ecosystem. InProc. ASE, 2023

    2023

  7. [7]

    Rules file backdoor in Copilot and Cur- sor

    Pillar Security. Rules file backdoor in Copilot and Cur- sor. Pillar Security Research, 2025

    2025

  8. [8]

    ToxicSkills: Malicious AI agent skills

    Snyk. ToxicSkills: Malicious AI agent skills. Snyk Blog, 2025

    2025

Show all 37 references
  1. [9]

    Prompts become shells: RCE vulnerabilities in AI agent frameworks

    Microsoft Security Response Center. Prompts become shells: RCE vulnerabilities in AI agent frameworks. Mi- crosoft Security Blog, May 2026

    2026

  2. [10]

    Liu et al

    Y . Liu et al. SoK: Prompt injection against agentic coding assistants.arXiv preprint arXiv:2501.17548, 2025

    2025 arXiv

  3. [11]

    Greshake, S

    K. Greshake, S. Abdelnabi, S. Mishra, C. Endres, T. Holz, and M. Fritz. Not what you’ve signed up for: Compromising real-world LLM-integrated appli- cations with indirect prompt injection. InProc. 16th ACM Workshop on Artificial Intelligence and Security (AISec), 2023

    2023

  4. [12]

    Debenedetti, J

    E. Debenedetti, J. Zhang, M. Balunovi ´c, L. Beurer- Kellner, M. Fischer, and F. Tramèr. AgentDojo: A dy- namic environment to evaluate attacks and defenses for LLM agents. InProc. NeurIPS Datasets and Bench- marks Track, 2024

    2024

  5. [13]

    Q. Zhan, Z. Liang, Z. Ying, and D. Kang. InjecA- gent: Benchmarking indirect prompt injections in tool- integrated large language model agents. InFindings of the Association for Computational Linguistics (ACL), 2024

    2024

  6. [14]

    S. Pang, Y . Yao, Z. Jiang, Z. Fan, H. Li, and B. Liu. PYPI- LINE: Malicious PyPI package detection via suspicious API knowledge and agent workflow.arXiv preprint arXiv:2606.19063, 2026

    2026 arXiv

  7. [15]

    Y . Qu, Y . Liu, T. Geng, G. Deng, Y . Li, L. Y . Zhang, Y . Zhang, and L. Ma. Supply-chain poisoning attacks against LLM coding agent skill ecosystems.arXiv preprint arXiv:2604.03081, 2026

    2026 arXiv

  8. [16]

    GitHub Copilot

    GitHub. GitHub Copilot. https://github.com/ features/copilot, 2024

    2024

  9. [17]

    Claude Code

    Anthropic. Claude Code. https://claude.ai/ code, 2025

    2025

  10. [18]

    Cursor: The AI-first Code Editor

    Cursor. Cursor: The AI-first Code Editor. https: //cursor.sh, 2024

    2024

  11. [19]

    pypiserver: Minimal PyPI server

    pypiserver contributors. pypiserver: Minimal PyPI server. https://github.com/pypiserver/ pypiserver

  12. [20]

    Ladisa, H

    P. Ladisa, H. Plate, M. Martinez, and O. Barais. SoK: Taxonomy of attacks on open-source software supply chains. InProc. IEEE Symposium on Security and Privacy (S&P), 2023

    2023

  13. [21]

    Ladisa, M

    P. Ladisa, M. Sahin, S. E. Ponta, M. Rosa, M. Martinez, and O. Barais. The hitchhiker’s guide to malicious third- party dependencies. InProc. SCORED Workshop at CCS, 2023

    2023

  14. [22]

    Zimmermann, C.-A

    M. Zimmermann, C.-A. Staicu, C. Tenny, and M. Pradel. Small world with high risks: A study of security threats in the npm ecosystem. InProc. USENIX Security Sym- posium, 2019

    2019

  15. [23]

    pip-audit: Auditing Python environ- ments and dependencies

    Trail of Bits. pip-audit: Auditing Python environ- ments and dependencies. https://github.com/ trailofbits/pip-audit

  16. [24]

    Sigstore: Software signing for everyone

    Sigstore. Sigstore: Software signing for everyone. https://sigstore.dev

  17. [25]

    Ty- posquatted npm packages used to steal cloud and CI/CD secrets

    Microsoft Defender Security Research Team. Ty- posquatted npm packages used to steal cloud and CI/CD secrets. Microsoft Security Blog, May 2026

    2026

  18. [26]

    Incident report: LiteLLM and Telnyx supply chain attack

    Python Software Foundation. Incident report: LiteLLM and Telnyx supply chain attack. PyPI Blog, April 2026. https://blog.pypi.org/posts/ 2026-04-02-incident-report- litellm-telnyx-supply-chain-attack/. 14

    2026

  19. [27]

    W. Larsen. Ultralytics YOLO compro- mised via GitHub Actions injection. https: //blog.yossarian.net/2024/12/06/ zizmor-ultralytics-injection, December 2024

    2024

  20. [28]

    PyTorch dependency confusion via torchtriton

    PyTorch Team. PyTorch dependency confusion via torchtriton. PyTorch Blog, December 2022

    2022

  21. [29]

    Tschacher

    N. Tschacher. Typosquatting in programming language package managers. Bachelor’s thesis, Universität Ham- burg, 2016

    2016

  22. [30]

    R. Duan, O. Alrawi, R. Kasturi, R. Elder, B. Saltafor- maggio, and W. Lee. Towards measuring supply chain attacks on package managers for interpreted languages. InProc. NDSS Symposium, 2021

    2021

  23. [31]

    Always contribute back

    D. Wermke, N. Wöhler, J. Klemmer, M. Fourné, Y . Acar, and S. Fahl. “Always contribute back”: A qualitative study on security challenges of the open source supply chain. InProc. IEEE S&P, 2023

    2023

  24. [32]

    Fischer, K

    F. Fischer, K. Böttinger, H. Xiao, C. Stransky, Y . Acar, M. Backes, and S. Fahl. Stack Overflow considered harmful? The impact of copy&paste on Android appli- cation security. InProc. IEEE S&P, 2017

    2017

  25. [33]

    Perry, M

    N. Perry, M. Srivastava, D. Kumar, and D. Boneh. Do users write more insecure code with AI assistants? In Proc. ACM CCS, 2023

    2023

  26. [34]

    Singla, B

    T. Singla, B. Çakar, P. C. Amusuo, and J. C. Davis. To- wards a benchmark for dependency decision-making. arXiv preprint arXiv:2601.00205, 2026

    2026

  27. [35]

    S. Zhao, D. Wang, K. Zhang, J. Luo, Z. Li, and L. Li. Is vibe coding safe? Benchmarking vulnerability of agent-generated code in real-world tasks.arXiv preprint arXiv:2512.03262, 2026

    2026

  28. [36]

    M. L. Siddiq, X. Zhao, V . C. Lopes, B. Casey, and J. C. S. Santos. Security in the age of AI teammates: An empirical study of agentic pull requests on GitHub. arXiv preprint arXiv:2601.00477, 2026

    2026

  29. [37]

    Default to caution. When uncertain whether an action is safe, pause and ask rather than proceed

    npm, Inc. npm-install: the audit config de- faults to true. npm CLI documentation, v10. https://docs.npmjs.com/cli/v10/ commands/npm-install. A Ethical Considerations We analyze ethics around the stakeholders our work affects, using the Menlo Report principles of Beneficence, ...

    2026

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.