Pith. sign in

REVIEW 27 references

When a third-party API router sits between a coding agent and the model, it can rewrite responses so the agent runs attacker-chosen repository actions that client safeguards never see coming.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-30 17:23 UTC pith:VW2ICC6J

load-bearing objection End-to-end evidence that on-path API routers can force 0% defense success on autonomous coding agents; the measurement is real, the generality claim needs tighter bounds.

arxiv 2607.23624 v2 pith:VW2ICC6J submitted 2026-07-26 cs.SE cs.AIcs.CL

Where Is the Cost of Third-Party API Routers in Agentic Software Development?

Donghao Fu , Jingxin Li , Xue Jiang , Yihong Dong This is my paper
classification cs.SE cs.AIcs.CL
keywords coding agentsAPI routersLLM supply chainrouter-side injectiontool-call integrityagentic software engineeringoutput integrityclient-side safeguards
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Coding agents increasingly reach language models through third-party API routers that unify keys, backends, and load balancing. In high-autonomy mode those agents auto-approve tool calls, so the router sits on the trusted path: it can read and rewrite every response before the agent acts. This paper argues that creates a control gap—nothing checks that the provider’s output matches the repository actions the agent actually runs—and that the gap is not theoretical. Using a replay-and-inject framework and 400 crafted malicious tool-call samples, the authors show that four levels of router-side tampering, from blunt response swap to subtle distribution-mixed rewriting, drive Claude Code, Codex, Cursor, and OpenCode to execute the injected actions with a 0% defense success rate when no extra mitigations are added. Permission modes and backend model choice do not close the gap. Whitelists and separate LLM reviewers raise resistance but still leave substantial leakage and collateral blocking, so the authors conclude that trustworthy agentic development needs provider-side output-integrity guarantees, not only client-side gates.

Core claim

Router-side intervention can systematically change what coding agents execute in a real repository while remaining hard for ordinary client-side checks to stop. Across four agents and four increasing-subtlety injection levels, defense success rate is 0% without extra mitigations: the injected tool call reaches execution, so provider output and agent action diverge whenever an untrusted router mediates the path.

What carries the argument

SIDEL, a record–replay–inject harness that freezes provider traces and applies a four-level router intervention taxonomy (L1 full substitution, L2 append, L3 polished rewrite, L4 next-token distribution mixing) so downstream repository actions can be attributed to the delivered response rather than model sampling noise.

Load-bearing premise

That testing agents in their most autonomous permission settings, on three SWE-bench Lite tasks, with a hand-built set of 400 malicious injections, is enough to show existing client-side permission mechanisms fail in real practice.

What would settle it

Re-run the same four agents on a broader set of live multi-repo workflows with default non-bypass permission UX and independently attested provider responses: if injected tool calls are blocked or never executed at high rates once the response path is integrity-checked end-to-end, the claimed practical control gap shrinks or disappears.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • High-autonomy coding-agent deployments that route through third-party gateways inherit a device-level integrity risk even when the upstream model is honest.
  • Client permission modes alone cannot restore control if they only gate what the agent already received from the router.
  • Whitelist and LLM-review gates reduce but do not eliminate successful injection and can block benign tools, so they are incomplete fixes.
  • Deployments that need trustworthy repository actions are pushed toward provider-supported output-integrity (signed or attested responses) rather than router-transparent trust.
  • Security evaluations of coding agents must treat the response path as an adversarial surface, not only prompts and tool outputs.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Enterprises standardizing on one multi-provider gateway may be concentrating supply-chain risk: a single compromised router becomes a universal tool-call injector across every connected agent.
  • Distribution-aligned (L4-style) rewriting foreshadows attacks that leave almost no edit boundary for human or LLM reviewers, raising the bar for any purely textual audit.
  • The same control-gap logic likely extends beyond coding agents to any tool-using agent whose actions are authorized from an unauthenticated intermediary response.
  • A practical next measurement would compare attested direct provider channels against the same agents and payloads to quantify how much integrity guarantees buy in DSR terms.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Circularity Check

0 steps flagged

No significant circularity: empirical DSR measurements do not reduce to fitted inputs or self-defined success criteria.

full rationale

This paper is an end-to-end empirical security study, not a first-principles derivation. The central claim (0% DSR without mitigations across agents, modes, backends, and L1–L4) is defined operationally as Malicious Denied / Total Malicious Samples on observed repository-level executions under SIDEL replay-inject (§4.4.1, Eq. 12), with a fixed 400-sample hand-built payload set and three SWE-bench Lite tasks. That quantity is measured against replayed provider baselines, not predicted from a theory whose parameters were fit to the same outcomes. L4’s mixing weight α=0.75 is an experimental hyperparameter with a separate rewrite-quality appendix (App. B); it does not force attack success by construction. Whitelist and LLM-review mitigations report independent DSR/WBR tradeoffs. Related-work citations support context (malicious routers in the wild, prompt injection) but are not load-bearing uniqueness theorems by the same authors that forbid alternatives. Manual dataset balance and autonomous-mode configuration are scope/external-validity choices, not circular reductions of the headline result to its inputs. No self-definitional loop, fitted-input-as-prediction, or renaming of a known closed-form result appears in the derivation chain.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 3 invented entities

The central claim rests on a standard threat model (honest provider, malicious-or-compromised application-layer router with plaintext access), an operational definition of defense success (malicious tool call denied before execution), and experimental instruments (SIDEL, 400 hand-built injections, three SWE-bench Lite tasks). No physical constants or deep mathematical postulates; free choices are experimental hyperparameters and scope limits.

free parameters (4)
  • L4 mixing weight α = 0.75
    Controls blend of clean vs injected next-token distributions; main experiments fix α=0.75 with a small qualitative/LLM-judge appendix over {0.5, 0.75, 1.0}.
  • Injection turn window = uniform over first 15 turns
    Injection round sampled from the first 15 turns of each run; early injection is argued to redirect workflows but is a design choice affecting measured success.
  • Task instance count = 3 tasks
    Denominator uses 400 samples × 3 SWE-bench Lite tasks = 1200; three tasks chosen under cost constraints and bound external validity.
  • LLM review token budget (stage 1) = 64
    First-stage reviewer max tokens set to 64; can affect block/allow behavior independently of the security thesis.
axioms (5)
  • domain assumption Upstream provider is honest and generates per its own safety policies; the only integrity break under study is router-mediated delivery.
    Stated in §3.2 Study Scope and Assumptions; excludes model jailbreaks and provider compromise.
  • domain assumption The router has authorized application-layer plaintext access to requests/responses (terminates TLS to client and re-originates to provider).
    Core threat model in Introduction and Fig. 1; distinguishes the setting from network MITM against encryption.
  • domain assumption Defense success is correctly operationalized as denying the malicious injected tool call before repository execution (DSR).
    §4.4.1; focuses on action integrity rather than textual similarity or task pass@k.
  • domain assumption Replaying recorded provider traces and injecting on matched turns isolates router effects from upstream sampling noise.
    §3.4 and §6.1; standard for internal validity but assumes replay fidelity to live multi-turn adaptation.
  • ad hoc to paper High-autonomy / auto-approve permission settings are the relevant default for measuring practical coding-agent risk.
    RQ1–RQ2 configure closest fully autonomous modes; paper argues users adopt them for productivity, which drives the 0% DSR headline.
invented entities (3)
  • SIDEL framework independent evidence
    purpose: Record, replay, inject, and evaluate router-side interventions on real coding-agent tool loops without modifying agent or provider.
    New experimental apparatus (§3.4); engineering artifact, not a physical entity. Independent evidence is the public code release claim.
  • L1–L4 router-side intervention taxonomy (incl. distribution-alignment L4) no independent evidence
    purpose: Grade injection subtlety from substitution through append and polish to mixed next-token regeneration.
    Organizing device for experiments (Fig. 2, §3.3). L4’s mixture is a methodological construct; success at L1–L3 already carries the main claim.
  • 400-sample malicious tool-call dataset no independent evidence
    purpose: Controlled, balanced payloads across execution, buggy code, exfiltration, and supply-chain categories for replay-inject.
    Manually constructed measurement instrument (Appendix A), not a prevalence sample from the wild.

pith-pipeline@v1.2.0-grok45-kimik3 · 24415 in / 3776 out tokens · 80541 ms · 2026-07-30T17:23:35.319137+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Where Is the Cost of Third-Party API Routers in Agentic Software Development?." pith.science (2026). https://pith.science/paper/VW2ICC6J

@misc{pith2026260723624,
  author       = {Pith},
  title        = {Pith review of: Where Is the Cost of Third-Party API Routers in Agentic Software Development?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VW2ICC6J}},
  note         = {Machine review of arXiv:2607.23624}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Third-party API routers have become a common layer that unifies access across increasingly diverse LLM providers. In coding-agent workflows, high-autonomy operation is widely adopted because it reduces interaction overhead. As a result, a third-party API router, which sits between the agent and the upstream provider, inevitably occupies the trusted path. It can inspect and modify every request and response, yet no mechanism verifies alignment between the provider's output and the repository-level actions ultimately executed by the agent. Consequently, client-side permission mechanisms may become ineffective in practice. Whether this control gap produces real, hard-to-detect effects on software development tasks remains empirically unmeasured. In this paper, we conduct an empirical study of router-side injection in coding agents, examining four intervention levels of increasing subtlety: Response Substitution (L1), Response Append (L2), LLM-Polished Injection (L3), and LLM-Polished with Distribution Alignment Injection (L4). Moreover, we develop SIDEL, a framework for trace recording, replay, injection, and defense evaluation, with a curated dataset of 400 samples. We evaluate four representative coding agents, and further evaluate whitelist-based execution control and LLM review. Router-side intervention substantially alters repository-level actions and remains difficult for existing client-side safeguards to detect. Without additional mitigations, all evaluated agents achieved a defense success rate of 0 percent across all injection levels. Client-side mitigations and reactive reviews improve resistance but do not fully restore end-to-end control, motivating provider-side output-integrity guarantees. Our code is available at https://github.com/Riyasushin/SIDEL.

Figures

Figures reproduced from arXiv: 2607.23624 by Donghao Fu, Jingxin Li, Xue Jiang, Yihong Dong.

Figure 1
Figure 1. Figure 1: An intermediary routing layer between coding agents and LLM providers creates a new [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Four router-side intervention levels. (a) L1 replaces the provider response with one con [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: SIDEL coordinates task execution through a manager, a shared router, and isolated task [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Tool-type breakdown of binary review outcomes under DeepSeek-V4-Pro and DeepSeek [PITH_FULL_IMAGE:figures/full_fig_p014_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

27 extracted references · 15 linked inside Pith

  1. [2]

    Dehghantanha and S

    A. Dehghantanha and S. Homayoun. SoK: The attack surface of agentic AI—tools and autonomy. arXiv preprint arXiv:2603.22928,

    arXiv

  2. [3]

    Y . Dong, J. He, S. Liu, Y . Hou, D. Du, Z. Xu, S. Yu, B. Yang, Y . Xia, and H. Chen. DeltaBox: Scaling stateful AI agents with millisecond-level sandbox checkpoint/rollback.arXiv preprint arXiv:2605.22781,

    Pith/arXiv arXiv

  3. [4]

    Y . Du, Z. Li, N. Li, and B. Ding. Beyond data privacy: New privacy risks for large language models. arXiv preprint arXiv:2509.14278,

    arXiv

  4. [7]

    C. E. Jimenez, J. Yang, A. Wettig, S. Yao, K. Pei, O. Press, and K. Narasimhan. SWE-bench: Can language models resolve real-world GitHub issues?arXiv preprint arXiv:2310.06770,

    Pith/arXiv arXiv

  5. [9]

    Kumar, Y

    A. Kumar, Y . Bajpai, S. Gulwani, G. Soares, and E. Murphy-Hill. Why AI agents still need you: Findings from developer-agent collaborations in the wild.arXiv preprint arXiv:2506.12347,

    arXiv

  6. [10]

    P. Li, S. Wang, Y . Huang, Y . Shi, C. Zhang, Q. Li, Y . Lyu, C. Shan, F. Li, C. Feng, C. Zhu, and L. Chen. AgentCanary: A security evaluation framework for autonomous AI agents in real exe- cutable environments.arXiv preprint arXiv:2606.10484,

    Pith/arXiv arXiv

  7. [11]

    Q. Lin, X. Ji, S. Zhai, Q. Shen, Z. Zhang, Y . Fang, and Y . Gao. Life-cycle routing vulnerabilities of LLM router.arXiv preprint arXiv:2503.08704,

    Pith/arXiv arXiv

  8. [12]

    X. Lin, Y . Liu, Y . Chen, Y . Wu, Y . Ning, Y . Liu, N. Sun, S. Zhang, B. Chong, C. Zhou, and Y . Cao. SafeHarness: Lifecycle-integrated security architecture for LLM-based agent deployment.arXiv preprint arXiv:2604.13630,

    Pith/arXiv arXiv

  9. [13]

    H. Liu, C. Shou, H. Wen, Y . Chen, R. J. Fang, and Y . Feng. Your agent is mine: Measuring malicious intermediary attacks on the LLM supply chain.arXiv preprint arXiv:2604.08407, 2026a. J. Liu, X. Zhao, X. Shang, and Z. Shen. Dive into Claude Code: The design space of today’s and future AI agent systems.arXiv preprint arXiv:2604.14228, 2026b. M. Luo, Z. Z...

    Pith/arXiv arXiv

  10. [14]

    16 Preprint, July 2026 Y . Mou, Z. Xue, L. Li, P. Liu, S. Zhang, W. Ye, and J. Shao. ToolSafe: Enhancing tool invoca- tion safety of LLM-based agents via proactive step-level guardrail and feedback.arXiv preprint arXiv:2601.10156,

    arXiv 2026

  11. [15]

    Pedro, M

    R. Pedro, M. E. Coimbra, D. Castro, P. Carreira, and N. Santos. Prompt-to-SQL injections in LLM- integrated web applications: Risks and defenses. InProceedings of the 2025 IEEE/ACM 47th International Conference on Software Engineering, pp. 1768–1780. IEEE/ACM,

    2025

  12. [16]

    Y . Qu, Y . Liu, T. Geng, G. Deng, Y . Li, L. Y . Zhang, Y . Zhang, and L. Ma. Supply-chain poisoning attacks against LLM coding agent skill ecosystems.arXiv preprint arXiv:2604.03081,

    Pith/arXiv arXiv

  13. [17]

    J. Shi, Z. Yuan, G. Tie, P. Zhou, N. Z. Gong, and L. Sun. Prompt injection attack to tool selection in LLM agents.arXiv preprint arXiv:2504.19793,

    Pith/arXiv arXiv

  14. [18]

    Takerngsaksiri, J

    W. Takerngsaksiri, J. Pasuksmit, P. Thongtanunam, C. Tantithamthavorn, R. Zhang, F. Jiang, J. Li, E. Cook, K. Chen, and M. Wu. Human-in-the-loop software development agents.arXiv preprint arXiv:2411.12924,

    Pith/arXiv arXiv

  15. [19]

    K. Tallam. Authorization propagation in multi-agent AI systems: Identity governance as infrastruc- ture.arXiv preprint arXiv:2605.05440,

    Pith/arXiv arXiv

  16. [20]

    H. Tang, Y . Yan, J. Lu, H. Liu, and E. Dai. Route to rome attack: Directing LLM routers to expensive models via adversarial suffix optimization.arXiv preprint arXiv:2604.15022,

    Pith/arXiv arXiv

  17. [21]

    M. N. Uddin, A. Saeidi, E. Blanco, and C. Baral. LedgerAgent: Structured state for policy-adherent tool-calling agents.arXiv preprint arXiv:2606.20529,

    Pith/arXiv arXiv

  18. [22]

    Xiang, D

    C. Xiang, D. Zagieboylo, S. Ghosh, S. Kariyappa, K. Greshake, H. Xiao, C. Xiao, and G. E. Suh. Architecting secure AI agents: Perspectives on system-level defenses against indirect prompt injection attacks.arXiv preprint arXiv:2603.30016,

    arXiv

  19. [23]

    C. Xiao, Z. Jiao, S. Wang, W. Wang, B. Zhao, H. Wei, L. Zhang, and L. Qu. Socratic-SWE: Self- evolving coding agents via trace-derived agent skills.arXiv preprint arXiv:2606.07412,

    Pith/arXiv arXiv

  20. [24]

    S. Xie, Q. Wu, H. Lu, Z. Sun, Q. Wu, B. Qin, and Q. Wang. The proxy knows too much: Sealing LLM API routers with attested TEEs.arXiv preprint arXiv:2606.16358,

    arXiv

  21. [25]

    Y . Xie, M. Luo, Z. Liu, Z. Zhang, K. Zhang, Y . Liu, Z. Li, P. Chen, S. Wang, and D. She. Red- teaming coding agents from a tool-invocation perspective: An empirical security assessment. arXiv preprint arXiv:2509.05755,

    Pith/arXiv arXiv

  22. [26]

    J. Yang, C. E. Jimenez, A. Wettig, K. Lieret, S. Yao, K. Narasimhan, and O. Press. SWE- agent: Agent-computer interfaces enable automated software engineering.arXiv preprint arXiv:2405.15793,

    Pith/arXiv arXiv

  23. [27]

    id": "malexec-041

    W. Zhang, H. Xu, Z. Wang, Z. Li, Z. He, X. Wei, and K. Ren. RerouteGuard: Understanding and mitigating adversarial risks for LLM routing.arXiv preprint arXiv:2601.21380, 2026a. Y . Zhang, X. Deng, J. Wu, Y . Xiao, K. Xu, and Q. Li. AgentWard: A lifecycle security architecture for autonomous AI agents.arXiv preprint arXiv:2604.24657, 2026b. Y . Zhang, Y . ...

    arXiv 2026

  24. [2023]

    Huang, X

    C. Huang, X. Huang, and A. Milani Fard. Auditing MCP servers for over-privileged tool capabilities. arXiv preprint arXiv:2603.21641,

    arXiv

  25. [2024]

    J. Kim, X. Liu, Z. Wang, S. Qiu, B. Li, W. Guo, and D. Song. The attack and defense landscape of agentic AI: A comprehensive survey.arXiv preprint arXiv:2603.11088,

    arXiv

  26. [2025]

    R. Fahey. CacheProbe: Auditing prompt cache isolation in gateway APIs. InProceedings of the 2026 IEEE Symposium on Security and Privacy Workshops (SPW), Secure Agents for Generative Artificial Intelligence (SAGAI). IEEE,

    2026

  27. [2026]

    Bouzenia and M

    I. Bouzenia and M. Pradel. Understanding software engineering agents: A study of thought-action- result trajectories.arXiv preprint arXiv:2506.18824, 2025a. I. Bouzenia and M. Pradel. You name it, i run it: An LLM agent to execute tests of arbitrary projects. arXiv preprint arXiv:2412.10133, 2025b. C. Bühler, M. Biagiola, L. Di Grazia, and G. Salvaneschi....

    arXiv