Pith. sign in

REVIEW 3 major objections 7 minor 61 references

This paper proposes a neuro-symbolic security plane, NCS, that gates every privileged AI-agent tool call behind an offline-signed, hash-chained instruction stream, so a prompt-injected planner cannot execute unauthorized actions.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 22:49 UTC pith:YFMOTO5Y

load-bearing objection A genuinely new enforcement mechanism with a real replay gap and partly self-earned evaluation numbers. the 3 major comments →

arxiv 2607.15596 v2 pith:YFMOTO5Y submitted 2026-07-17 cs.CR

From Neural Intent to Cryptographic Authorization: Securing AI-Driven Enterprise Workflows

Jiasi Weng , Jian Weng , Minrong Chen , Ming Li , Jia-Nan Liu , Zhi Li , Yue Zhang This is my paper
classification cs.CR
keywords neural-symbolic securityprompt injection defenseLLM agentstool-call authorizationstream signaturehash-chain verificationfail-closed executioncontrol-flow integrity
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that high-consequence AI workflows should not try to make the language model itself robust against prompt injection; instead, the model should be treated as intrinsically untrusted and stripped of execution authority. It proposes Neural Cryptographic Services (NCS), a deterministic symbolic controller placed between the neural planner and privileged tools. Before runtime, a human signs a worksheet whose instruction blocks are linked by a hash chain; at runtime, NCS verifies the head signature once, walks the chain step by step, and releases only one instruction payload at a time. The agent's proposed tool call must match that payload exactly—including its arguments—before the call is dispatched. The central claim is that even a fully injected planner cannot execute any tool call outside this pre-authorized stream, shifting agent security from model-intent verification to cryptographic authorization.

Core claim

NCS's central claim, formalized as Runtime Control-Flow Integrity, is that for any probabilistic polynomial-time adversary acting through direct or indirect prompt injection, the probability that an unauthorized control action receives 'allow' is negligible. The proof reduces this guarantee to two cryptographic assumptions: collision resistance of the hash function and existential unforgeability of the stream signature. Every privileged tool dispatch must pass a Verifier that requires canonical equality between the agent's proposed arguments and the currently released signed payload. The paper's conclusion is that prompt injection can hijack the model's reasoning, but cannot forge execution

What carries the argument

The load-bearing mechanism is stateful instruction authentication over an offline-signed, hash-chained instruction stream. The user signs only a single head block containing the digest of the first chain block and the stream length; each subsequent block commits to its successor via a hash digest. At runtime, NCS verifies the head signature once (Phase A), then incrementally checks each block's hash against an expected digest register, releases the plain payload, and requires the agent's tool-call arguments to be canonically equal to that payload before dispatch (Phase B). A terminal zero-vector anchor prevents appending extra blocks. This offline-signing, online-verification, exact-binding

Load-bearing premise

The entire guarantee rests on the premise that every security-relevant tool dispatch can be enumerated and signed offline before runtime, with parameters fixed in advance; workflows that require amounts, recipients, or tools chosen only at runtime lie outside the boundary.

What would settle it

A concrete falsifier: take a workflow inside NCS's stated scope, sign the worksheet normally, run a fully adversarial injection campaign, and look for any allowed privileged dispatch whose tool or arguments do not equal the currently released signed block. A single such event—reproduced without breaking the underlying hash or signature—would falsify the Runtime Control-Flow Integrity theorem. A weaker check is to instrument the Verifier to log every allowed dispatch and every denied dispatch and confirm that the allowed set is exactly the signed stream; any mismatch is a counterexample.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • A prompt-injected LLM can be coerced into proposing malicious tool calls, but those calls are denied unless they exactly match a signed, released instruction block; the attacker must instead break the signature or find a hash collision.
  • The boundary is deterministic and independent of model weights, prompting, or classifier accuracy—no trained guardrail is needed to enforce it.
  • Because only the stream head uses an asymmetric signature and each step is a hash check plus parameter-equality check, per-step cryptographic latency is under a couple of milliseconds, far below a single LLM inference turn.
  • NCS can be layered beneath information-flow and least-privilege controls, adding step-bound parameter integrity that those mechanisms do not provide.
  • When verification or binding fails, execution stops before side effects, and the failure is appended to an immutable audit log, preserving prior allow/deny decisions.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • My inference: the practical bottleneck shifts from defending the model to authoring the worksheet—deployment quality depends on a human's ability to foresee and sign every legitimate action and parameter in advance, so tooling that helps analysts write precise signed workflows would determine real-world applicability.
  • My inference: because the argument check is byte-exact, legitimate workflows with runtime-determined parameters (for example, a transfer amount chosen mid-session) will either fail closed or require coarser signed blocks; a natural extension is a template-with-commitment scheme that signs parameter placeholders and binds concrete values through a secondary authenticated channel.
  • My inference: the guarantee covers tool dispatch, not generated text—a compromised model can still leak information through its natural-language reply, so deployments where agent output itself is sensitive would need an independent output filter on top of NCS.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes Neural Cryptographic Services (NCS), a neuro-symbolic enforcement plane for AI-driven enterprise workflows. An untrusted neural planner may draft actions, but a deterministic symbolic controller gates execution using an offline-signed, hash-chained instruction stream. Phase A verifies a signature over the stream head; Phase B incrementally verifies hash-chain continuity and requires exact binding between the agent's proposed tool arguments and the released instruction block. The paper formalizes instruction authenticity, runtime control-flow integrity, and fail-closed traceability, proves security under collision resistance and EUF-CMA/EU-CSA assumptions, and evaluates on AgentDojo and a custom argument-hijacking benchmark, reporting near-zero attack success rates. The stated scope excludes open-ended agents that freely explore tools or parameters at runtime.

Significance. If the central guarantee holds, the paper makes a useful contribution: it shifts the security boundary for LLM agents from probabilistic intent-compliance checking to deterministic cryptographic authorization of each dispatched action. The architectural separation of neural planning from symbolic execution is clean, and the core proofs are standard reductions to hash collision resistance and signature unforgeability. The paper is honest about not preventing neural planner compromise and about excluding open-ended workflows. It also gives explicit credit to complementary IFC/privilege-control defenses. However, the headline claim as stated is stronger than what is actually proved because the signed stream is not bound to a session or single-use context, and part of the empirical evaluation measures the defense mechanism itself rather than model behavior.

major comments (3)
  1. [Section V-C, Step 4; Section VI-B (Theorem 2)] The signed head M0 = h1 || uint64(k) contains no nonce, timestamp, or session identifier, and the paper nowhere states that signed streams are secret (Section IV protects only signing keys). A captured valid worksheet in q_lo (e.g., from logs or retrieved documents) can be replayed in a new session: Phase A accepts the valid signature, Phase B releases blocks, and the Verifier sees canonical equality, so allow is granted. Since the replayed action lies in J, Definition 2 is vacuously satisfied, but the workflow executes a second time (duplicate payment/submission), violating single-use user intent. Theorem 2's assumption that q_lo cannot authorize any control action is therefore false when q_lo contains a valid signed stream. The EU-CSA definition in Section II-B does not address replay. This is a load-bearing gap in the central claim; the authors should bind M0 to a session/nonce and en
  2. [Section VII-B, Table III] ASR_args is defined as 'the rate of argument mismatch detections where the model attempts to invoke the correct tool but with parameters that deviate from the authorized block B_i,' and NCS-Full is defined as adding 'exact JSON-to-argument schema binding on all control tool dispatches' (Section VII). Therefore NCS-Full rejects every such mismatch by construction, and ASR_args = 0% is an entailment of the enforcement rule, not an empirical finding. Table III cannot validate the defense; it only confirms the gate is active. To evaluate argument hijacking empirically, report the model's attempted-deviation rate before the gate or compare with a gate-off configuration. The same caveat applies to ASR_tool for tool-dispatch attacks; the GPT-4o-mini 8.3% residual in Table II is the informative non-tautological part.
  3. [Section II-B; Section V-C; Section VI-A] The EU-CSA game defines forgery only when M* is not a prefix of a queried stream, so prefixes are explicitly allowed. The text claims the security goal prevents 'truncating the stream' (Section II-B), and Theorem 1's deletion case only handles deleting a block and substituting another (requiring a hash collision). The construction registers k in the signed head, but no mechanism forces the workflow to consume all k blocks. A compromised planner can stop after an authorized early block (e.g., retrieve data before the HITL step in Fig. 2) and still receive allow for those actions. If truncation is meant to be prevented, the formal game and proof must address it; otherwise the guarantee should be scoped to 'no unauthorized action' and the truncation claim should be removed.
minor comments (7)
  1. [Section VII.B] Typo: 'Thst is' should be 'That is'; 'Benchark' should be 'Benchmark'.
  2. [Fig. 5] The legend contains 'NCS/uni00ADFull' artifacts, likely from a soft-hyphen character. Please fix the rendering.
  3. [Reference [14]] The venue name is garbled: 'Empirizhu2025melon Engineering' and page ranges are malformed. Please correct the bibliographic entry.
  4. [Section VI.B] The sentence 'q_lo authorize any control action' is missing 'cannot'; it should read 'q_lo cannot authorize any control action.'
  5. [Section VII.A] 'In addition, we so use the adaptive Greedy Coordinate Gradient' should be 'we also use'.
  6. [Section V-C] The notation '0 256' should be typeset consistently (e.g., 0^{256}) and defined clearly as the terminal zero-vector.
  7. [Appendix C] The FATH attacks are informative but currently disconnected from the main evaluation. Consider integrating a summary into the main text or explicitly labeling this as a supplementary adversarial case study.

Circularity Check

0 steps flagged

No significant circularity; formal guarantees rest on external cryptographic assumptions and the zero-ASR evaluation is an enforcement consequence, not a circular prediction.

full rationale

The claimed derivation chain is not circular. Theorems 1 and 2 assume collision resistance of H and EUF-CMA security of the signature scheme, and prove that only signed, hash-chain-verified, exactly-bound instruction blocks can be dispatched. The Verifier's canonical-equality rule makes the conclusion follow by construction, but that is an architectural enforcement property, not an assumption of the target result; no fitted parameter, self-citation, or imported uniqueness theorem is load-bearing. The evaluation's ASR tool / ASR args of 0% are direct consequences of NCS-Full's exact binding gate: a mismatched tool call is denied by definition of the metric ('ASR args: The rate of argument mismatch detections where the model attempts to invoke the correct tool but with parameters that deviate from the authorized block B_i'), so the zero values restate the enforcement rule. That is a tautological evaluation headline, but it is not a derivation of a security claim from that same claim, and the central guarantee is independently grounded in cryptography. The paper also states its scope limitations explicitly (open-ended runtime-flexible agents are out of scope; exact pre-signable bindings are required), which narrows applicability but does not create circularity. The replay/session-binding concern is a real security-scope gap, not a circularity. Overall: no significant circularity.

Axiom & Free-Parameter Ledger

0 free parameters · 5 axioms · 0 invented entities

The central claim rests on standard cryptographic assumptions, a trusted signing boundary, and the architectural premise that every privileged dispatch is expressible as an exact signed payload and crosses the Verifier. No new physical or mathematical entities are introduced; NCS is a composition of stream signatures, hash chains, RBAC credentials, and a symbolic controller.

axioms (5)
  • standard math Collision resistance of H and EUF-CMA security of the signature scheme
    Invoked in Theorem 1 (Section VI-A); the paper instantiates these with SHA-256 and Ed25519 (Section VII).
  • domain assumption Workflow signing keys remain secure and under staff control; the cryptographic server's hardware boundary is trusted
    Section IV Assumptions: 'We trust the cryptographic server's hardware boundary and assume workflow signing keys remain secure under staff control.' If false, an attacker can sign malicious streams and bypass NCS.
  • domain assumption All untrusted external content is confined to q_lo, outside the signed envelope q_hi
    Theorem 2 is conditional on 'if all untrusted external content resides exclusively in q_lo'; the system must correctly separate signed stream markers from unstructured text at the Neural Planner boundary.
  • domain assumption Canonical serialization enc() yields a unique byte representation for every instruction
    Section V-C Step 1: the hash chain and exact binding depend on canonical byte encodings. If two encodings of the same instruction exist, valid calls can be rejected or mismatched.
  • domain assumption The Verifier sits on the only execution path to every privileged tool, with no planner-bypassable side channels
    Section V-D.4 places the Verifier on the hot path; the guarantee that no neural output triggers privileged execution without symbolic gates requires that every privileged tool is mediated.

pith-pipeline@v1.3.0-alltime-deepseek · 23503 in / 12029 out tokens · 153401 ms · 2026-08-01T22:49:06.630297+00:00 · methodology

0 comments
read the original abstract

The rapid adoption of artificial intelligence (AI)-driven workflows is transforming high-consequence} government and enterprise systems into language-based, tool-using and increasingly autonomous infrastructures. While these workflows can delegate planning autonomously, security-critical execution should be strictly mediated. Conventional key management services authenticate who may invoke a cryptographic primitive, but remain agnostic to which workflow steps are authorized at runtime. An AI-driven workflow can still be hijacked by injection attacks into executing malicious actions that satisfy identity checks yet violate user intent. We propose \emph{Neural Cryptographic Services} (NCS), a neuro-symbolic security enforcement plane interposed between neural planners and privileged tools. NCS decouples cognitive planning from execution authority: an untrusted neural planner drafts structured plans, while a deterministic symbolic controller gates execution using an offline-signed, hash-chained instruction stream. Specifically, NCS validates cryptographic signatures and hash chains incrementally, releasing a single instruction payload at a time while strictly binding proposed tool parameters to the verified payload. Out-of-order or altered tool calls fail-closed, and state transitions are logged for post-hoc auditing. NCS does not attempt to prevent neural planner compromise under injection; it guarantees that a compromised planner cannot dispatch actions outside the cryptographic authorization. We evaluate NCS on AgentDojo and a custom argument-hijacking benchmark. NCS drives attack success rates to near zero while preserving acceptable utility on benign workflows. NCS thus shifts security for high-consequence AI-powered workflows from verifying model-intent compliance to verifying that a proposed dispatch matches a cryptographically authorized step.

Figures

Figures reproduced from arXiv: 2607.15596 by Jia-Nan Liu, Jian Weng, Jiasi Weng, Ming Li, Minrong Chen, Yue Zhang, Zhi Li.

Figure 1
Figure 1. Figure 1: Comparison of the traditional deterministic information processing and dynamic AI-driven workflow, along with the [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Illustration of the AI-driven financial report generation [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Overview of two sequential phases with stateful instruction authentication. Phase A certifies [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 3
Figure 3. Figure 3: First, only the signed worksheet prefix is authen [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: IPI attacks across four evaluated models and different defenses. [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: Latency measurement. not increase its ASR. For both models, NCS-Full reduces ASR to 0% against both static and adaptive attacks. Under the adaptive attack on Llama3-8B, the model still calls the attacker-specified tool in 76% of cases (the target rate), but cryptographic verification and parameter checking reject such calls. Cryptographic Overhead. We evaluate the end-to-end latency overhead on the determi… view at source ↗
Figure 8
Figure 8. Figure 8: Security performance visualization of stream-integrity [PITH_FULL_IMAGE:figures/full_fig_p016_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: NCS-Full defense workflow for this attack case. [PITH_FULL_IMAGE:figures/full_fig_p017_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

61 extracted references · 18 linked inside Pith

  1. [1]

    Agent-ops: A multi-agent orchestration framework for end-to-end sop automation in e-commerce operations,

    A. Singh, S. Agrawal, S. Adhikari, V . S. Puranik, S. Tiwari, and D. Assudani, “Agent-ops: A multi-agent orchestration framework for end-to-end sop automation in e-commerce operations,” inProceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), 2026, pp. 436–446

    2026

  2. [2]

    Proagent: From robotic process automation to agentic process automation,

    Y . Ye, X. Cong, S. Tian, J. Cao, H. Wang, Y . Qin, Y . Lu, H. Yu, H. Wang, Y . Linet al., “Proagent: From robotic process automation to agentic process automation,” https://arxiv.org/pdf/2311.10751, 2023

    Pith/arXiv arXiv 2023

  3. [3]

    Workflowllm: En- hancing workflow orchestration capability of large lan- guage models,

    S. Fan, X. Cong, Y . Fu, Z. Zhang, S. Zhang, Y . Liu, Y . Wu, Y . Lin, Z. Liu, and M. Sun, “Workflowllm: En- hancing workflow orchestration capability of large lan- guage models,” inInternational Conference on Learning Representations, vol. 2025, 2025, pp. 24 498–24 525

    2025

  4. [4]

    Sap help portal: What is joule,

    SAP, “Sap help portal: What is joule,” https://help.sap. com/docs/joule/serviceguide/what-is-joule, 2024

    2024

  5. [5]

    Flowmind: automatic work- flow generation with llms,

    Z. Zeng, W. Watson, N. Cho, S. Rahimi, S. Reynolds, T. Balch, and M. Veloso, “Flowmind: automatic work- flow generation with llms,” inProceedings of the Fourth ACM International Conference on AI in Finance, 2023, pp. 73–81

    2023

  6. [6]

    Automating repetitive tasks: Ai for invoicing, inventory and more,

    T. E. Times, “Automating repetitive tasks: Ai for invoicing, inventory and more,” https: //economictimes.indiatimes.com/small-biz/sme-sector/ automating-repetitive-tasks-ai-for-invoicing-inventory-and-more/ articleshow/111306183.cms?from=mdr, 2025

    arXiv 2025

  7. [7]

    Genai can revolutionize erp transformations,

    B. C. Group, “Genai can revolutionize erp transformations,” https://www.bcg.com/publications/ 2025/gen-ai-can-revolutionize-erp-transformations, 2025

    2025

  8. [8]

    Is llm-as-a-judge robust? investigating universal adversarial attacks on zero-shot llm assessment,

    V . Raina, A. Liusie, and M. Gales, “Is llm-as-a-judge robust? investigating universal adversarial attacks on zero-shot llm assessment,” inProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024, pp. 7499–7517

    2024

  9. [9]

    One token to fool llm-as-a-judge,

    Y . Zhao, H. Liu, D. Yu, S. Kung, M. Chen, H. Mi, and D. Yu, “One token to fool llm-as-a-judge,”arXiv preprint arXiv:2507.08794, 2025

    Pith/arXiv arXiv 2025

  10. [10]

    Ignore previous prompt: Attack techniques for language models,

    F. Perez and I. Ribeiro, “Ignore previous prompt: Attack techniques for language models,” inNeurIPS ML Safety Workshop, 2022

    2022

  11. [11]

    Prompt injection attack to tool selection in llm agents,

    J. Shi, Z. Yuan, G. Tie, P. Zhou, N. Z. Gong, and L. Sun, “Prompt injection attack to tool selection in llm agents,” arXiv preprint arXiv:2504.19793, 2025

    Pith/arXiv arXiv 2025

  12. [12]

    Obliinjection: Order- oblivious prompt injection attack to llm agents with multi-source data,

    R. Wang, Y . Jia, and N. Z. Gong, “Obliinjection: Order- oblivious prompt injection attack to llm agents with multi-source data,”arXiv preprint arXiv:2512.09321, 2025

    arXiv 2025

  13. [13]

    Agentdojo: A dynamic environment to evaluate prompt injection at- tacks and defenses for llm agents,

    E. Debenedetti, J. Zhang, M. Balunovic, L. Beurer- Kellner, M. Fischer, and F. Tram `er, “Agentdojo: A dynamic environment to evaluate prompt injection at- tacks and defenses for llm agents,”Advances in Neural Information Processing Systems, vol. 37, pp. 82 895– 82 920, 2024

    2024

  14. [14]

    Llms as hackers: Autonomous linux privilege escalation attacks,

    A. Happe, A. Kaplan, and J. Cito, “Llms as hackers: Autonomous linux privilege escalation attacks,”Empi- rizhu2025melon Engineering, vol. 31, no. 3, p. 70, 2026

    2026

  15. [15]

    Regulation and deregulation: understanding the evolution of the chinese cryptography legal regime from the newly released cryptography law of china,

    J. Chen, “Regulation and deregulation: understanding the evolution of the chinese cryptography legal regime from the newly released cryptography law of china,” International Cybersecurity Law Review, vol. 1, no. 1, pp. 73–86, 2020

    2020

  16. [16]

    Regulation (eu) 2024/2847 of the european parliament and of the council,

    O. J. of the European Union, “Regulation (eu) 2024/2847 of the european parliament and of the council,” https://digital-strategy.ec.europa.eu/en/library/ cyber-resilience-act, 2024

    2024

  17. [17]

    Not what you’ve signed up for: Compromising real-world llm-integrated applica- tions with indirect prompt injection,

    K. Greshake, S. Abdelnabi, S. Mishra, C. Endres, T. Holz, and M. Fritz, “Not what you’ve signed up for: Compromising real-world llm-integrated applica- tions with indirect prompt injection,” inProceedings of the 16th ACM workshop on artificial intelligence and security, 2023, pp. 79–90

    2023

  18. [18]

    Malicious openclaw skills used to distribute atomic macos stealer,

    A. Oliveira, B. Tancio, D. Fiser, P. Lin, and R. Reyes, “Malicious openclaw skills used to distribute atomic macos stealer,”Trend Micro Research, 2026

    2026

  19. [19]

    Automatic and universal prompt injection attacks against large language models,

    X. Liu, Z. Yu, Y . Zhang, N. Zhang, and C. Xiao, “Automatic and universal prompt injection attacks against large language models,”arXiv preprint arXiv:2403.04957, 2024

    Pith/arXiv arXiv 2024

  20. [21]

    Neurosymbolic ai: The 3 rd wave,

    A. d. Garcez and L. C. Lamb, “Neurosymbolic ai: The 3 rd wave,”Artificial Intelligence Review, vol. 56, no. 11, pp. 12 387–12 406, 2023

    2023

  21. [22]

    Language agents: Foundations, prospects, and risks,

    Y . Su, D. Yang, S. Yao, and T. Yu, “Language agents: Foundations, prospects, and risks,” inProceedings of the 2024 conference on empirical methods in natural language processing: tutorial abstracts, 2024, pp. 17– 24

    2024

  22. [23]

    Fath: Authentication-based test- time defense against indirect prompt injection attacks,

    J. Wang, F. Wu, W. Li, J. Pan, E. Suh, Z. M. Mao, M. Chen, and C. Xiao, “Fath: Authentication-based test- time defense against indirect prompt injection attacks,” arXiv preprint arXiv:2410.21492, 2024

    Pith/arXiv arXiv 2024

  23. [24]

    How good old-fashioned ai could spark the field’s next revolution,

    N. Jones, “How good old-fashioned ai could spark the field’s next revolution,”Nature, vol. 647, no. 8091, pp. 842–844, 2025

    2025

  24. [25]

    The third ai summer: Aaai robert s. engel- more memorial lecture,

    H. Kautz, “The third ai summer: Aaai robert s. engel- more memorial lecture,”Ai magazine, vol. 43, no. 1, pp. 105–125, 2022

    2022

  25. [26]

    Neuro-symbolic approaches in artificial intelligence,

    P. Hitzler, A. Eberhart, M. Ebrahimi, M. K. Sarker, and L. Zhou, “Neuro-symbolic approaches in artificial intelligence,”National Science Review, vol. 9, no. 6, p. nwac035, 2022

    2022

  26. [27]

    The neuro-symbolic concept learner: Inter- preting scenes, words, and sentences from natural su- pervision,

    J. Mao, C. Gan, P. Kohli, J. B. Tenenbaum, and J. Wu, “The neuro-symbolic concept learner: Inter- preting scenes, words, and sentences from natural su- pervision,” inInternational Conference on Learning Representations, 2019

    2019

  27. [28]

    Mastering symbolic operations: Augmenting language models with compiled neural networks,

    Y . Weng, M. Zhu, F. Xia, B. Li, S. He, K. Liu, and J. Zhao, “Mastering symbolic operations: Augmenting language models with compiled neural networks,” in International Conference on Learning Representations, vol. 2024, 2024, pp. 26 979–27 009. 14

    2024

  28. [29]

    Deepproblog: Neural probabilistic logic programming,

    R. Manhaeve, S. Dumancic, A. Kimmig, T. Demeester, and L. De Raedt, “Deepproblog: Neural probabilistic logic programming,”Advances in neural information processing systems, vol. 31, 2018

    2018

  29. [30]

    Injeca- gent: Benchmarking indirect prompt injections in tool- integrated large language model agents,

    Q. Zhan, Z. Liang, Z. Ying, and D. Kang, “Injeca- gent: Benchmarking indirect prompt injections in tool- integrated large language model agents,” inFindings of the Association for Computational Linguistics: ACL 2024, 2024, pp. 10 471–10 506

    2024

  30. [31]

    For- malizing and benchmarking prompt injection attacks and defenses,

    Y . Liu, Y . Jia, R. Geng, J. Jia, and N. Z. Gong, “For- malizing and benchmarking prompt injection attacks and defenses,” in33rd USENIX Security Symposium (USENIX Security 24), 2024, pp. 1831–1847

    2024

  31. [32]

    Symbolic mixture-of-experts: Adaptive skill-based routing forheterogeneous reasoning,

    J. C.-Y . Chen, S. Yun, E. Stengel-Eskin, T. Chen, and M. Bansal, “Symbolic mixture-of-experts: Adaptive skill-based routing forheterogeneous reasoning,” https: //arxiv.org/pdf/2503.05641, 2025

    Pith/arXiv arXiv 2025

  32. [33]

    Adap- tive attacks break defenses against indirect prompt in- jection attacks on llm agents,

    Q. Zhan, R. Fang, H. S. Panchal, and D. Kang, “Adap- tive attacks break defenses against indirect prompt in- jection attacks on llm agents,” inFindings of the As- sociation for Computational Linguistics: NAACL 2025, 2025, pp. 7101–7117

    2025

  33. [34]

    Defending against indirect prompt injection attacks with spotlighting,

    K. Hines, G. Lopez, M. Hall, F. Zarfati, Y . Zunger, and E. Kiciman, “Defending against indirect prompt injection attacks with spotlighting,”arXiv preprint arXiv:2403.14720, 2024

    Pith/arXiv arXiv 2024

  34. [35]

    Securing ai agents with information-flow control,

    M. Costa, B. K ¨opf, A. Kolluri, A. Paverd, M. Russinovich, A. Salem, S. Tople, L. Wutschitz, and S. Zanella-B ´eguelin, “Securing ai agents with information-flow control,”arXiv preprint arXiv:2505.23643, 2025

    Pith/arXiv arXiv 2025

  35. [36]

    Training language models to follow instructions with human feedback,

    L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray et al., “Training language models to follow instructions with human feedback,”Advances in neural information processing systems, vol. 35, pp. 27 730–27 744, 2022

    2022

  36. [37]

    The instruction hierarchy: Training llms to prioritize privileged instructions,

    E. Wallace, K. Xiao, R. Leike, L. Weng, J. Heidecke, and A. Beutel, “The instruction hierarchy: Training llms to prioritize privileged instructions,”arXiv preprint arXiv:2404.13208, 2024

    Pith/arXiv arXiv 2024

  37. [38]

    Gradsafe: Detecting jailbreak prompts for llms via safety-critical gradient analysis,

    Y . Xie, M. Fang, R. Pi, and N. Gong, “Gradsafe: Detecting jailbreak prompts for llms via safety-critical gradient analysis,” inProceedings of the 62nd annual meeting of the association for computational linguistics (volume 1: Long papers), 2024, pp. 507–518

    2024

  38. [39]

    Embedding-based classifiers can detect prompt injection attacks,

    M. A. Ayub and S. Majumdar, “Embedding-based classifiers can detect prompt injection attacks,”arXiv preprint arXiv:2410.22284, 2024

    Pith/arXiv arXiv 2024

  39. [40]

    Improving alignment and robustness with circuit breakers,

    A. Zou, L. Phan, J. Wang, D. Duenas, M. Lin, M. An- driushchenko, R. Wang, Z. Kolter, M. Fredrikson, and D. Hendrycks, “Improving alignment and robustness with circuit breakers,”Advances in Neural Information Processing Systems, vol. 37, pp. 83 345–83 373, 2024

    2024

  40. [41]

    Get my drift? catching llm task drift with activation deltas,

    S. Abdelnabi, A. Fay, G. Cherubin, A. Salem, M. Fritz, and A. Paverd, “Get my drift? catching llm task drift with activation deltas,” in2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), 2025, pp. 43–67

    2025

  41. [42]

    Cacheprune: Neural- based attribution defense against indirect prompt injec- tion attacks,

    R. Wang, J. Wu, Y . Xia, T. Yu, R. Zhang, R. Rossi, S. Mitra, L. Yao, and J. McAuley, “Cacheprune: Neural- based attribution defense against indirect prompt injec- tion attacks,”arXiv preprint arXiv:2504.21228, 2025

    Pith/arXiv arXiv 2025

  42. [43]

    Defending against indirect prompt injection by instruction detection,

    T. Wen, C. Wang, X. Yang, H. Tang, Y . Xie, L. Lyu, Z. Dou, and F. Wu, “Defending against indirect prompt injection by instruction detection,” inFindings of the As- sociation for Computational Linguistics: EMNLP 2025, 2025, pp. 19 472–19 487

    2025

  43. [44]

    Secalign: Defending against prompt injection with preference optimization,

    S. Chen, A. Zharmagambetov, S. Mahloujifar, K. Chaudhuri, D. Wagner, and C. Guo, “Secalign: Defending against prompt injection with preference optimization,” inProceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security, 2025, pp. 2833–2847

    2025

  44. [45]

    System-level de- fense against indirect prompt injection attacks: An information flow control perspective,

    F. Wu, E. Cecchetti, and C. Xiao, “System-level de- fense against indirect prompt injection attacks: An information flow control perspective,”arXiv preprint arXiv:2409.19091, 2024

    Pith/arXiv arXiv 2024

  45. [46]

    Rtbas: Defending llm agents against prompt injection and privacy leakage,

    P. Y . Zhong, S. Chen, R. Wang, M. McCall, B. L. Titzer, H. Miller, and P. B. Gibbons, “Rtbas: Defending llm agents against prompt injection and privacy leakage,” arXiv preprint arXiv:2502.08966, 2025

    Pith/arXiv arXiv 2025

  46. [47]

    Defeating prompt injections by design,

    E. Debenedetti, I. Shumailov, T. Fan, J. Hayes, N. Car- lini, D. Fabian, C. Kern, C. Shi, A. Terzis, and F. Tram`er, “Defeating prompt injections by design,”arXiv preprint arXiv:2503.18813, 2025

    Pith/arXiv arXiv 2025

  47. [48]

    Prompt flow integrity to prevent privilege escalation in llm agents,

    J. Kim, W. Choi, and B. Lee, “Prompt flow integrity to prevent privilege escalation in llm agents,”arXiv preprint arXiv:2503.15547, 2025

    Pith/arXiv arXiv 2025

  48. [49]

    Injecguard: Benchmarking and mitigating over-defense in prompt injection guardrail models,

    H. Li and X. Liu, “Injecguard: Benchmarking and mitigating over-defense in prompt injection guardrail models,”arXiv preprint arXiv:2410.22770, 2024

    Pith/arXiv arXiv 2024

  49. [50]

    Promptarmor: Simple yet effective prompt injection defenses,

    T. Shi, K. Zhu, Z. Wang, Y . Jia, W. Cai, W. Liang, H. Wang, H. Alzahrani, J. Lu, K. Kawaguchiet al., “Promptarmor: Simple yet effective prompt injection defenses,”arXiv preprint arXiv:2507.15219, 2025

    Pith/arXiv arXiv 2025

  50. [51]

    Can indirect prompt injection attacks be detected and removed?

    Y . Chen, H. Li, Y . Sui, Y . He, Y . Liu, Y . Song, and B. Hooi, “Can indirect prompt injection attacks be detected and removed?” inProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2025, pp. 18 189– 18 206

    2025

  51. [52]

    Datasentinel: A game-theoretic detection of prompt injection attacks,

    Y . Liu, Y . Jia, J. Jia, D. Song, and N. Z. Gong, “Datasentinel: A game-theoretic detection of prompt injection attacks,” in2025 IEEE Symposium on Security and Privacy (SP). IEEE, 2025, pp. 2190–2208

    2025

  52. [53]

    Promptlocate: Localizing prompt injection attacks,

    Y . Jia, Y . Liu, Z. Shao, J. Jia, and N. Z. Gong, “Promptlocate: Localizing prompt injection attacks,” in 2026 IEEE Symposium on Security and Privacy (SP). IEEE, 2026, pp. 4243–4261

    2026

  53. [54]

    The task shield: Enforcing task alignment to defend against in- direct prompt injection in llm agents,

    F. Jia, T. Wu, X. Qin, and A. Squicciarini, “The task shield: Enforcing task alignment to defend against in- direct prompt injection in llm agents,” inProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2025, pp. 29 680–29 697

    2025

  54. [55]

    Melon: Provable defense against indirect prompt injec- tion attacks in ai agents,

    K. Zhu, X. Yang, J. Wang, W. Guo, and W. Y . Wang, “Melon: Provable defense against indirect prompt injec- tion attacks in ai agents,” inInternational Conference on 15 Machine Learning, 2025, pp. 80 310–80 329

    2025

  55. [56]

    Benchmarking and defending against indirect prompt injection attacks on large language models,

    J. Yi, Y . Xie, B. Zhu, E. Kiciman, G. Sun, X. Xie, and F. Wu, “Benchmarking and defending against indirect prompt injection attacks on large language models,” in Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V . 1, 2025, pp. 1809–1820

    2025

  56. [57]

    {StruQ}: Defending against prompt injection with structured queries,

    S. Chen, J. Piet, C. Sitawarin, and D. Wagner, “{StruQ}: Defending against prompt injection with structured queries,” in34th USENIX Security Symposium (USENIX Security 25), 2025, pp. 2383–2400

    2025

  57. [58]

    Attention is all you need to defend against in- direct prompt injection attacks in llms,

    Y . Zhong, Q. Miao, Y . Chen, J. Deng, Y . Cheng, and W. Xu, “Attention is all you need to defend against in- direct prompt injection attacks in llms,” in33rd Annual Network and Distributed System Security Symposium, 2026

    2026

  58. [59]

    Aside: Archi- tectural separation of instructions and data in language models,

    E. Zverev, E. Kortukov, A. Panfilov, S. Tabesh, S. La- puschkin, W. Samek, and C. H. Lampert, “Aside: Archi- tectural separation of instructions and data in language models,” inICLR 2025 Workshop on Building Trust in Language Models and Applications, 2025

    2025

  59. [60]

    Defense against indirect prompt injection via tool result parsing,

    Q. Yu, X. Cheng, and C. Liu, “Defense against indirect prompt injection via tool result parsing,”arXiv preprint arXiv:2601.04795, 2026

    arXiv 2026

  60. [61]

    Planguard: Defending agents against indirect prompt injection via planning- based consistency verification,

    G. Gong and Z. Deng, “Planguard: Defending agents against indirect prompt injection via planning- based consistency verification,”arXiv preprint arXiv:2604.10134, 2026

    Pith/arXiv arXiv 2026

  61. [62]

    Progent: Programmable privilege control for llm agents,

    T. Shi, J. He, Z. Wang, L. Wu, H. Li, W. Guo, and D. Song, “Progent: Programmable privilege control for llm agents,”arXiv e-prints, pp. arXiv–2504, 2025. APPENDIX A. Workflow Instantiation We instantiate the motivating workflow of Fig. 2 in the notation of §V-B. Control tools and data sources are Cctrl ={delegate,read,retrieve, submit,abort}, D={general l...

    2025