Pith. sign in

REVIEW 4 major objections 4 minor 55 references

Agent Against Agent: An Agentic System for Automatic Prompt Injection Red Teaming

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Prompt-injection attacks on LLM agents are usually generated by RL-trained attackers that transfer poorly, or by search-based attackers that start from scratch each time; this paper claims that a memory-augmented search agent, PIMiner…

desk verdict PIMiner has a genuinely new memory-loop architecture and strong numbers, but the test-time protocol looks grey-box despite the black-box claim, so the transferability result needs a cleaner black-box run. read the letter →

arxiv 2608.05108 v1 pith:ZMSFG7P6 submitted 2026-08-05 cs.CR

Yanting Wang , Chenlong Yin , Runpeng Geng , Jinyuan Jia This is my paper
classification cs.CR
keywords promptinjectionLLMagentsredteamingagenticsystemstrategylibraryattacksuccessratemodeltransfermemory-augmentedsearch
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

PIMiner is an agentic system for automatically red-teaming LLM agents against prompt injection. Its central claim is that a search-based attacker can be made as effective as reinforcement-learning-trained attackers, and much more transferable, by letting it accumulate past attack experience in a hierarchical memory and distill that experience into human-readable strategy files. During training the system processes a sequence of (dataset, target model) pairs, building a strategy library from scratch and refining it on successes and failures; at test time the library is applied to a previously unseen target model without further attacker training. On the IPIArena and AgentDojo benchmarks, PIMiner reaches 40–87% attack success rates at ten queries per sample across Gemini-2.5-Pro, GPT-5.1, and Claude-Sonnet-4.5, and on InjecAgent it matches the best RL-based baselines. If the claim holds, prompt-injection red teaming no longer needs a per-target RL training run to be competitive, and the attack knowledge itself becomes inspectable.

What carries the argument

The load-bearing mechanism is a three-level hierarchical memory with a router. Long-term memory is the strategy library: Markdown files, grown from a single template, each recording the target-LLM scope, user-task and injected-task scope, a general injection template, in-context examples, a fingerprint table, and documented failure conditions. Mid-term memory is intra-dataset memory: digester-curated summaries of previously attacked samples within the same (dataset, target model) pair. Short-term memory is intra-sample memory: the accumulated prompts, target trajectories, success or failure judgments, and attacker analyses for the current sample. A router agent reads only each strategy's routing summary and picks the top-K strategies for a sample, keeping the attacker's context compact; the digester then classifies each hit as fitting, widening, or novel relative to the library and updates the library accordingly, so even runs with no successes refine failure conditions. The core object is therefore the strategy library itself: it is the artifact that converts search history into reusable, transferable, human-readable attack knowledge.

What would settle it

Run PIMiner in exactly the same configuration but train with only black-box final outputs, withholding all intermediate tool calls and per-step observations, on IPIArena and AgentDojo. If the attack success rates fall to roughly the 13% and 51% ASRs the paper reports for a PAIR-style iterative attacker with weak and strong backbones, the grey-box training assumption is what carries the reported 40–87% results. A second check: re-train with a weak attacker backbone instead of the strong default and see whether the strategy library still produces high ASRs on unseen targets.

Watch

Extended reading notes

Core claim

PIMiner's central claim is that the gap between search-based and RL-based prompt-injection red teaming is not a gap in search power but a gap in memory: search-based attackers restart from scratch for every sample, while RL attackers implicitly accumulate attack knowledge during training. PIMiner makes that accumulation explicit and inspectable. It trains on a sequence of (dataset, target model) pairs, uses an iterative attacker that refines a prompt for up to ten attempts per sample, and then runs a digester agent that converts wins and losses into a Markdown strategy library with scopes, templates, in-context examples, and failure conditions. At test time, a router selects the top-K relevant strategies per sample, and the attacker combines the routed strategies with intra-dataset memory (summaries of earlier samples in the same pair) and intra-sample memory (the current sample's full attempt history). The reported result is that on IPIArena and AgentDojo this memory-augmented search agent reaches 76.2% and 86.7% ASR against Gemini-2.5-Pro, 61.9% and 53.3% against GPT-5.1, and 42.9% and 40.0% against Claude-Sonnet-4.5, while on InjecAgent it matches the RL baselines at 1.0 ASR on all three target models; on AgentDojo it also beats PISmith on GPT-5-nano (0.53 vs 0.38). The authors conclude that PIMiner effectively closes the performance gap between search-based and RL-based red-teaming methods without training a separate attacker per target.

Load-bearing premise

The reported attack rates assume that during training the attacker can observe the target agent's output at every step, including the step right after the injection, because a simulated environment can closely resemble the real target; if a real target exposes only its final answer, the iterative diagnosis and cross-sample memory that drive PIMiner's results would lose most of their signal.

Editorial extensions

If this is right

  • Prompt-injection red teaming becomes a cumulative process: a team that has trained PIMiner once can apply its strategy library to a new target model directly, without running an RL fine-tuning job per target.
  • Because the library artifacts are human-readable, they can serve as documentation of attack surface and as seed material for generating defense-training data.
  • Because the library transfers across attacker LLMs, users can train with a strong backbone and test with a cheap one; the paper reports PAIR's ASR rising from 0.14 to 0.52 on Gemini-2.5-Flash when given PIMiner's strategies.
  • On InjecAgent, PIMiner reaches ASR@10 of 1.0 on GPT-4o-mini, GPT-4.1-nano, and GPT-5-nano, matching RL-Hammer and PISmith while avoiding target-specific attacker training.
  • Training cost is roughly $20 of out-of-pocket API spend given a Claude Code subscription, plus target-agent queries, and test-time cost scales with the per-sample iteration budget, making frontier-model red teaming affordable at scale.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: if grey-box trajectory access is the main driver, then agents that expose intermediate traces, such as coding CLIs that print tool calls, are systematically more vulnerable to this class of automated attack, implying a transparency-security trade-off the paper does not quantify.
  • Inference: the strategy library's fingerprint tables and failure conditions could be repurposed as detection features for prompt-injection defense, since each strategy names its structural move and its failure modes; PIMiner itself only uses this knowledge offensively.
  • Inference: the same accumulate-and-distill memory loop should apply to other agent red-teaming goals with trajectory feedback, such as tool-abuse or data-exfiltration testing, because nothing in the mechanism is specific to injection text.
  • Inference: a natural untested extension is to use the strategy library to seed or reward-shape RL attackers, potentially cutting the tens of thousands of queries RL methods need; the paper compares against RL but does not combine the two.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces PIMiner, an agentic prompt-injection red-teaming system built around a hierarchical memory mechanism: a long-term strategy library, an intra-dataset memory, and an intra-sample memory, together with a strategy router, an iterative attack module, and an experience digester. During training, PIMiner is applied to a sequence of (dataset, target-model) pairs and builds the strategy library from scratch; at test time it uses the frozen library to attack previously unseen target models with a small query budget per sample. The authors report strong ASRs on IPIArena and AgentDojo across several frontier LLMs, compare against static, search-based, and RL-based baselines, ablate the memory components, and show that the learned strategies transfer across attacker LLMs.

Significance. If the claims hold, the paper makes a useful contribution: it shows that a memory-augmented search-based agent can approach the effectiveness of RL-based prompt-injection red-teaming while being cheaper, more interpretable, and directly transferable to new target models. The system design is described in unusual detail, the strategy library is human-readable and inspectable, and the evaluation uses public benchmarks with a released codebase. The main risks are that the black-box test-time protocol is stated but not evidently followed, and that the headline comparisons rest on small samples with no variance estimation. These issues are fixable and do not invalidate the core idea, but they need to be addressed before the central claims can be accepted.

major comments (4)
  1. [§3, Eq. (3), Appendix D] The paper states in Section 3 that during testing 'the attacker only has access to the agent's final output and a ground-truth indicator of whether the attack succeeds,' but Eq. (3) stores 'target-agent trajectories' in the intra-sample memory, and the per-iteration attacker prompt in Appendix D includes 'Target trajectory: {{observed_trajectory}}' on every attempt. The manuscript never specifies what {{observed_trajectory}} contains in the test harness. If it includes intermediate tool calls, observations, or reasoning, then the reported ASRs in Figure 2 and Tables 1 and 2 were obtained with grey-box test-time feedback, not the stated black-box protocol. If it contains only the final output, then several failure-mode diagnoses described in Appendix F (e.g., distinguishing a discarded role marker from an emission block) cannot be performed as described. Please define the exact content of the observed trajectory in the test-time harness and, to support the abstract's black-box claim, report a run in which the attacker receives only the final output plus the binary success indicator.
  2. [§5.2, Figure 2, Tables 1 and 2] The headline ASRs are computed from n=21 test samples on IPIArena and n=30 on AgentDojo, with no confidence intervals, no repeated runs, and no account of the stochasticity of the LLM-based router, attacker, and digester. On those sample sizes the differences between many of the reported numbers are within one or two sample flips: for example, Claude-Opus-4.5's 4.8% (1/21) and 3.3% (1/30) ASRs are statistically indistinguishable, and the Section 5.3 claim that PIMiner is 'comparable' to PISmith on AgentDojo rests on 0.73 vs. 0.78 and 0.63 vs. 0.81. Please report at least binomial 95% confidence intervals and, ideally, multiple independent runs, and soften the strength of the comparative claims until those are provided.
  3. [§5.3, Tables 1 and 2] The RL baselines are not re-run but 'directly use the evaluation results from PISmith [17]'. This makes the comparison difficult to interpret unless the PISmith configuration exactly matches the PIMiner evaluation in target-model versions, benchmark samples, success criteria, and query budget. Since PISmith is one of the authors' own prior systems, the comparison also needs a statement of what changed between the two evaluations. Please either re-run the baselines under the same harness or report the precise PISmith protocol and per-sample results for the specific test splits used here.
  4. [§5.4, Figure 3] The ablation excludes Claude-Opus-4.5 after the fact, with the justification that its ASR is low and therefore 'less informative for comparing design choices.' This means the claim that both long-term and intra-dataset memory are essential is demonstrated only on a favorable subset of four target models and only on IPIArena. Please report the per-model ablation results, including the excluded model or an explicit justification for why the ablation claim should not be expected to hold there.
minor comments (4)
  1. [Figure 2] The ASR@1, ASR@5, and ASR@10 values in the two bar panels are difficult to distinguish because the three bars are visually close; different markers or a table with exact proportions would make the results easier to read.
  2. [§5.1 vs. §5.4] The paper refers to 'Claude-Sonnet-4.6' in the ablation section but to 'Claude-Sonnet-4.5' elsewhere; please clarify whether these are intended to be the same model or different model versions.
  3. [§5.1] The definition of ASR@N is clear, but the abstract and Section 5.1 emphasize the 10-query budget while Figure 2 also reports ASR@1 and ASR@5; please state explicitly which metric is the primary one for the transferability claim.
  4. [§4.2, Eq. (4)] The intra-dataset memory is curated only from samples that finish earlier in the same dataset–model pair, and the rolling parallelization means later samples see more experience; the paper should state whether the reported ASRs are sensitive to sample ordering and pool size.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: PIMiner's ASR is measured on held-out public benchmark samples from real target responses; the strategy library is built from prior experience and not fitted to test-set items.

full rationale

The derivation chain is self-contained: PIMiner trains by attacking training splits of IPIArena/AgentDojo, digesting successes and failures into a strategy library, and then evaluates with that library frozen on separate test splits and unseen target LLMs; ASR@10 is computed from target-agent replies, not from any quantity that PIMiner's optimization defines. Equation (3) includes target-agent trajectories in memory, but this is feedback for iterative attack refinement, not a fitted parameter later renamed a prediction. The only notable self-citation is Section 5.1's use of PISmith's published baseline numbers ('we directly use the evaluation results from PISmith [17] for the baselines'); because PIMiner's own ASRs are independently measured on public benchmarks, this self-citation affects comparison strength but is not load-bearing for PIMiner's central effectiveness claim. A separate threat-model concern is that Section 3 promises test-time black-box access ('the attacker only has access to the agent's final output') while Eq. (3) and Appendix D's prompt template feed 'Target trajectory: {{observed_trajectory}}'; if intermediate steps are included, the reported numbers may not be strictly black-box. That is a validity/threat-model issue, not a circular derivation, so it does not raise the circularity score.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The central claim rests on the grey-box training assumption, the fidelity of the simulated environment, and the reliability of the router and digester agents. The hand-chosen hyperparameters (K, Nmax, pool size, clip length, backbone) are not fitted to the test targets but they do shape the reported ASR. No new physical or mathematical entities are introduced.

free parameters (6)
  • Top-K router count = 3
    Chosen by hand; smaller K reduces context length but risks missing the best strategy.
  • Nmax (max iterations per sample) = 10
    Chosen by hand; ASR@10 is the reported metric, so this sets the query budget and the headline numbers.
  • Rolling pool size = 5
    Chosen by hand; controls how much intra-dataset memory is available to later samples.
  • Intra-dataset memory clip length = 20000 chars
    Chosen by hand; limits attacker context length.
  • Backbone LLM = Claude-Opus-4.7
    Chosen by hand; training uses xhigh reasoning and testing uses low; effectiveness depends on this strong backbone.
  • Number of training pairs = 8
    Chosen by hand; the strategy library is built from these 8 (dataset, model) pairs.
assumptions (6)
  • domain assumption Grey-box training access to the target agent
    Section 3 states the attacker observes the target's output at each agent step during training, justified by open-sourced agents. If false, trajectory-based memories are unavailable.
  • domain assumption Simulated environment fidelity
    Section 3 assumes the attacker can build a simulated environment resembling the target agent; transfer to closed commercial agents is untested.
  • domain assumption Benchmark success indicators are correct
    ASR is computed from benchmark ground-truth labels; errors or label noise would propagate.
  • domain assumption Router summary sufficiency
    Section 4.2 assumes the routing summary (scopes plus examples) lets the router pick top-K strategies; a poor selection degrades the attacker.
  • domain assumption Digester classification reliability
    Appendix E assumes the digester correctly classifies hits as fits-as-is, fits-but-widens, or novel; misclassification can create misleading strategy files.
  • domain assumption Claude Code execution reliability
    Implementation section assumes claude -p sessions and the Python harness correctly execute the orchestration; failures would affect results.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Agent Against Agent: An Agentic System for Automatic Prompt Injection Red Teaming." pith.science (2026). https://pith.science/paper/ZMSFG7P6

@misc{pith2026260805108,
  author       = {Pith},
  title        = {Pith review of: Agent Against Agent: An Agentic System for Automatic Prompt Injection Red Teaming},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZMSFG7P6}},
  note         = {Machine review of arXiv:2608.05108}
}
read the original abstract

Prompt injection poses significant security risks to LLM agents. Efficient and effective red-teaming is therefore critical, both for evaluating these risks and for collecting training data to improve defenses. Existing state-of-the-art prompt injection red-teaming methods primarily rely on reinforcement learning (RL), producing attacker models that often generalize poorly to new target LLMs. In this work, we develop PIMiner, an agentic system for prompt injection red-teaming. During training, PIMiner is trained on a sequence of (dataset, target model) pairs and builds a strategy library from scratch. At test time, the learned strategy library can be directly transferred to a previously unseen target LLM without additional training. PIMiner requires only a small number of queries to a target agent (e.g., 10) per test sample. Experimental results demonstrate that PIMiner achieves strong performance. On IPIArena, it attains a 76.2% ASR against Gemini-2.5-Pro, 61.9% ASR against GPT-5.1, and 42.9% ASR against Claude-Sonnet-4.5. On AgentDojo, it achieves an 86.7% ASR against Gemini-2.5-Pro, 53.3% ASR against GPT-5.1, and 40.0% ASR against Claude-Sonnet-4.5.

Figures

Figures reproduced from arXiv: 2608.05108 by the authors.

Figure 1
Figure 1. PIMiner Pipeline. Given a dataset–model pair, PIMiner routes each sample to the most relevant strategies, performs iterative attack optimization, and then digests the resulting experiences to update its strategy library. During iterative attack optimization, an intra-dataset memory is maintained to facilitate knowledge sharing across samples within the same dataset. attack examples, failure cases, and model (or data… view at source ↗
Figure 2
Figure 2. Evaluation of PIMiner on 9 frontier LLMs on IPIArena [23] and AgentDojo [5]. We report ASR@1, ASR@5, and ASR@10. close behind on both benchmarks (76.2% and 86.7% ASR@10, respectively). In contrast, Claude-Opus-4.5 demonstrates by far the strongest robustness, with only 4.8% ASR@10 on IPIArena and 3.3% ASR@10 on AgentDojo. We further observe that model capability generally correlates with prompt injection robustness … view at source ↗
Figure 3
Figure 3. Ablation for the memory mechanism. w/o Strategy Lib. removes the strategies from the attacker agent’s input, w/o Intra-dataset Mem. removes the intra-dataset memory, and Vanilla Iter. Atk. removes both, leaving only the intra-sample memory. 5.5 Transferability of the Strategy Library Across Attacker LLMs We show that the strategy library learned by PIMiner improves attack performance even when used with attacker LLM… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: The strategy library learned by PIMiner improves attack performance for a wide range of attacker LLMs. Haiku 4.5 Sonnet 4.6 PIMiner backbone LLM 0 50 100 150 200 250 300 350 400 Avg. attacker input len.(tokens) 237K 236K 136K 92K w/o Router (Full Lib.) w/ Router (PIMin…
Figure 5
Figure 5. Figure 5: Comparison of PIMiner with and without the router, in terms of the attacker agent’s average input length (a) and the resulting ASR (b) [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: Distribution of in-context examples from the seven distinct attack strategies across different target LLMs. several others remain highly model-specific, having been discovered and refined against a single target model. A Close Look at the Learned Strategies: Here, we p…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

55 extracted references · 31 canonical work pages

  1. [17]

    Pismith: Reinforcement learning-based red teaming for prompt injection defenses,

    C. Yin, R. Geng, Y. Wang, and J. Jia, “Pismith: Reinforcement learning-based red teaming for prompt injection defenses, ”arXiv preprint arXiv:2603.13026, 2026

    arXiv 2026

  2. [1]

    Ignore previous prompt: Attack techniques for language models,

    F. Perez and I. Ribeiro, “Ignore previous prompt: Attack techniques for language models, ” inNeurIPS ML Safety Workshop, 2022

    2022

  3. [2]

    Not what you’ve signed up for: Compromising real-world llm-integrated applications with indirect prompt injection,

    K. Greshake, S. Abdelnabi, S. Mishra, C. Endres, T. Holz, and M. Fritz, “Not what you’ve signed up for: Compromising real-world llm-integrated applications with indirect prompt injection, ” inProceedings of the 16th ACM workshop on artificial intelligence and security, 2023

    work page 2023

  4. [3]

    Formalizing and benchmarking prompt injection attacks and defenses,

    Y. Liu, Y. Jia, R. Geng, J. Jia, and N. Z. Gong, “Formalizing and benchmarking prompt injection attacks and defenses, ” inUSENIX Security, 2024

    work page 2024

  5. [4]

    Injecagent: Benchmarking indirect prompt injections in tool- integrated large language model agents,

    Q. Zhan, Z. Liang, Z. Ying, and D. Kang, “Injecagent: Benchmarking indirect prompt injections in tool- integrated large language model agents, ” inFindings of the Association for Computational Linguistics: ACL 2024, 2024, pp. 10 471–10 506

    work page 2024

  6. [5]

    Agentdojo: A dynamic environment to evaluate prompt injection attacks and defenses for llm agents,

    E. Debenedetti, J. Zhang, M. Balunovic, L. Beurer-Kellner, M. Fischer, and F. Tramèr, “Agentdojo: A dynamic environment to evaluate prompt injection attacks and defenses for llm agents, ”NeurIPS, 2024

    work page 2024

  7. [6]

    The attacker moves second: Stronger adaptive attacks bypass defenses against llm jailbreaks and prompt injections,

    M. Nasr, N. Carlini, C. Sitawarin, S. V. Schulhoff, J. Hayes, M. Ilie, J. Pluto, S. Song, H. Chaudhari, I. Shumailovet al., “The attacker moves second: Stronger adaptive attacks bypass defenses against llm jailbreaks and prompt injections, ”arXiv preprint arXiv:2510.09023, 2025

    arXiv

  8. [7]

    Muse spark safety & preparedness report,

    Meta Superintelligence Labs, “Muse spark safety & preparedness report, ” Meta Platforms, Inc., Tech. Rep., Apr. 2026, accessed: 2026-05-29. [Online]. Available: https://ai.meta.com/static-resource/ muse-spark-safety-and-preparedness-report/

    work page 2026

Show all 55 references
  1. [8]

    Claude opus 4.7 system card,

    Anthropic, “Claude opus 4.7 system card, ”Anthropic Research, 2026, accessed: 2026-05-29. [Online]. Available: https://cdn.sanity.io/files/4zrzovbb/website/037f06850df7fbe871e206dad004c3db5fd50340. pdf

    2026

  2. [9]

    Datasentinel: A game-theoretic detection of prompt injection attacks,

    Y. Liu, Y. Jia, J. Jia, D. Song, and N. Z. Gong, “Datasentinel: A game-theoretic detection of prompt injection attacks, ” inIEEE Symposium on Security and Privacy, 2025. Preprint version

    2025

  3. [10]

    PromptGuard Prompt Injection Guardrail,

    Meta, “PromptGuard Prompt Injection Guardrail, ” https://www.llama.com/docs/ model-cards-and-prompt-formats/prompt-guard/, 2024

    2024

  4. [11]

    Agentwatcher: A rule-based prompt injection monitor,

    Y. Wang, W. Zou, R. Geng, and J. Jia, “Agentwatcher: A rule-based prompt injection monitor, ”arXiv preprint arXiv:2604.01194, 2026

    2026 arXiv

  5. [12]

    Meta secalign: A secure foundation llm against prompt injection attacks,

    S. Chen, A. Zharmagambetov, D. Wagner, and C. Guo, “Meta secalign: A secure foundation llm against prompt injection attacks, ”arXiv preprint arXiv:2507.02735, 2025

    2025

  6. [13]

    Purple-teaming llms with adversarial defender training,

    J. Zhou, K. Li, J. Li, J. Kang, M. Hu, X. Wu, and H. Meng, “Purple-teaming llms with adversarial defender training, ”arXiv preprint arXiv:2407.01850, 2024

    2024 arXiv

  7. [14]

    Black-box red-teaming of multi-agent systems via reinforcement learning,

    Y. Jia, H. Zhang, C. Liu, J. Liang, and X. Chen, “Black-box red-teaming of multi-agent systems via reinforcement learning, ” 2026

    2026

  8. [15]

    Learning to inject: Automated prompt injection via reinforcement learning,

    X. Chen, J. Zhang, and F. Tramer, “Learning to inject: Automated prompt injection via reinforcement learning, ”arXiv preprint arXiv:2602.05746, 2026

    2026 arXiv

  9. [16]

    Rl is a hammer and llms are nails: A simple reinforcement learning recipe for strong prompt injection,

    Y. Wen, A. Zharmagambetov, I. Evtimov, N. Kokhlikyan, T. Goldstein, K. Chaudhuri, and C. Guo, “Rl is a hammer and llms are nails: A simple reinforcement learning recipe for strong prompt injection, ” arXiv preprint arXiv:2510.04885, 2025

  10. [18]

    Tree of attacks: Jailbreaking black-box llms automatically,

    A. Mehrotra, M. Zampetakis, P. Kassianik, B. Nelson, H. Anderson, Y. Singer, and A. Karbasi, “Tree of attacks: Jailbreaking black-box llms automatically, ”Advances in Neural Information Processing Systems, vol. 37, pp. 61 065–61 105, 2024

    2024

  11. [19]

    Jailbreaking black box large language models in twenty queries,

    P. Chao, A. Robey, E. Dobriban, H. Hassani, G. J. Pappas, and E. Wong, “Jailbreaking black box large language models in twenty queries, ” in2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML). IEEE, 2025, pp. 23–42

    2025

  12. [20]

    Agentvigil: Automatic black-box red-teaming for indirect prompt injection against llm agents,

    Z. Wang, V. Siu, Z. Ye, T. Shi, Y. Nie, X. Zhao, C. Wang, W. Guo, and D. Song, “Agentvigil: Automatic black-box red-teaming for indirect prompt injection against llm agents, ”Findings of the Association for Computational Linguistics: EMNLP 2025, 2025

    2025

  13. [21]

    Piarena: A platform for prompt injection evaluation,

    R. Geng, C. Yin, Y. Wang, Y. Chen, and J. Jia, “Piarena: A platform for prompt injection evaluation, ” https://github.com/sleeepeer/PIArena, 2026

    2026

  14. [22]

    Gpt-red: Automated red teaming via self-play at scale

    E. Wallace, C. A. Choquette-Choo, N. Kandpal, S. Toyer, D. Hunn, S. Lin, Y. Wen, X. Q. C. W. Z. Wang, M. N. S. Z. C. Guo, J. F. C. U. K. Wanget al., “Gpt-red: Automated red teaming via self-play at scale. ”

  15. [23]

    How vulnerable are ai agents to indirect prompt injections? insights from a large-scale public competition,

    M. Dziemian, M. Lin, X. Fu, M. Nowak, N. Winter, E. Jones, A. Zou, L. Ahmad, K. Chaudhuri, S. Chennabasappaet al., “How vulnerable are ai agents to indirect prompt injections? insights from a large-scale public competition, ”arXiv preprint arXiv:2603.15714, 2026

    2026

  16. [24]

    Muzzle: Adaptive agentic red-teaming of web agents against indirect prompt injection attacks,

    G. Syros, E. Rose, B. Grinstead, C. Kerschbaumer, W. Robertson, C. Nita-Rotaru, and A. Oprea, “Muzzle: Adaptive agentic red-teaming of web agents against indirect prompt injection attacks, ”arXiv preprint arXiv:2602.09222, 2026

    2026

  17. [25]

    Autodan- turbo: A lifelong agent for strategy self-exploration to jailbreak llms,

    X. Liu, P. Li, G. E. Suh, Y. Vorobeychik, Z. Mao, S. Jha, P. McDaniel, H. Sun, B. Li, and C. Xiao, “Autodan- turbo: A lifelong agent for strategy self-exploration to jailbreak llms, ” inInternational Conference on Learning Representations, vol. 2025, 2025, pp. 10 313–10 360

    2025

  18. [26]

    Decodingtrust- agent platform (dtap): A controllable and interactive red-teaming platform for ai agents,

    Z. Chen, X. Liu, H. Tong, C. Guo, Y. Nie, J. Zhang, M. Kang, C. Xu, Q. Liu, X. Liuet al., “Decodingtrust- agent platform (dtap): A controllable and interactive red-teaming platform for ai agents, ”arXiv preprint arXiv:2605.04808, 2026. Preprint version

    2026 arXiv

  19. [27]

    React: Synergizing reasoning and acting in language models,

    S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao, “React: Synergizing reasoning and acting in language models, ”arXiv preprint arXiv:2210.03629, 2022

    2022 arXiv

  20. [28]

    Toolllm: Facilitating large language models to master 16000+ real-world apis,

    Y. Qin, S. Liang, Y. Ye, K. Zhu, L. Yan, Y. Lu, Y. Lin, X. Cong, X. Tang, B. Qianet al., “Toolllm: Facilitating large language models to master 16000+ real-world apis, ” inInternational Conference on Learning Representations, vol. 2024, 2024, pp. 9695–9717

    2024

  21. [29]

    Autoharness: improving llm agents by automatically synthesizing a code harness,

    X. Lou, M. Lázaro-Gredilla, A. Dedieu, C. Wendelken, W. Lehrach, and K. P. Murphy, “Autoharness: improving llm agents by automatically synthesizing a code harness, ”arXiv preprint arXiv:2603.03329, 2026

    2026

  22. [30]

    Multi-agent architecture search via agentic supernet,

    G. Zhang, L. Niu, J. Fang, K. Wang, L. Bai, and X. Wang, “Multi-agent architecture search via agentic supernet, ”arXiv preprint arXiv:2502.04180, 2025

    2025 arXiv

  23. [31]

    Textgrad: Automatic

    M. Yuksekgonul, F. Bianchi, J. Boen, S. Liu, Z. Huang, C. Guestrin, and J. Zou, “Textgrad: Automatic" differentiation" via text, ”arXiv preprint arXiv:2406.07496, 2024

    2024 arXiv

  24. [32]

    Test-time training with self-supervision for generalization under distribution shifts,

    Y. Sun, X. Wang, Z. Liu, J. Miller, A. Efros, and M. Hardt, “Test-time training with self-supervision for generalization under distribution shifts, ” inInternational conference on machine learning. PMLR, 2020, pp. 9229–9248

    2020

  25. [33]

    Gemini CLI: An open-source AI agent for the terminal,

    Google, “Gemini CLI: An open-source AI agent for the terminal, ” https://github.com/google-gemini/ gemini-cli, 2025

    2025

  26. [34]

    Codex CLI: A lightweight coding agent that runs in your terminal,

    OpenAI, “Codex CLI: A lightweight coding agent that runs in your terminal, ” https://github.com/ openai/codex, 2025

    2025

  27. [35]

    Claw code: A clean-room open-source coding-agent CLI,

    S. Jin, “Claw code: A clean-room open-source coding-agent CLI, ” https://github.com/instructkr/ claw-code, Mar. 2026, maintained under theinstructkrorganization

    2026

  28. [36]

    Hermes agent: An open-source self-hosted autonomous AI agent,

    Nous Research, “Hermes agent: An open-source self-hosted autonomous AI agent, ” https://github. com/NousResearch/hermes-agent, 2026

    2026

  29. [37]

    Claude code: An agentic coding tool for the terminal,

    Anthropic, “Claude code: An agentic coding tool for the terminal, ” https://docs.anthropic.com/en/ docs/claude-code/overview, 2025

    2025

  30. [38]

    Attention tracker: Detecting prompt injection attacks in llms,

    K.-H. Hung, C.-Y. Ko, A. Rawat, I.-H. Chung, W. H. Hsu, and P.-Y. Chen, “Attention tracker: Detecting prompt injection attacks in llms, ” inFindings of the Association for Computational Linguistics: NAACL 2025, 2025

    2025

  31. [39]

    Piguard: Prompt injection guardrail via mitigating overdefense for free,

    H. Li, X. Liu, N. Zhang, and C. Xiao, “Piguard: Prompt injection guardrail via mitigating overdefense for free, ” inACL, 2025

    2025

  32. [40]

    Pishield: Detecting prompt injection attacks via intrinsic llm features,

    W. Zou, Y. Liu, Y. Wang, Y. Chen, N. Gong, and J. Jia, “Pishield: Detecting prompt injection attacks via intrinsic llm features, ”arXiv preprint arXiv:2510.14005, 2025

    2025

  33. [41]

    Pisanitizer: Preventing prompt injection to long-context llms via prompt sanitization,

    R. Geng, Y. Wang, C. Yin, M. Cheng, Y. Chen, and J. Jia, “Pisanitizer: Preventing prompt injection to long-context llms via prompt sanitization, ”arXiv preprint arXiv:2511.10720, 2025

    2025

  34. [42]

    Promptarmor: Simple yet effective prompt injection defenses,

    T. Shi, K. Zhu, Z. Wang, Y. Jia, W. Cai, W. Liang, H. Wang, H. Alzahrani, J. Lu, K. Kawaguchiet al., “Promptarmor: Simple yet effective prompt injection defenses, ”arXiv preprint arXiv:2507.15219, 2025

    2025 arXiv

  35. [43]

    Defending against prompt injection with datafilter,

    Y. Wang, S. Chen, R. Alkhudair, B. Alomair, and D. Wagner, “Defending against prompt injection with datafilter, ”arXiv preprint arXiv:2510.19207, 2025

    2025

  36. [44]

    Defeating prompt injections by design,

    E. Debenedetti, I. Shumailov, T. Fan, J. Hayes, N. Carlini, D. Fabian, C. Kern, C. Shi, A. Terzis, and F. Tramèr, “Defeating prompt injections by design, ”arXiv preprint arXiv:2503.18813, 2025

    2025 arXiv

  37. [45]

    Drift: Dynamic rule-based defense with injection isolation for securing llm agents,

    H. Li, X. Liu, H.-C. Chiu, D. Li, N. Zhang, and C. Xiao, “Drift: Dynamic rule-based defense with injection isolation for securing llm agents, ”NeurIPS, 2025. Preprint version

    2025

  38. [46]

    Agentdyn: A dynamic open-ended benchmark for evalu- ating prompt injection attacks of real-world agent security system,

    H. Li, R. Wen, S. Shi, N. Zhang, and C. Xiao, “Agentdyn: A dynamic open-ended benchmark for evalu- ating prompt injection attacks of real-world agent security system, ”arXiv preprint arXiv:2602.03117, 2026

    2026 arXiv

  39. [47]

    Claude opus 4.8 system card,

    Anthropic, “Claude opus 4.8 system card, ” Anthropic, Tech. Rep., May 2026. [Online]. Available: https://www.anthropic.com/claude-opus-4-8-system-card

    2026

  40. [48]

    Secalign: Defend- ing against prompt injection with preference optimization,

    S. Chen, A. Zharmagambetov, S. Mahloujifar, K. Chaudhuri, D. Wagner, and C. Guo, “Secalign: Defend- ing against prompt injection with preference optimization, ” inProceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security, 2025, pp. 2833–2847

    2025

  41. [49]

    The instruction hierarchy: Training llms to prioritize privileged instructions,

    E. Wallace, K. Xiao, R. Leike, L. Weng, J. Heidecke, and A. Beutel, “The instruction hierarchy: Training llms to prioritize privileged instructions, ”arXiv preprint arXiv:2404.13208, 2024. Preprint version DeepSeek-V4-FlashGemini-2.5-Flash Gemini-3.1-Flash-Lite GPT-5-miniGPT-5...

    2024 arXiv

  42. [50]

    Fetch the run's hits and existing-strategy summaries: python iterative_attack_orchestrator/iterative_attack_claude_code.py digest --run-dir $ARGUMENTS

  43. [51]

    Read ALL hits before classifying any, so shared mechanisms across hits cluster correctly

  44. [52]

    Read each existing strategy file in full (the fetch shows only the head)

  45. [53]

    Classify each hit by its core MECHANISM (the structural move that did the work), not its surface form. Decide one of three (prefer earlier options): (a) fits-as-is (mechanism AND application surface already covered by a strategy) (b) fits-but-widens (mechanism matches, but the...

  46. [54]

    ## In-context examples

    Apply the decision: (a) Append a new example block to the strategy's canonical "## In-context examples" section: ### Example N -- <suite>/<user_task>/<injection_task> (target: <target_model>) Paste the FULL verbatim polluted context (with the {INJECTION} token), the winning in...

  47. [55]

    ## When this strategy is expected to fail

    Miss-pattern analysis (ALWAYS, even on zero-hit runs). Read the run's misses: - If a strategy predicted hits on samples that missed, downgrade its scope wording. - If misses cluster on a structural pattern not already described, add a bullet to that strategy's "## When this st...

    2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.