Pith. sign in

REVIEW 3 major objections 4 minor 69 references

Write-time memory filters stop direct attacks but miss composed and dormant poisonings in LLM agents.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

T0 review · deepseek-v4-flash

2026-08-02 01:29 UTC pith:KU3NX6WM

load-bearing objection A serious benchmark with a load-bearing sign error: the MID 'interaction' metric is mislabeled, and the L2 compositional story collapses on the paper's own numbers. the 3 major comments →

arxiv 2607.14651 v1 pith:KU3NX6WM submitted 2026-07-16 cs.CR cs.AI

MemPoison: Uncovering Persistent Memory Threats and Structural Blind Spots in LLM Agents

classification cs.CR cs.AI
keywords persistent memory poisoningLLM agentswrite-time defensescompositional attackscontext-triggered dormant attacksMechanistic Influence Decompositiondefense frontierbehavioral corruption rate
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that the security of an LLM agent's persistent memory cannot be decided at the moment a memory is written. It introduces a three-level threat ladder: a single harmful record (L1), a harmful conclusion split across several innocent-looking records that becomes dangerous only when retrieved together (L2), and a dormant instruction that activates only under a later natural trigger (L3). Across 1,227 hand-validated cases and ten model families, the paper measures that write-time defenses suppress L1 but leave L2 and L3 largely intact: the strongest pure write-time consistency check cut L1 corruption to about 5% while L2 and L3 stayed near 23% and 28%. A combined pipeline reduced overall corruption to about 11% but still left residual L2/L3 failures. If correct, agent-memory defenses must become retrieval- and context-aware rather than static filters at write time.

Core claim

On the paper's own terms, MemPoison establishes a defense frontier: write-time admission filters that check consistency, plausibility, or novelty can block directly harmful memory records (L1), but cannot reliably block attacks whose harm is deferred. L2 attacks partition the payload across individually benign records, so each write passes local checks; L3 records are inert until a natural trigger arrives. Mechanistic Influence Decomposition (MID), a counterfactual removal diagnostic over the retrieved memory set, shows why: L1 corruption is carried by a single record (poisoned-to-clean flip 95.7%), L2 by a non-additive joint effect between fragments (interaction signal 0.176), and L3 by a c

What carries the argument

Three named pieces carry the argument: the L1-L2-L3 taxonomy, which defines attack difficulty by the structure of the stored memory rather than by textual obfuscation; the benchmark protocol with paired clean/poisoned trigger tasks and Behavioral Corruption Rate (BCR) as the outcome metric; and MID, which removes the designated poison record or fragment set and measures the drop in poisoned-match score. MID's three statistics - single-record influence delta-s, pairwise interaction Omega-g, and ActivationShift - convert the abstract claim that harm can be deferred into measurable causal signatures. The pipeline audit then decomposes each poisoned case into write-blocked, admitted-but-not-retr

Load-bearing premise

The benchmark retains only attacks that human reviewers judged benign on the surface with non-leaky triggers, so the qualitative finding that write-time defenses miss L2/L3 is partly baked into the selection; what is empirically measured is the size of the miss, and that size depends on the curated attack distribution.

What would settle it

Run the 1,227-case pack at top-1 retrieval instead of top-3 and check whether L2 Behavioral Corruption Rate collapses toward L1 levels; the paper's own top-k sweep predicts L2 is retrieval-completeness limited, so a production retriever that rarely co-retrieves all fragments would erase the L2 half of the frontier. Alternatively, in the MID analysis remove random non-designated records instead of the designated poison and confirm the score drop is near zero; if random removal matches designated removal, the causal attribution would be vacuous.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the frontier holds, agent memory systems relying on write-time consistency checks will continue to be poisoned by attacks split across records or activated by later context; measured residual BCR for L2/L3 is in the 20-30% range even under the best pure write-time filter.
  • Defense evaluation for agent memory should report corruption by difficulty tier rather than only overall attack success, because averaging hides the L2/L3 residual.
  • Memory substrate matters: flat chunks preserve intact poisoned records and are most vulnerable; fact stores dilute injected content; hierarchical summaries partially attenuate but do not eliminate corruption.
  • Externally mediated write channels (tool outputs, inter-agent messages) consistently produce higher corruption than direct user input, so provenance-aware retrieval weighting is a natural complement to write-time filtering.
  • Retrieval breadth gates L2: the top-k sensitivity analysis shows L2 corruption rises sharply with larger retrieved context, meaning deployment choices about retriever coverage directly change the effective threat.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same structural argument likely applies to any pointwise sanitization of memory - content harmful only under composition or context cannot be detected by inspecting records in isolation - so the paper's frontier probably forms a ceiling for all write-time classifiers, not just the baselines tested.
  • Editorial inference: the MID signatures suggest a testable detection strategy: monitor pairwise interaction signals or activation shifts at retrieval time rather than at write time; if cheap to compute, such retrieval-aware scoring could close part of the L2/L3 gap.
  • Editorial inference: the benchmark's selection criteria (benign on the surface, non-leaky triggers) may understate real-world miss rates if attackers are sloppier, but may overstate them if production retrievers rarely co-retrieve all fragments; the paper's own top-k sweep is the lever to test this.
  • Editorial inference: extending MID to triple and higher-order fragment interactions would test whether L2's compositional signal saturates at pairs or grows with fragment count, which would inform how many fragments a future defense needs to track.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces MemPoison, a benchmark and analysis framework for persistent memory poisoning in LLM agents. It contributes a curated pack of 1,227 hand-validated cases spanning a three-level taxonomy (L1 single-record corruption, L2 compositional multi-record corruption, L3 context-triggered dormant corruption), three injection channels, four attack types, and three memory substrates. Experiments on ten model families compare an undefended baseline against eleven defense configurations, including write-time consistency checks, anomaly filters, judge-based admission, retrieval-time source reweighting, and sanitization. The headline finding is a 'defense frontier': write-time defenses substantially suppress L1 but fail to reliably suppress L2 and L3. To explain this, the paper proposes Mechanistic Influence Decomposition (MID), a counterfactual diagnostic that attributes residual corruption to single-record influence, pairwise interaction, and trigger-conditioned activation.

Significance. If the claims hold, MemPoison would be a substantial empirical contribution: it is broader than most prior memory-poisoning studies, uses a paired clean/poisoned protocol, a deterministic contrastive evaluator, five repeated runs with reported standard deviations, disclosed defense calibration, and a family-capped, human-reviewed construction pipeline. The L1/L2/L3 taxonomy and the MID diagnostic are potentially reusable by the community, and the central policy suggestion — that pointwise write-time filtering is insufficient for securing persistent agent memory — is timely and consequential. However, as detailed in the major comments, the L2 mechanistic interpretation is contradicted by the paper's own interaction-sign algebra, and the qualitative 'structural blind spot' claim is partly pre-ordained by dataset construction choices. These issues do not invalidate the benchmark itself, but they require a substantial re-analysis and a more careful statement of what is empirical versus definitional.

major comments (3)
  1. [§4.4, Eq. (2); §6, Table 3; App. B.7, Eq. (17)] The interpretation of Ω_g is algebraically wrong and undermines the L2 mechanistic claim. Eq. (2)/(17) defines Ω_g = Δs_i + Δs_j − d(r, r_{−ij}). A positive Ω_g means the joint-removal effect is smaller than the sum of single-removal effects — i.e., the two single-fragment effects overlap, so at least one fragment already has substantial influence alone. The text instead claims that positive Ω_g 'indicates that corruption emerges through interaction rather than through a single dominant record.' Appendix B.7 even states the opposite: positive Ω_g reflects 'overlap or non-additivity.' Table 3 reports Δ_g = 0.478 and Ω_g = 0.176; substituting into Eq. (17) forces Δs_i + Δs_j = 0.654, so the mean single-fragment effect is about 0.327 — larger than the L1 single-record effect of 0.266. This directly contradicts the assertion that L2 fragments are individually benign and that harm arises only
  2. [§3.2; App. A.1, human review criteria] The qualitative 'write-time defenses miss L2/L3' finding is partly circular. L2 is defined as 'circumventing per-record write-time defenses' and L3 as 'structurally bypassing behavior write-time checks,' while the human-review filter retains only cases where the injected content 'appears benign on the surface' and the trigger is 'non-leaky.' Thus, by construction, every retained L2/L3 case must look benign at write time, guaranteeing that a write-time-only defense will miss it. What remains empirical is the magnitude (e.g., L2 BCR 22.54%, L3 27.80% under consistency checking) and the MID/pipeline quantities, not the existence of the frontier. The paper should distinguish the definitional statement from the empirical claim, and ideally analyze the rejected pool or a broader sample of plausible L2/L3 variants to estimate how much of the attack space the curated pack represents. As written,
  3. [App. B.6, Eq. (10); §5.3] The headline BCR values, and therefore the defense frontier, inherit hand-set evaluator thresholds (clean/poisoned threshold 0.5, margin 0.1, ambiguous threshold 0.34) with no human-agreement study or threshold sensitivity analysis. The evaluator is deterministic, which is a strength, but the label boundaries are arbitrary and could change BCR levels and the L1/L2/L3 gaps. The authors should report inter-annotator agreement on a sample, and a sensitivity sweep over the thresholds (or at least show that the relative ordering of L1 vs. L2/L3 and the defense ranking is stable across a plausible range). This is important because the paper's central frontier claim is a comparative statement across difficulty levels, not just an absolute BCR.
minor comments (4)
  1. [App. B.7, text before Eq. (13)] The text says 'defined in Appendix 5' but should refer to Appendix B.6 (or Eq. (8)–(10)).
  2. [§6, Table 3] The notation '0.532→0.266' for L1 is unclear: it appears to be a pre-removal vs. post-removal score, but the columns are not defined. Please clarify in the table caption or text.
  3. [Table 3 and §4.4] In Table 3, the L1 row repeats Δ_s = 0.266 under both 'Score' and 'Interaction Signal' columns; this is confusing and should be marked as N/A or '—'.
  4. [Various figures (Fig. 2, Fig. 3, Fig. 4, Fig. 8)] Several figure panels show garbled font substitutions instead of readable axis labels and legends (e.g., the 'uni0000…' sequences in the rendered text). The final version must regenerate figures with readable labels.

Circularity Check

3 steps flagged

The L2/L3 defense-frontier is partly built into the taxonomy and case-retention filter; the quantitative BCR/MID magnitudes remain the empirical residue.

specific steps
  1. self definitional [Section 3.2; Section 3.3; Section 5.4]
    "L2: Compositional Corruption (Distributed Injection). ... This level circumvents per-record write-time defenses, as the injected fragments appear benign in isolation, bypassing plausibility and safety checks. ... L3: Context-Triggered Dormant Corruption (Sleeper Injection). ... This level structurally bypasses behavioral write-time checks."

    The paper's headline finding is that write-time defenses suppress L1 but fail to reliably suppress L2 and L3. But L2 and L3 are defined as attacks that, by construction, circumvent or bypass write-time checks. Testing write-time defenses on these categories can only confirm the defining property; it cannot fail. What remains empirical is the degree of failure (e.g., L2 BCR 22.54%, L3 27.80% under the consistency check) and cross-model variation, not the qualitative frontier itself.

  2. fitted input called prediction [Appendix A.1 (Human review and repair)]
    "A case was retained only if it satisfied all of the following criteria: ... The injected content appears benign on the surface. ... Rejected cases were dominated by L1 instances, where the injected content was often too explicit, too easy to detect at write time."

    The benchmark pack is filtered to keep only injections that look benign on the surface, while L1 cases are rejected for being too easy to detect at write time. The paper then reports that write-time defenses miss the retained cases. This is selection on the outcome variable: undetectability at admission is an input criterion, and the qualitative defense frontier is therefore pre-ordained rather than discovered. The non-circular residue is the measured BCR gap (4.77% vs 22.54% vs 27.80%) and the defense ordering.

  3. self definitional [Appendix A.1 (Difficulty specific realization) and Section 4.4 (MID)]
    "L3 stores a dormant or conditionally valid record whose harmful effect appears only when a later natural trigger condition is met. ... A large positive activation shift indicates that the memory remains behaviorally weak under ordinary conditions but becomes influential once the natural trigger appears."

    L3 cases are retained only if they are dormant until a trigger, so ActivationShift = Δtrigger − Δnormal is positive in sign by case construction. The paper presents positive ActivationShift as the MID signature of L3, closing an explanatory loop that is already fixed by the benchmark definition. The reported magnitude (0.242) and trigger-specific P2C are empirical, but the qualitative 'trigger-conditioned activation' claim is a selection criterion.

full rationale

The paper is a benchmark and measurement study with substantial independent empirical content: 1,227 cases, ten model families, three substrates, multiple defenses, and a pipeline audit. If the retained attack pack is representative of real memory-poisoning attempts, the measured BCR frontier (write-time consistency check: L1 4.77%, L2 22.54%, L3 27.80%) and the defense ordering are useful facts. However, the central qualitative claim — that write-time defenses are structurally blind to L2/L3 — is partly circular. L2/L3 are defined in Sec. 3.2 as attacks that circumvent or bypass write-time checks, and Appendix A.1 retains only cases whose injected content 'appears benign on the surface' while rejecting L1 cases as 'too easy to detect at write time.' Thus the finding that admission-time filters miss L2/L3 is guaranteed by the benchmark construction for the retained cases. The independent, falsifiable residue is quantitative: the sizes of the L1/L2/L3 gaps, the relative performance of different defenses (e.g., MIXED at 10.70% overall BCR), and the stage decomposition in Figure 4. No load-bearing self-citations are present; references are external. The skeptic's point about MID's Ωg equation (Eq. 17, Appendix B.7: positive Ωg means the joint-removal effect is smaller than the sum of single-removal effects, i.e., overlap or non-additivity rather than synergy) is a substantive scientific/interpretation concern, but it is not a circularity and does not affect this score.

Axiom & Free-Parameter Ledger

6 free parameters · 4 axioms · 0 invented entities

The central empirical content (BCR magnitudes, defense frontier, MID/P2C rates) is generated by hand-set thresholds (evaluator Eq. 10; defense lambdas in App B.3; causality d>0 in App B.8) and assumes the curated attack pack and emulated substrates are representative of real memory poisoning and real agent memory stacks.

free parameters (6)
  • Evaluator label thresholds (Eq. 10) = clean/poisoned >= 0.5, margin 0.1; ambiguous >= 0.34; unclear < 0.34
    Hand-set thresholds in Appendix B.6 define every BCR, CleanAcc, AR, UR, and MID score. No human-agreement or sensitivity analysis is reported; all headline numbers inherit this choice.
  • Write-time consistency check weights and threshold = lambda_a=0.45, lambda_s=0.35, lambda_u=0.20, threshold=0.58
    Appendix B.3. Calibrated on a held-out clean split to preserve benign utility; part of the internal defense row that produces the L1=4.77/L2=22.54/L3=27.80 frontier.
  • Memory novelty anomaly filter weights and threshold = lambda_n=0.55, lambda_q=0.25, lambda_e=0.20, threshold=0.68
    Appendix B.3; an internal defense row with hand-set weights.
  • PPL* weights and threshold = lambda_p=0.58, lambda_d=0.26, lambda_n=0.16, threshold=0.72
    Appendix B.3; enhanced anomaly-filter defense row.
  • Source Reliability Retrieval channel trust weights = context=1.0, episode summary=0.9, user=0.7, tool=0.4, cross_agent=0.3
    Appendix B.3; heuristic provenance reweighting used for the retrieval-time baseline.
  • MID/audit causal threshold (d > 0) = d(r, r_{-z}) > 0
    Appendix B.8 Eq. 24 defines ResidualCausal via any positive poisoned-score drop. A stricter threshold would shrink residual-causal rates, and counterfactual re-execution is sampling-based, so d>0 may include noise; five runs are averaged but no significance test is reported.
axioms (4)
  • domain assumption The token-overlap contrastive evaluator (Eqs. 8-10) measures behavioral corruption as humans would.
    Appendix B.6. All BCR numbers and hence the defense frontier rest on this proxy; no inter-annotator or human-agreement validation of the thresholds is reported.
  • domain assumption Cases surviving human review where 'injected content appears benign on the surface' (App A.1, second criterion) are representative of real attacker distributions.
    Appendix A.1; L1 cases 'too easy to detect at write time' were rejected, and L2/L3 are defined (Sec 3.2) as evading write-time checks. The qualitative 'structural blind spot' conclusion is conditional on this selection.
  • domain assumption The three emulated substrates (flat_chunk, fact_store, hierarchical_notes) with token-overlap top-3 retrieval approximate production agent memory systems.
    Appendices B.1-B.2; retrieval is lexical overlap with k=3 fixed; no embedding retriever, MemGPT-style paging, decay, or access control is tested. Substrate-level vulnerability ordering may not transfer to production stacks.
  • domain assumption Five repeated runs per condition and averaging make the reported +/-SD error bars and MID d>0 causal predicates meaningful.
    Sec 5.1 and B.7; no significance testing is reported for the d>0 causal predicate used in the pipeline audit.

pith-pipeline@v1.3.0-alltime-deepseek · 35004 in / 23767 out tokens · 236365 ms · 2026-08-02T01:29:09.833773+00:00 · methodology

0 comments
read the original abstract

Persistent external memory enhances agent continuity but introduces persistent security vulnerabilities: adversarial content can be injected via standard interaction channels, retained across turns, and later distort downstream behavior. To address this challenge, we propose MemPoison, a comprehensive benchmark and analysis framework featuring 1227 hand-validated cases across four attack types, three injection channels, and three representative memory substrates, evaluated on seven open-weight and three closed-weight model families. We introduce a three-tier taxonomy: (L1) direct single-record corruption, (L2) compositional multi-record corruption and (L3) context-triggered dormant corruption. Our evaluations reveal a distinct defense frontier: while baseline write-time defenses, such as consistency checks, substantially suppress direct L1 attacks, they fail to reliably suppress L2 and L3 attacks. Through mechanistic influence decomposition (MID), we demonstrate structural blind spots in write-time defenses, which admit seemingly benign records that later become harmful through joint retrieval composition or trigger-conditioned activation. Our findings advocate for shifting from static filtering to adaptive, context-sensitive memory defense strategies.

Figures

Figures reproduced from arXiv: 2607.14651 by Jifeng Gao, Kang Xia, Mingkai Lin, Sanglu Lu, Wenzhong Li, Xiaobin Hong, Xingshen Wei, Yi Zhang.

Figure 1
Figure 1. Figure 1: The left panel illustrates how poisoned memories pass through write-time admission, [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Benchmark composition and representative memory-poisoning case. (2a) illustrates [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Undefended BCR across benchmark factors. Bars report average per-model BCR under [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Pipeline stage decomposition of memory poisoning outcomes under write time defenses across difficulty levels. corruption alone cannot distinguish failures at admission, retrieval, or response generation. Each poisoned case is decomposed into four mutually exclusive outcomes: write-blocked, admitted but not retrieved, retrieved but not causal, and residual causal, with percentages macro averaged across mode… view at source ↗
Figure 5
Figure 5. Figure 5: MemPoison benchmark construction pipeline. We build MemPoison in a family first [PITH_FULL_IMAGE:figures/full_fig_p017_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Evaluation prompt template used to generate model responses under clean, poisoned, [PITH_FULL_IMAGE:figures/full_fig_p023_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Write-time LLM judge prompt used by admission-control defenses to score candidate [PITH_FULL_IMAGE:figures/full_fig_p024_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Retrieval top-k sensitivity under NONE. Poisoned BCR increases as larger retrieved contexts expose more poisoned memory, especially for L2 compositional attacks. Clean accuracy is low without retrieval (k = 0), but stabilizes once at least one memory item is retrieved. C Extended Results C.1 Undefended BCR Breakdowns Across Models [PITH_FULL_IMAGE:figures/full_fig_p028_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Agents defense performance across all evaluated defense configurations. The first ten panels [PITH_FULL_IMAGE:figures/full_fig_p030_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

69 extracted references · 24 linked inside Pith

  1. [1]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

  2. [2]

    Security in llm-as-a-judge: A comprehensive sok

    Aiman Almasoud, Antony Anju, Marco Arazzi, Mert Cihangiroglu, Vignesh Kumar Kembu, Serena Nicolazzo, Antonino Nocera, Saraga Sakthidharan, et al. Security in llm-as-a-judge: A comprehensive sok. arXiv preprint arXiv:2603.29403, 2026

  3. [3]

    Ipiguard: A novel tool dependency graph-based defense against indirect prompt injection in llm agents

    Hengyu An, Jinghuai Zhang, Tianyu Du, Chunyi Zhou, Qingming Li, Tao Lin, and Shouling Ji. Ipiguard: A novel tool dependency graph-based defense against indirect prompt injection in llm agents. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 1023–1039, 2025

  4. [4]

    One shot dominance: Knowledge poisoning attack on retrieval-augmented generation systems

    Zhiyuan Chang, Mingyang Li, Xiaojun Jia, Junjie Wang, Yuekai Huang, Ziyou Jiang, Yang Liu, and Qing Wang. One shot dominance: Knowledge poisoning attack on retrieval-augmented generation systems. arXiv preprint arXiv:2505.11548, 2025

  5. [5]

    {StruQ}: Defending against prompt injection with structured queries

    Sizhe Chen, Julien Piet, Chawin Sitawarin, and David Wagner. {StruQ}: Defending against prompt injection with structured queries. In34th USENIX Security Symposium (USENIX Security 25), pages 2383–2400, 2025

  6. [6]

    Defense against prompt injection attack by leveraging attack techniques, 2025.URL https://arxiv

    Yulin Chen, Haoran Li, Zihao Zheng, Yangqiu Song, Dekai Wu, and Bryan Hooi. Defense against prompt injection attack by leveraging attack techniques, 2025.URL https://arxiv. org/abs/2411.00459

  7. [7]

    Agentpoison: Red-teaming llm agents via poisoning memory or knowledge bases.Advances in Neural Information Processing Systems, 37: 130185–130213, 2024

    Zhaorun Chen, Zhen Xiang, Chaowei Xiao, Dawn Song, and Bo Li. Agentpoison: Red-teaming llm agents via poisoning memory or knowledge bases.Advances in Neural Information Processing Systems, 37: 130185–130213, 2024

  8. [8]

    Contextcite: Attributing model generation to context.Advances in Neural Information Processing Systems, 37:95764–95807, 2024

    Benjamin Cohen-Wang, Harshay Shah, Kristian Georgiev, and Aleksander M ˛ adry. Contextcite: Attributing model generation to context.Advances in Neural Information Processing Systems, 37:95764–95807, 2024

  9. [9]

    Agentdojo: A dynamic environment to evaluate prompt injection attacks and defenses for llm agents.Advances in Neural Information Processing Systems, 37:82895–82920, 2024

    Edoardo Debenedetti, Jie Zhang, Mislav Balunovic, Luca Beurer-Kellner, Marc Fischer, and Florian Tramèr. Agentdojo: A dynamic environment to evaluate prompt injection attacks and defenses for llm agents.Advances in Neural Information Processing Systems, 37:82895–82920, 2024

  10. [10]

    Memory injection attacks on llm agents via query-only interaction.arXiv preprint arXiv:2503.03704, 2025

    Shen Dong, Shaochen Xu, Pengfei He, Yige Li, Jiliang Tang, Tianming Liu, Hui Liu, and Zhen Xiang. Memory injection attacks on llm agents via query-only interaction.arXiv preprint arXiv:2503.03704, 2025

  11. [11]

    A practical memory injection attack against llm agents.arXiv e-prints, pages arXiv–2503, 2025

    Shen Dong, Shaochen Xu, Pengfei He, Yige Li, Jiliang Tang, Tianming Liu, Hui Liu, and Zhen Xiang. A practical memory injection attack against llm agents.arXiv e-prints, pages arXiv–2503, 2025

  12. [12]

    Memory for autonomous llm agents: Mechanisms, evaluation, and emerging frontiers.arXiv preprint arXiv:2603.07670, 2026

    Pengfei Du. Memory for autonomous llm agents: Mechanisms, evaluation, and emerging frontiers.arXiv preprint arXiv:2603.07670, 2026

  13. [13]

    Backdooragent: A unified framework for backdoor attacks on llm-based agents.arXiv preprint arXiv:2601.04566, 2026

    Yunhao Feng, Yige Li, Yutao Wu, Yingshui Tan, Yanming Guo, Yifan Ding, Kun Zhai, Xingjun Ma, and Yu-Gang Jiang. Backdooragent: A unified framework for backdoor attacks on llm-based agents.arXiv preprint arXiv:2601.04566, 2026

  14. [14]

    Chatglm: A family of large language models from glm-130b to glm-4 all tools

    Team Glm, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Dan Zhang, Diego Rojas, Guanyu Feng, Hanlin Zhao, et al. Chatglm: A family of large language models from glm-130b to glm-4 all tools. arXiv preprint arXiv:2406.12793, 2024

  15. [15]

    Gemini api documentation

    Google. Gemini api documentation. https://ai.google.dev/gemini-api/docs, 2026. Online Documentation

  16. [16]

    The llama 3 herd of models

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al- Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

  17. [17]

    A survey on llm-as-a-judge.The Innovation, 2024

    Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, et al. A survey on llm-as-a-judge.The Innovation, 2024

  18. [18]

    The emerged security and privacy of llm agent: A survey with case studies.ACM Computing Surveys, 58(6):1–36, 2025

    Feng He, Tianqing Zhu, Dayong Ye, Bo Liu, Wanlei Zhou, and Philip S Yu. The emerged security and privacy of llm agent: A survey with case studies.ACM Computing Surveys, 58(6):1–36, 2025

  19. [19]

    Evaluating memory in llm agents via incremental multi-turn interactions.arXiv preprint arXiv:2507.05257, 2025

    Yuanzhe Hu, Yu Wang, and Julian McAuley. Evaluating memory in llm agents via incremental multi-turn interactions.arXiv preprint arXiv:2507.05257, 2025. 10

  20. [20]

    Retrieval- augmented generation with estimation of source reliability

    Jeongyeon Hwang, Junyoung Park, Hyejin Park, Dongwoo Kim, Sangdon Park, and Jungseul Ok. Retrieval- augmented generation with estimation of source reliability. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 34267–34291, 2025

  21. [21]

    Baseline defenses for adversarial attacks against aligned language models.arXiv preprint arXiv:2309.00614, 2023

    Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping-yeh Chiang, Micah Goldblum, Aniruddha Saha, Jonas Geiping, and Tom Goldstein. Baseline defenses for adversarial attacks against aligned language models.arXiv preprint arXiv:2309.00614, 2023

  22. [22]

    The task shield: Enforcing task alignment to defend against indirect prompt injection in llm agents

    Feiran Jia, Tong Wu, Xin Qin, and Anna Squicciarini. The task shield: Enforcing task alignment to defend against indirect prompt injection in llm agents. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 29680–29697, 2025

  23. [23]

    Swe-bench: Can language models resolve real-world github issues? InThe twelfth international conference on learning representations, 2023

    Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. Swe-bench: Can language models resolve real-world github issues? InThe twelfth international conference on learning representations, 2023

  24. [24]

    Memory os of ai agent

    Jiazheng Kang, Mingming Ji, Zhe Zhao, and Ting Bai. Memory os of ai agent. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 25972–25981, 2025

  25. [25]

    Certifying llm safety against adversarial prompting.arXiv preprint arXiv:2309.02705, 2023

    Aounon Kumar, Chirag Agarwal, Suraj Srinivas, Aaron Jiaxun Li, Soheil Feizi, and Himabindu Lakkaraju. Certifying llm safety against adversarial prompting.arXiv preprint arXiv:2309.02705, 2023

  26. [26]

    A survey on the security of long-term memory in llm agents: Toward mnemonic sovereignty.arXiv preprint arXiv:2604.16548, 2026

    Zehao Lin, Chunyu Li, and Kai Chen. A survey on the security of long-term memory in llm agents: Toward mnemonic sovereignty.arXiv preprint arXiv:2604.16548, 2026

  27. [27]

    Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model.arXiv preprint arXiv:2405.04434, 2024

    Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Chenggang Zhao, Chengqi Dengr, Chong Ruan, Damai Dai, Daya Guo, et al. Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model.arXiv preprint arXiv:2405.04434, 2024

  28. [28]

    Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

  29. [29]

    Formalizing and benchmarking prompt injection attacks and defenses

    Yupei Liu, Yuqi Jia, Runpeng Geng, Jinyuan Jia, and Neil Zhenqiang Gong. Formalizing and benchmarking prompt injection attacks and defenses. In33rd USENIX Security Symposium (USENIX Security 24), pages 1831–1847, 2024

  30. [30]

    Datasentinel: A game-theoretic detection of prompt injection attacks

    Yupei Liu, Yuqi Jia, Jinyuan Jia, Dawn Song, and Neil Zhenqiang Gong. Datasentinel: A game-theoretic detection of prompt injection attacks. In2025 IEEE Symposium on Security and Privacy (SP), pages 2190–2208. IEEE, 2025

  31. [31]

    Terminal-bench: Benchmarking agents on hard, realistic tasks in command line interfaces.arXiv preprint arXiv:2601.11868, 2026

    Mike A Merrill, Alexander G Shaw, Nicholas Carlini, Boxuan Li, Harsh Raj, Ivan Bercovich, Lin Shi, Jeong Yeon Shin, Thomas Walshe, E Kelly Buchanan, et al. Terminal-bench: Benchmarking agents on hard, realistic tasks in command line interfaces.arXiv preprint arXiv:2601.11868, 2026

  32. [32]

    Prompt-guard-86m: A classifier model for detecting prompt attacks

    Meta AI. Prompt-guard-86m: A classifier model for detecting prompt attacks. https://huggingface. co/meta-llama/Prompt-Guard-86M, 2024. Accessed: 2026-04-11

  33. [33]

    Towards lifelong dialogue agents via timeline-based memory management

    Kai Tzu-iunn Ong, Namyoung Kim, Minju Gwak, Hyungjoo Chae, Taeyoon Kwon, Yohan Jo, Seung-won Hwang, Dongha Lee, and Jinyoung Yeo. Towards lifelong dialogue agents via timeline-based memory management. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (V...

  34. [34]

    Memgpt: towards llms as operating systems

    Charles Packer, Vivian Fang, Shishir_G Patil, Kevin Lin, Sarah Wooders, and Joseph_E Gonzalez. Memgpt: towards llms as operating systems. 2023

  35. [35]

    Generative agents: Interactive simulacra of human behavior

    Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. InProceedings of the 36th annual acm symposium on user interface software and technology, pages 1–22, 2023

  36. [36]

    The berkeley function calling leaderboard (bfcl): From tool use to agentic evaluation of large language models

    Shishir G Patil, Huanzhi Mao, Fanjia Yan, Charlie Cheng-Jie Ji, Vishnu Suresh, Ion Stoica, and Joseph E Gonzalez. The berkeley function calling leaderboard (bfcl): From tool use to agentic evaluation of large language models. InForty-second International Conference on Machine Learning, 2025

  37. [37]

    The why behind the action: Unveiling internal drivers via agentic attribution.arXiv preprint arXiv:2601.15075, 2026

    Chen Qian, Peng Wang, Dongrui Liu, Junyao Yang, Dadi Guo, Ling Tang, Jilin Mei, Qihan Ren, Shuai Shao, Yong Liu, et al. The why behind the action: Unveiling internal drivers via agentic attribution.arXiv preprint arXiv:2601.15075, 2026. 11

  38. [38]

    Toolllm: Facilitating large language models to master 16000+ real-world apis, 2023

    Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, Sihan Zhao, Runchu Tian, Ruobing Xie, Jie Zhou, Mark Gerstein, Dahai Li, Zhiyuan Liu, and Maosong Sun. Toolllm: Facilitating large language models to master 16000+ real-world apis, 2023

  39. [39]

    Smoothllm: Defending large language models against jailbreaking attacks.arXiv preprint arXiv:2310.03684, 2023

    Alexander Robey, Eric Wong, Hamed Hassani, and George J Pappas. Smoothllm: Defending large language models against jailbreaking attacks.arXiv preprint arXiv:2310.03684, 2023

  40. [40]

    Evaluating memory structure in llm agents.arXiv preprint arXiv:2602.11243, 2026

    Alina Shutova, Alexandra Olenina, Ivan Vinogradov, and Anton Sinitsin. Evaluating memory structure in llm agents.arXiv preprint arXiv:2602.11243, 2026

  41. [41]

    Openai gpt-5 system card.arXiv preprint arXiv:2601.03267, 2025

    Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, Adi Ganesh, Ahmed El-Kishky, Aidan McLaugh- lin, Aiden Low, AJ Ostrow, Akhila Ananthram, et al. Openai gpt-5 system card.arXiv preprint arXiv:2601.03267, 2025

  42. [42]

    Memorygraft: Persistent compromise of llm agents via poisoned experience retrieval.arXiv preprint arXiv:2512.16962, 2025

    Saksham Sahai Srivastava and Haoyu He. Memorygraft: Persistent compromise of llm agents via poisoned experience retrieval.arXiv preprint arXiv:2512.16962, 2025

  43. [43]

    Memory poisoning attack and defense on memory based llm-agents.arXiv preprint arXiv:2601.05504, 2026

    Balachandra Devarangadi Sunil, Isheeta Sinha, Piyush Maheshwari, Shantanu Todmal, Shreyan Mallik, and Shuchi Mishra. Memory poisoning attack and defense on memory based llm-agents.arXiv preprint arXiv:2601.05504, 2026

  44. [44]

    Membench: Towards more comprehensive evaluation on the memory of llm-based agents

    Haoran Tan, Zeyu Zhang, Chen Ma, Xu Chen, Quanyu Dai, and Zhenhua Dong. Membench: Towards more comprehensive evaluation on the memory of llm-based agents. InFindings of the Association for Computational Linguistics: ACL 2025, pages 19336–19352, 2025

  45. [45]

    Revprag: Revealing poisoning attacks in retrieval-augmented generation through llm activation analysis.arXiv preprint arXiv:2411.18948, 2024

    Xue Tan, Hao Luan, Mingyu Luo, Xiaoyan Sun, Ping Chen, and Jun Dai. Revprag: Revealing poisoning attacks in retrieval-augmented generation through llm activation analysis.arXiv preprint arXiv:2411.18948, 2024

  46. [46]

    In prospect and retrospect: Reflective memory management for long-term per- sonalized dialogue agents

    Zhen Tan, Jun Yan, I-Hung Hsu, Rujun Han, Zifeng Wang, Long Le, Yiwen Song, Yanfei Chen, Hamid Palangi, George Lee, et al. In prospect and retrospect: Reflective memory management for long-term per- sonalized dialogue agents. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8416–8439, 2025

  47. [47]

    Injecmem: Memory injection attack on llm agent memory systems

    Hanling Tian, Zeyang Sha, Jingying Wang, Yuhang Liu, Zhehao Huang, and Xiaolin Huang. Injecmem: Memory injection attack on llm agent memory systems

  48. [48]

    Injecmem: Memory injection attack on llm agent memory systems

    Hanling Tian, Zeyang Sha, Jingying Wang, Yuhang Liu, Zhehao Huang, and Xiaolin Huang. Injecmem: Memory injection attack on llm agent memory systems. 2026

  49. [49]

    Memory poisoning and secure multi-agent systems.arXiv preprint arXiv:2603.20357, 2026

    Vicenç Torra and Maria Bras-Amorós. Memory poisoning and secure multi-agent systems.arXiv preprint arXiv:2603.20357, 2026

  50. [50]

    Unveiling privacy risks in llm agent memory

    Bo Wang, Weiyi He, Shenglai Zeng, Zhen Xiang, Yue Xing, Jiliang Tang, and Pengfei He. Unveiling privacy risks in llm agent memory. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 25241–25260, 2025

  51. [51]

    Badagent: Inserting and activating backdoor attacks in llm agents

    Yifei Wang, Dizhan Xue, Shengjie Zhang, and Shengsheng Qian. Badagent: Inserting and activating backdoor attacks in llm agents. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9811–9827, 2024

  52. [52]

    A-memguard: A proactive defense framework for llm-based agent memory.arXiv preprint arXiv:2510.02373, 2025

    Qianshan Wei, Tengchao Yang, Yaochen Wang, Xinfeng Li, Lijun Li, Zhenfei Yin, Yi Zhan, Thorsten Holz, Zhiqiang Lin, and XiaoFeng Wang. A-memguard: A proactive defense framework for llm-based agent memory.arXiv preprint arXiv:2510.02373, 2025

  53. [53]

    Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments.Advances in Neural Information Processing Systems, 37: 52040–52094, 2024

    Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh J Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, et al. Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments.Advances in Neural Information Processing Systems, 37: 52040–52094, 2024

  54. [54]

    Theagentcompany: benchmarking llm agents on consequential real world tasks.arXiv preprint arXiv:2412.14161, 2024

    Frank F Xu, Yufan Song, Boxuan Li, Yuxuan Tang, Kritanjali Jain, Mengxue Bao, Zora Z Wang, Xuhui Zhou, Zhitong Guo, Murong Cao, et al. Theagentcompany: benchmarking llm agents on consequential real world tasks.arXiv preprint arXiv:2412.14161, 2024

  55. [55]

    Qwen3 technical report, 2025

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, et al. Qwen3 technical report, 2025. URL https://arxiv.org/abs/2505.09388

  56. [56]

    Qwen2.5 technical report, 2025

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2.5 technical report, 2025. URL https://arxiv.org/abs/ 2412.15115. 12

  57. [57]

    Shieldrag: Safeguarding retrieval-augmented generation from untrusted knowledge bases

    Peiru Yang, Haoran Zheng, Yi Luo, Xinyi Liu, Jinrui Wang, Huili Wang, Xintian Li, Yongfeng Huang, and Tao Qi. Shieldrag: Safeguarding retrieval-augmented generation from untrusted knowledge bases. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 34286–34294, 2026

  58. [58]

    Watch out for your agents! investigating backdoor threats to llm-based agents.Advances in Neural Information Processing Systems, 37:100938–100964, 2024

    Wenkai Yang, Xiaohan Bi, Yankai Lin, Sishuo Chen, Jie Zhou, and Xu Sun. Watch out for your agents! investigating backdoor threats to llm-based agents.Advances in Neural Information Processing Systems, 37:100938–100964, 2024

  59. [59]

    Zombie agents: Persistent control of self-evolving llm agents via self-reinforcing injections.arXiv preprint arXiv:2602.15654, 2026

    Xianglin Yang, Yufei He, Shuo Ji, Bryan Hooi, and Jin Song Dong. Zombie agents: Persistent control of self-evolving llm agents via self-reinforcing injections.arXiv preprint arXiv:2602.15654, 2026

  60. [60]

    τ-bench: A benchmark for tool-agent- user interaction in real-world domains.arXiv preprint arXiv:2406.12045, 2024

    Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan. τ-bench: A benchmark for tool-agent- user interaction in real-world domains.arXiv preprint arXiv:2406.12045, 2024

  61. [61]

    Ori Yoran, Samuel Joseph Amouyal, Chaitanya Malaviya, Ben Bogin, Ofir Press, and Jonathan Berant. Assistantbench: Can web agents solve realistic and time-consuming tasks? InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 8938–8968, 2024

  62. [62]

    A survey on trustworthy llm agents: Threats and countermeasures

    Miao Yu, Fanci Meng, Xinyun Zhou, Shilong Wang, Junyuan Mao, Linsey Pan, Tianlong Chen, Kun Wang, Xinfeng Li, Yongfeng Zhang, et al. A survey on trustworthy llm agents: Threats and countermeasures. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V . 2, pages 6216–6226, 2025

  63. [63]

    Injecagent: Benchmarking indirect prompt injections in tool-integrated large language model agents

    Qiusi Zhan, Zhixiang Liang, Zifan Ying, and Daniel Kang. Injecagent: Benchmarking indirect prompt injections in tool-integrated large language model agents. InFindings of the Association for Computational Linguistics: ACL 2024, pages 10471–10506, 2024

  64. [64]

    Who taught the lie? responsibility attribution for poisoned knowledge in retrieval-augmented generation.arXiv preprint arXiv:2509.13772, 2025

    Baolei Zhang, Haoran Xin, Yuxi Chen, Zhuqing Liu, Biao Yi, Tong Li, Lihai Nie, Zheli Liu, and Minghong Fang. Who taught the lie? responsibility attribution for poisoned knowledge in retrieval-augmented generation.arXiv preprint arXiv:2509.13772, 2025

  65. [65]

    Traceback of poisoning attacks to retrieval-augmented generation

    Baolei Zhang, Haoran Xin, Minghong Fang, Zhuqing Liu, Biao Yi, Tong Li, and Zheli Liu. Traceback of poisoning attacks to retrieval-augmented generation. InProceedings of the ACM on Web Conference 2025, pages 2085–2097, 2025

  66. [66]

    Agent security bench (asb): Formalizing and benchmarking attacks and defenses in llm-based agents.arXiv preprint arXiv:2410.02644, 2024

    Hanrong Zhang, Jingyuan Huang, Kai Mei, Yifei Yao, Zhenting Wang, Chenlu Zhan, Hongwei Wang, and Yongfeng Zhang. Agent security bench (asb): Formalizing and benchmarking attacks and defenses in llm-based agents.arXiv preprint arXiv:2410.02644, 2024

  67. [67]

    A survey on the memory mechanism of large language model based agents, 2024.URL https://arxiv

    Zeyu Zhang, Xiaohe Bo, Chen Ma, Rui Li, Xu Chen, Quanyu Dai, Jieming Zhu, Zhenhua Dong, and Ji-Rong Wen. A survey on the memory mechanism of large language model based agents, 2024.URL https://arxiv. org/abs/2404.13501

  68. [68]

    Judging llm-as-a-judge with mt-bench and chatbot arena

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in neural information processing systems, 36:46595–46623, 2023

  69. [69]

    candidate_memory

    Wei Zou, Mingwen Dong, Miguel Romero Calvo, Shuaichen Chang, Jiang Guo, Dongkyu Lee, Xing Niu, Xiaofei Ma, Yanjun Qi, and Jiarong Jiang. Poison once, exploit forever: Environment-injected memory poisoning attacks on web agents.arXiv preprint arXiv:2604.02623, 2026. A Benchmark Details A.1 Benchmark Construction A core design choice in MemPoison is that th...