Pith. sign in

REVIEW 2 major objections 3 minor 34 references

Agentic systems that keep persistent memory files are vulnerable to stored prompt injection: a payload already planted in a trusted file can steer current and future sessions, even when the model refuses to act on it.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

T0 review · deepseek-v4-flash

2026-08-02 01:34 UTC pith:YEU4FU2M

load-bearing objection A useful, honestly-scoped empirical study of memory-based prompt injection; the central claim largely holds, but the quantitative ASR tables need no-payload baselines before you trust the headline numbers. the 2 major comments →

arxiv 2607.14611 v1 pith:YEU4FU2M submitted 2026-07-16 cs.CR cs.AIcs.MA

Bad Memory: Evaluating Prompt Injection Risks from Memory in Agentic Systems

classification cs.CR cs.AIcs.MA
keywords agentic systemsprompt injectionmemory poisoningpersistent memoryLLM securitystored XSS analogmulti-session attacksattack persistence
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Agentic systems that keep persistent memory files — auto-loaded instruction files, behavior rules, and knowledge notes — trust those files as the user's own instructions. This paper shows that when an attacker can plant a malicious instruction in such a file, that instruction can steer the agent's behavior in the current session and, if it persists, in later sessions as well. In a sandboxed workspace with two coding-agent products and four frontier models, the authors measured success across three adversarial goals: copying an SSH key, installing a known-vulnerable package, and recommending a specific credit card. They found that payloads already in memory often succeed, that auto-loaded files are the most reliable vector, and that model refusal does not equal cleanup — a model can refuse to act yet leave the payload armed for future sessions or weaker models. The central claim is that persistent memory converts prompt injection from a one-shot event into a stored, compounding vulnerability.

Core claim

On the paper's own terms, the central discovery is that the trust boundary around an agent's persistent workspace files is exploitable: a payload already present in a trusted memory file — an auto-loaded file such as CLAUDE.md or AGENTS.md, a universal behavior file, or a referenced knowledge file — can influence the agent even when the user's task is benign, and the influence can compound across sessions. The paper quantifies this as attack success rate and payload persistence rate, reporting that state-of-the-art models remain vulnerable to memory-based prompt injection: credential theft succeeds on weaker models and is refused by stronger ones, but installing a pinned vulnerable package a

What carries the argument

The central mechanism is the workspace memory hierarchy and the agent's self-modification loop. Auto-loaded root files (CLAUDE.md or AGENTS.md) enter the model's context at session start without any retrieval decision; referenced behavior and knowledge files (core/behaviors.md, knowledge/*.md) are read when the agent decides they are relevant; and the agent is both instructed to trust and maintain these files. The attack plants a malicious instruction in one of these files, and the evaluation protocol uses a probe session (benign task that reveals whether the payload acted), an optional stabilization session (benign task that may cause the agent to rewrite memory), and a second probe session

Load-bearing premise

The paper's numbers assume that the injected payload — not the model's own tendency — produced the observed actions, because no no-payload control trials are reported.

What would settle it

Re-run the same probe sessions with the memory files in their clean baseline state (no payload) and compare the rate at which the agent reads the SSH key, installs the pinned PyYAML version, or recommends the targeted credit card. If the clean-session rate matches the reported ASR — especially for the model that recommended the card in every trial — then the numbers reflect base-rate behavior rather than the injection; if the clean rate is near zero, the attack is the cause.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Auto-loaded instruction files are the highest-risk vector; any payload placed there reaches the model's context at the start of every session.
  • Refusal is not cleanup: models that decline to follow an injected instruction often leave it in place, arming later sessions or a weaker model that shares the workspace.
  • Attack success and payload persistence are orthogonal failure modes; a model can be the most resistant on one metric and the worst on the other, so security evaluation must track both.
  • Multiple attacks can be stacked in the same persistent state, and their success depends on order — a first attack that the model resists can make it more suspicious of the second.
  • The threat model shifts from one-shot injection to a stored, compounding one, motivating permission boundaries and validation for memory updates rather than relying on the model's judgment.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The reported ASRs may overstate the payload's causal effect because no no-payload control trials are described; for example, the model that recommended the targeted credit card in 100% of trials could simply have a strong baseline propensity to recommend that card. A clean-workspace control group would settle this.
  • The stored-XSS analogy points to a defense the paper leaves implicit: treat memory files as untrusted data at session start and apply a capability policy that separates facts from executable directives, so a planted 'preference' cannot invoke tools or read secrets.
  • The results imply practical hygiene for users: any memory file copied from a public repository or shared setup should be treated as potentially adversarial, since the agent treats its contents as self-authored trusted instructions.
  • Because persistence rates differ sharply across models, a team that switches models on the same workspace may inherit a payload that the previous model refused to delete — a concrete operational hazard the paper notes in passing.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 3 minor

Summary. The paper studies prompt-injection attacks that target persistent memory files in agentic systems. Using a sandboxed synthetic workspace modeled after Claude Code and OpenAI Codex, it evaluates four models (Claude Haiku 4.5, Claude Opus 4.7, GPT-5.2, GPT-5.5) against three adversarial goals: credential exfiltration, unauthorized tool use (installing a vulnerable PyYAML version), and brand targeting (recommending an Amex card). The experimental protocol includes single-session probes and multi-session sequences (Probe → Stabilization → Probe) to measure both attack success rate (ASR) and payload persistence. The authors report that while it is difficult to make an agent overwrite its own memory files from untrusted external content, a payload already planted in an auto-loaded or referenced file can influence current and future sessions. They also report substantial model- and goal-dependent variation, with case studies showing agents that flag, remove, or relocate injected rules.

Significance. If the central claim holds, the paper identifies a practically important attack surface: persistent memory files function like stored XSS for agentic systems, enabling a single planted instruction to affect multiple sessions and potentially weaker later models. The multi-session evaluation protocol is a useful methodological contribution, and the case studies (e.g., GPT-5.2 moving a malicious install instruction into AGENTS.md; Opus flagging but not deleting a brand rule) are concrete and informative. The authors release their code and experimental artifacts, which supports replication. However, the quantitative support is weakened by the absence of no-payload control trials and by the small per-cell sample size, so the causal interpretation of the reported ASRs is not yet established.

major comments (2)
  1. [§3.3, §4.1 (Table 1)] The central claim that planted payloads 'successfully attack' sessions rests on ASR being interpreted as the effect of the payload. ASR is defined in §3.3 as the fraction of trials in which the probe performs the attacker's action, but no no-payload control condition is reported. For brand targeting, the benign task is 'provide recommendations for managing everyday spending,' which could naturally elicit credit-card recommendations; GPT-5.5's 100% ASR might be a base-rate propensity rather than an effect of the injected rule. For credential exfiltration and unauthorized tool use, spontaneous occurrence is less likely, but the paper provides no measurement for any model/goal. Without a control, the headline 'state-of-the-art models are still vulnerable' conflates attack effect with prior behavior. The manuscript should add control trials with clean memory files, report base rates, and eit
  2. [§4.1–§4.3 (Tables 1–5)] All quantitative results are based on N=10 trials per condition, with no confidence intervals, significance tests, or measures of trial-to-trial variability. Rates such as 80% vs 60% (e.g., Haiku 4.5 in Table 1 vs Table 2) are not statistically distinguishable at this sample size, and the paper's claims of 'substantial variation' across models and sequences are not quantified. This is load-bearing because the paper's contribution is empirical. At minimum, the authors should report binomial confidence intervals and apply an appropriate test (e.g., Fisher exact) for comparisons central to the narrative, or explicitly justify the sample size and treat the rates as descriptive only.
minor comments (3)
  1. [Abstract, §3.1, §5 (Limitations)] The abstract and text state that 'it is difficult to make an agent overwrite its own memory files using untrusted external content,' but this is supported only by unreported 'preliminary experiments' (§3.1, footnote 1). Since this is a secondary empirical claim, it should either be measured systematically or explicitly reframed as an assumption/scope limitation.
  2. [§3.5, Tables 1–4] The paper correctly notes in §3.5 that ASRs are 'not directly comparable across attack goals,' yet it computes and discusses mean ASR across goals in Tables 1, 2, and 4. The mean column should be removed or heavily caveated, because the mix of goals and vectors makes the aggregate rate hard to interpret.
  3. [Appendix / front matter] The ACM template placeholders remain in the manuscript: 'Conference acronym ’XX, June 03–05, 2018, Woodstock, NY', '©2018', and 'Make sure to enter the correct conference title from your rights confirmation email.' These should be cleaned up before any publication.

Circularity Check

0 steps flagged

No circularity: an empirical measurement study with no fitted parameters, no self-citations, and no derivation chain that reduces to its inputs.

full rationale

This paper is an empirical evaluation, not a derivation. Attack success rate and persistence rate are directly measured outcomes defined in Section 3.3, and no parameters are fitted to data and then renamed as predictions. The threat model in Section 3.1 explicitly assumes payloads are already present in memory files; this defines the scope of the study rather than assuming the conclusion. The closest epistemological concern is that Section 4.1 reports ASRs without a no-payload control condition, so the causal attribution of the target actions to the planted payload (rather than to base-rate model behavior) is not fully established. That is a correctness/validity limitation, not a circularity by construction: the paper does not define the payload effect in terms of itself, and no equation or fitted input is reused as evidence. The paper's references to prior work are contextual, and none is load-bearing as a substitute for the empirical measurements. Therefore no circular step meets the required evidentiary bar.

Axiom & Free-Parameter Ledger

0 free parameters · 3 axioms · 0 invented entities

No fitted free parameters appear; the hand-designed payloads and 10-trial count are experimental choices, not parameters fit to data. The central empirical claims rest on domain assumptions about agent trust of memory files, representativeness of the synthetic workspace, and validity of observable success without baseline controls. No new physical or conceptual entities are introduced; 'Probe Session' and 'Stabilization Session' are methodological constructs, not invented entities.

axioms (3)
  • domain assumption Agentic systems treat instructions in auto-loaded memory files (CLAUDE.md/AGENTS.md) as trusted user-authored context.
    This is the threat model's premise; the paper states it in Section 3.1 and relies on it for all attack vectors. If false (e.g., if memory files are sandboxed or de-prioritized), the measured attacks would not generalize.
  • domain assumption The synthetic workspace for fictional user 'Jordan Kim' is representative of real users' memory-file configurations and benign tasks.
    The paper acknowledges in Limitations (Section 5) that real workflows may be more complex. The central claim does not require full realism, but the quantitative rates are conditional on this setup.
  • domain assumption Observable performance of the attacker's action (ASR) is a valid causal measure of payload influence without a no-payload baseline.
    ASR is computed from trials that all contain a payload; the paper never reports control runs without payloads, so natural base rates for actions like recommending credit cards could confound the measure. This is the weakest premise and is load-bearing for the numeric claims.

pith-pipeline@v1.3.0-alltime-deepseek · 17705 in / 12438 out tokens · 126826 ms · 2026-08-02T01:34:55.931099+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Bad Memory: Evaluating Prompt Injection Risks from Memory in Agentic Systems." pith.science (2026). https://pith.science/paper/YEU4FU2M

@misc{pith2026260714611,
  author       = {Pith},
  title        = {Pith review of: Bad Memory: Evaluating Prompt Injection Risks from Memory in Agentic Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YEU4FU2M}},
  note         = {Machine review of arXiv:2607.14611}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

A growing class of agentic systems maintain persistent state across sessions through memory files, behavioral preferences, and knowledge bases. While this makes agents more useful and self-improving, it also creates a new attack surface for prompt injections in which malicious instructions can be embedded within persistent files and influence future behavior. In this work, we study prompt injection attacks in memory-based agentic systems using a sandboxed synthetic workspace. We evaluate two agentic systems, Anthropic Claude Code and OpenAI Codex, across four models: Claude Haiku 4.5, Claude Opus 4.7, GPT-5.2, and GPT-5.5. Our results show that although it is difficult to make an agent overwrite its own memory files using untrusted external content, payloads already planted in those files can successfully attack current and future sessions. Attack success and payload persistence vary substantially across systems, models, adversarial goals, and multi-session attack sequences. These findings show that persistent memory changes the threat model for prompt injection and motivate defenses that protect memory updates without removing useful agent adaptation.

Figures

Figures reproduced from arXiv: 2607.14611 by David Alexander, Franziska Roesner, Sai Sunku, Soham Gadgil.

Figure 1
Figure 1. Figure 1: Workspace layout. Files named CLAUDE.md are auto￾loaded. All other files (behavior files under core/ and free￾form notes under knowledge/) are only loaded if the agent decides to read them. 2Example of a public CLAUDE.md file: https://github.com/multica-ai/andrej-karpathy￾skills/blob/main/CLAUDE.md [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Detailed setup for the adversarial goal of Credential Exfiltration. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Detailed setup for the adversarial goal of Unauthorized Tool Use. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Detailed setup for the adversarial goal of Brand Targeting. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: The trial in which Opus refused the planted install: [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Both GPT-5.2 and Haiku 4.5 remove the injected [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Opus identifies the brand injection and refuses to [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: GPT-5.2 does not recognise the unauthorized tool [PITH_FULL_IMAGE:figures/full_fig_p009_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Both GPT-5.2 and GPT-5.5 remove the injected at [PITH_FULL_IMAGE:figures/full_fig_p009_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

34 extracted references · 9 linked inside Pith

  1. [1]

    Deepak Bhaskar Acharya, Karthigeyan Kuppan, and B Divya. 2025. Agentic AI: Autonomous intelligence for complex goals—A comprehensive survey.IEEe Access13 (2025), 18912–18936

  2. [2]

    Anthropic. 2026. Claude 4.7 Opus. https://claude.ai Large language model

  3. [3]

    Anthropic. 2026. How Claude Remembers Your Project. https://docs.anthropic. com/en/docs/claude-code/memory. Accessed: 2026-06-10

  4. [4]

    Anthropic. 2026. Long-running Claude for Scientific Computing. https://www. anthropic.com/research/long-running-Claude. Accessed: 2026-06-10

  5. [5]

    Pallak Chahar, Yashika Kumari Vishwakarma, Renu Mishra, and Girish Paliwal

  6. [6]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qinyuan Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harrison Haris, Yuri Burda, Alex Hallacy, Scott Broadrick, et al. 2021. Evaluating Large Language Models Trained on Code.arXiv preprint arXiv:2107.03374(2021)

  7. [7]

    Edoardo Debenedetti, Jie Zhang, Mislav Balunović, Luca Beurer-Kellner, Marc Fischer, and Florian Tramèr. 2024. AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents. InNeurIPS Datasets and Benchmarks Track

  8. [8]

    Elicit. 2026. Elicit: AI for Scientific Research. https://elicit.com/. Accessed: 2026-06-10

  9. [9]

    Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. 2023. Not what you’ve signed up for: Compromising real- world llm-integrated applications with indirect prompt injection. InProceedings of the 16th ACM workshop on artificial intelligence and security. 79–90

  10. [10]

    Saidakhror Gulyamov, Said Gulyamov, Andrey Rodionov, Rustam Khursanov, Kambariddin Mekhmonov, Djakhongir Babaev, and Akmaljon Rakhimjonov. 2026. Prompt Injection Attacks in Large Language Models and AI Agent Systems: A Comprehensive Review of Vulnerabilities, Attack Vectors, and Defense Mecha- nisms.Information17, 1 (2026), 54

  11. [11]

    Idan Habler and Amy Chang. 2026. Identifying and Remediating a Persistent Memory Compromise in Claude Code. Accessed: 2026-06-10

  12. [12]

    Yuyang Hu, Shichun Liu, Yanwei Yue, Guibin Zhang, Boyang Liu, Fangyi Zhu, Jiahang Lin, Honglin Guo, Shihan Dou, Zhiheng Xi, et al. 2025. Memory in the age of ai agents.arXiv preprint arXiv:2512.13564(2025)

  13. [13]

    Yi Liu, Gelei Deng, Yuekang Li, Kailong Wang, Zihao Wang, Xiaofeng Wang, Tianwei Zhang, Yepang Liu, Haoyu Wang, Yan Zheng, et al. 2023. Prompt injection attack against llm-integrated applications.arXiv preprint arXiv:2306.05499(2023)

  14. [14]

    Pan Lu, Baolin Peng, Hao Cheng, Michel Galley, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, and Jianfeng Gao. 2023. Chameleon: Plug-and-play composi- tional reasoning with large language models.Advances in Neural Information Processing Systems36 (2023), 43447–43478

  15. [15]

    Junyu Luo, Weizhi Zhang, Ye Yuan, Yusheng Zhao, Junwei Yang, Yiyang Gu, Bohan Wu, Binqi Chen, Ziyue Qiao, Qingqing Long, et al. 2025. Large language model agent: A survey on methodology, applications and challenges.arXiv preprint arXiv:2503.21460(2025)

  16. [16]

    Schulhoff, Jamie Hayes, Michael Ilie, Juliette Pluto, Shuang Song, Harsh Chaudhari, Ilia Shumailov, Abhradeep Thakurta, Kai Yuanqing Xiao, Andreas Terzis, and Florian Tramèr

    Milad Nasr, Nicholas Carlini, Chawin Sitawarin, Sander V. Schulhoff, Jamie Hayes, Michael Ilie, Juliette Pluto, Shuang Song, Harsh Chaudhari, Ilia Shumailov, Abhradeep Thakurta, Kai Yuanqing Xiao, Andreas Terzis, and Florian Tramèr

  17. [17]

    OpenAI. 2026. Custom Instructions with AGENTS.md. https://developers.openai. com/codex/guides/agents-md. Accessed: 2026-06-10

  18. [18]

    OWASP Foundation. 2025. OWASP Top 10 for Agentic Applications. https: //genai.owasp.org/. Accessed: May 2026

  19. [19]

    Charles Packer, Vivian Fang, Shishir_G Patil, Kevin Lin, Sarah Wooders, and Joseph_E Gonzalez. 2023. MemGPT: towards LLMs as operating systems. (2023)

  20. [20]

    Dario Pasquini, Martin Strohmeier, and Carmela Troncoso. 2024. Neural exec: Learning (and learning from) execution triggers for prompt injection attacks. In Proceedings of the 2024 Workshop on Artificial Intelligence and Security. 89–100

  21. [21]

    Fábio Perez and Ian Ribeiro. 2022. Ignore previous prompt: Attack techniques for language models.arXiv preprint arXiv:2211.09527(2022)

  22. [22]

    Mark Russinovich, Ahmed Salem, and Ronen Eldan. 2025. Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack. In34th USENIX Security Symposium (USENIX Security 25). 2421–2440

  23. [23]

    Shoumik Saha, Kazem Faghih, and Soheil Feizi. 2026. Under the Hood of SKILL. md: Semantic Supply-chain Attacks on AI Agent Skill Registry.arXiv preprint arXiv:2605.11418(2026)

  24. [24]

    Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools.Advances in neural information processing systems36 (2023), 68539–68551

  25. [25]

    Snyk Security Labs. 2026. ToxicSkills: Malicious AI Agent Skills on ClawHub. https://snyk.io/blog/toxicskills-malicious-ai-agent-skills-clawhub/. Accessed: May 2026

  26. [26]

    2025.OpenClaw: Your own personal AI assistant

    Peter Steinberger. 2025.OpenClaw: Your own personal AI assistant. https://github. com/openclaw/openclaw

  27. [27]

    Md Nayem Uddin, Kumar Shubham, Eduardo Blanco, Chitta Baral, and Gengyu Wang. 2026. From recall to forgetting: Benchmarking long-term memory for personalized agents.arXiv preprint arXiv:2604.20006(2026)

  28. [28]

    Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et al. 2024. A survey on large language model based autonomous agents.Frontiers of Computer Science18, 6 (2024), 186345

  29. [29]

    Xianglin Yang, Yufei He, Shuo Ji, Bryan Hooi, and Jin Song Dong. 2026. Zom- bie agents: Persistent control of self-evolving LLM agents via self-reinforcing injections.arXiv preprint arXiv:2602.15654(2026)

  30. [30]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629(2022)

  31. [31]

    Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang. 2024. Memo- rybank: Enhancing large language models with long-term memory. InProceedings of the AAAI conference on artificial intelligence, Vol. 38. 19724–19731. Bad Memory: Evaluating Prompt Injection Risks from Memory in Agentic Systems Conference acronym ’XX, June 03–05, 2018, Woodstock, NY...

  32. [2025]

    The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against LLM Jailbreaks and Prompt Injections.arXiv preprint arXiv:2510.09023 (2025)

  33. [2026]

    A vailable at SSRN 6377518(2026)

    Artificial Intelligence Powered Personal Finance Management System. A vailable at SSRN 6377518(2026)

  34. [3248]

    I'm not sure about X -- want me to check ?

    The repository contains the code used to construct the sand- boxed synthetic workspace, run the multi-session agent experi- ments, and evaluate attack success and payload persistence. It also includes the experimental setup for the memory-based prompt injection scenarios studied in this paper, including the persistent workspace files, adversarial payloads...