Pith. sign in

REVIEW 3 major objections 21 references

An agent's remembered reasoning can be forged so it skips safety checks, and structural analysis of those traces can block the attack.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 09:50 UTC pith:5UTCRFDJ

load-bearing objection Clean attack on a real gap (reasoning-store integrity); the defense half is honest but brittle once the attacker leaves the template pool. the 3 major comments →

arxiv 2607.05029 v1 pith:5UTCRFDJ submitted 2026-07-06 cs.CR cs.AI

Your Agent's Memories Are Not Its Own: Forged Reasoning Attacks on LLM Agent Memory and Defenses

Neeraj Karamchandani , Piyush Nagasubramaniam , Sencun Zhu , Dinghao Wu This is my paper
classification cs.CR cs.AI
keywords LLM agentsagentic securitymemory poisoningreasoning trace forgerypersistent memoryFARMASENTINELadversarial machine learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Modern LLM agents keep persistent memory of their own past decisions and rationales so they can reuse work across tasks. This paper argues that those remembered reasoning traces are a distinct attack surface: an adversary who can write to the store can plant forged decision logs that use neutral language to claim validation or comparison was already done, then amplify them with self-citing entries until the forgeries become the consensus the agent trusts. The attack, FARMA, does not change facts or issue an overt 'skip safety' command; it makes the agent treat a required safety step as already completed. Existing keyword filters and consensus defenses fail because the language is evasive and the amplification makes the forged traces look normal. The authors answer with SENTINEL, a write-path pipeline whose central Reasoning Guard scores candidate entries on five structural forgery signals and rejects those above a threshold. Across three agent domains and three models, FARMA reaches up to 100% success while SENTINEL drops success to as low as 0% with no false positives on hundreds of benign traces.

Core claim

Poisoning an agent's own reasoning history—rather than its factual knowledge—is enough to induce unsafe behavior, because the agent treats retrieved decision logs as evidence of work it already performed. FARMA achieves this with evasive, structurally similar forged traces followed by self-referential amplification that defeats keyword and consensus defenses; SENTINEL's Reasoning Guard, scoring five weighted structural signals on write, is necessary and sufficient to reduce that attack success to near zero without false positives on the benign traces tested.

What carries the argument

FARMA (two-phase forged amplifying rationale memory attack) and SENTINEL's Reasoning Guard: a weighted score over five forgery signals—provenance anomaly, self-reference count mismatch against store size, decision-log phrase density, implausible perfect-consistency claims, and structural opening patterns—applied at write time to reasoning and reasoning-like memory entries.

Load-bearing premise

Forged reasoning entries will keep showing the fixed structural fingerprints (phrase density, self-reference counts, opening patterns, and the rest) that the Reasoning Guard thresholds on; the paper itself notes a paraphrased adaptive attacker that knows those patterns can bypass them.

What would settle it

A controlled experiment in which an adaptive attacker rewrites FARMA templates to avoid the five Reasoning Guard signals and still drives the agent to skip validation or comparison at high rate while SENTINEL remains in place; if that rate stays near the paper's undefended baseline, the Guard's load-bearing claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 0 minor

Summary. The paper identifies an LLM agent's reasoning store (decision logs, intermediate rationales, self-reflections) as a distinct memory-poisoning surface and introduces FARMA, a two-phase attack that injects evasive forged decision-log traces and then amplifies them via self-referential entries that cite prior forged counts. Amplification raises retrieval probability, manufactures consensus against A-MemGuard-style defenses, and supplies apparent precedent so the agent treats safety steps (validation, lookup, comparison) as already completed. The authors propose SENTINEL, a five-layer write-time pipeline whose central Reasoning Guard scores five structural signals (provenance anomaly, self-reference mismatch vs. store size, suspicious decision-log phrase density, implausible perfect-consistency claims, and opening-structure patterns). Evaluation across EHRAgent, ReAct-QA, and RAP on GPT-4o-mini, GPT-4o, and Llama 3.3 70B (50 trials, Wilson CIs, clean-memory differential protocol) reports FARMA ASR up to 100% against no defense, keyword filter, and both A-MemGuard modes, while SENTINEL reduces ASR to 0% (EHR/RAP) or 6% (ReAct-QA) with 0 FPR on 326 benign traces; ablation shows Reasoning Guard is necessary and sufficient against the evaluated FARMA.

Significance. If the results hold, the work cleanly separates reasoning-history integrity from factual/retrieval poisoning (AgentPoison, MINJA, MemoryGraft, PoisonedRAG) and shows that consensus-deviation defenses are defeated by self-amplifying forgeries that become the majority. The differential evaluation protocol, multi-model/multi-domain design, Wilson CIs, and clean ablation (Table II) are strengths that make the attack half of the claim reproducible and falsifiable. The practical implication—that agents must protect what they remember having reasoned, not only what they retrieve—is timely given real-world memory-write vectors (SpAIware, plugin OAuth flaws) and the OWASP ASI06 class. The defense contribution is more provisional: it demonstrates that structural write-time analysis can stop the template-based attacker, but the paper itself flags adaptive paraphrase as an open problem.

major comments (3)
  1. §VI and the central claim pairing (Abstract, §V, Table III): the 0% ASR result for SENTINEL is demonstrated only against the non-adaptive template attacker of Alg. 1. The paper states that a paraphrased adaptive FARMA variant that knows the Guard's patterns defeats Reasoning Guard and that the Guard 'did not provide significant protection.' Because Table II shows Layers 1–4 contribute nothing against FARMA and Layer 5 alone is load-bearing, the defense half of the strongest claim does not transfer to an attacker who rewrites the same semantic claims without the surface markers (s2 regexes, s3 phrase list, s5 openings). Either quantify adaptive ASR under a stated paraphrase budget or reframe the claim as 'effective against the template FARMA evaluated here; adaptive robustness left open.'
  2. §IV-E (signals s2–s5) and free parameters: the Guard's effectiveness rests on hand-curated phrase lists, tiered density thresholds (0.5/0.75/1.0 for 3+/4+/6+ matches), equal weights w_i=0.2, and τ_guard=0.5, plus the axiom that authentic logs exhibit lower density of those phrases and do not claim perfect consistency over large counts. No sensitivity analysis or cross-validation of these thresholds is reported. A short ablation over τ_guard and the density tiers (or leave-one-signal-out) would show whether the 0% ASR / 0% FPR result is robust or tuned to the evaluated template pool.
  3. Table III domain gap and §V-E interpretation: FARMA reaches 100% ASR on EHR (binary import decision, forged pipeline ID, no competing evidence) but only 48–52% on RAP/ReAct-QA where competing product listings or passages remain visible. The paper notes this but still leads with 'up to 100%.' The claim should be conditioned on decision structure (binary vs. multi-option with competing evidence), and the evaluation should state whether the forged traces were retrieved into the top-k context in the failed trials; otherwise it is unclear whether the lower ASR is a retrieval failure or a planner that ignores the forged precedent.

Circularity Check

0 steps flagged

No circularity: empirical attack/defense evaluation with differential ASR defined by observable agent actions, not by construction from fitted inputs or self-citation.

full rationale

The paper is an empirical systems-security contribution introducing FARMA (two-phase template-based injection + amplification of forged reasoning traces) and SENTINEL (layered filters whose load-bearing component is the five-signal Reasoning Guard). Attack success is measured differentially against a clean-memory baseline on the same task/seed: success requires the clean run to produce a safe action and the poisoned run to produce the attacker-target action (import_direct, skip lookup, purchase without comparison). These are externally observable agent behaviors, not quantities derived from parameters fitted to the same data. Thresholds (τ_guard = 0.5, phrase-density tiers, etc.) are free parameters of the defense; they do not define the ASR metric itself. Ablation (Table II) and cross-domain/model results (Table III) are experimental measurements, not first-principles predictions. Related-work citations (AgentPoison, MINJA, MemoryGraft, A-MemGuard) are used for qualitative comparison of attack surfaces and for baselines; none is a uniqueness theorem or load-bearing self-citation that forces the central claims. The adaptive-paraphrase limitation in §VI is an honesty about brittleness of the heuristic signals, not a circular reduction. No equation, definition, or derivation reduces a claimed result to its own inputs by construction. Score 0 is therefore the correct finding.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 3 invented entities

Empirical security paper; load-bearing content is the threat model, the hand-designed forgery templates and the five heuristic signals with their thresholds. No deep mathematical axioms; free parameters are the defense thresholds and attack injection volumes chosen by the authors.

free parameters (5)
  • τ_guard = 0.5
    Forgery-score rejection threshold for Reasoning Guard; set to 0.5 after experiments; directly controls ASR/FPR trade-off.
  • signal weights w_i = 0.2 each
    Equal weights 0.2 for the five Reasoning Guard signals; chosen by default rather than optimized.
  • seed count n_s and amplification cycles n_a = n_s=3, n_a=10
    Default 3 seeds and 10 amplification cycles; chosen to match prior injection volumes and to manufacture consensus.
  • suspicious-pattern density tiers = tiered 3/4/6 matches
    Match-count thresholds (0.5 for 3+, 0.75 for 4+, 1.0 for 6+) derived from analyzing FARMA templates vs. benign traces.
  • τ_taint and τ_risk
    Taint and risk thresholds for Layers 3–4; stated as configurable and set experimentally.
axioms (4)
  • domain assumption Attacker can write entries into the agent's persistent memory store (including spoofing source=AGENT and trust=VERIFIED) but cannot modify model weights, system prompt or observe internal execution state.
    Core threat model §II-C; justified by real incidents (SpAIware, Salt Labs plugins) and prior work (AgentPoison, MINJA).
  • domain assumption When an agent retrieves a stored reasoning trace it may treat that trace as evidence of work it already performed and therefore skip re-validation.
    Stated in §I and measured on 100 benign tasks (98% retrieval, 63% use for EHRAgent); without this reuse the attack has no effect.
  • ad hoc to paper Authentic agent decision logs exhibit lower density of certain decision-log phrases and do not claim perfect consistency over large run counts.
    Underpins signals s3 and s4 of the Reasoning Guard; derived from the authors' analysis of their own templates vs. captured benign traces.
  • domain assumption Standard information-flow / taint and keyword-filter techniques can be adapted as cheap outer layers of a memory-write filter.
    Layers 1–4; classical security techniques cited via provenance and TaintDroid literature.
invented entities (3)
  • FARMA (Forged Amplifying Rationale Memory Attack) no independent evidence
    purpose: Two-phase attack that injects evasive forged reasoning traces then amplifies them by self-reference to defeat keyword and consensus defenses.
    Central attack contribution; no independent existence outside this paper.
  • SENTINEL Reasoning Guard (five weighted structural signals) no independent evidence
    purpose: Detect forged reasoning entries that survive lexical and provenance filters by scoring provenance anomaly, self-reference mismatch, pattern density, implausible claims and structure.
    Core defense contribution; signals and aggregation are defined in this paper.
  • looks_like_reasoning classifier no independent evidence
    purpose: Apply Reasoning Guard also to general-memory entries that resemble decision logs so attackers cannot hide forgeries outside the reasoning store.
    Closes an obvious bypass path; defined ad hoc in §IV-F.

pith-pipeline@v1.1.0-grok45 · 18572 in / 3561 out tokens · 34218 ms · 2026-07-11T09:50:27.228000+00:00 · methodology

0 comments
read the original abstract

Persistent memory has enabled large language model (LLM) agents to store factual knowledge, prior decisions, reasoning histories, tool usage information, and context. While this has improved the agent's functionality and continuity across tasks, it has also introduced a new attack surface: the agent's own reasoning history. In this paper, we introduce the Forged Amplifying Rationale Memory Attack (FARMA), which poisons an agent's remembered reasoning rather than its factual knowledge. It inserts forged reasoning traces using evasive language that bypasses keyword-based defenses, then amplifies them through self-referential reinforcement that defeats consensus-based defenses. To address FARMA, we introduce SENTINEL, a layered defense pipeline to detect forged reasoning entries. Its central component is the Reasoning Guard that structurally analyzes candidate entries for forgery using five weighted signals. We evaluate FARMA and SENTINEL across multiple agents and different LLM models with 50 trials and show that FARMA achieves an attack success rate of up to 100% under baseline conditions and is capable of defeating defense mechanisms like keyword filter and A-MemGuard. Our evaluation also shows that SENTINEL reduces FARMA's attack success rate to as low as 0% with no false positives observed across 326 benign agent traces. Our work demonstrates the need to protect not only an agent's retrieved content but also the integrity of its reasoning history.

Figures

Figures reproduced from arXiv: 2607.05029 by Dinghao Wu, Neeraj Karamchandani, Piyush Nagasubramaniam, Sencun Zhu.

Figure 1
Figure 1. Figure 1: FARMA: The attacker first injects forged reasoning traces into [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: SENTINEL defense pipeline. Candidates pass through lightweight [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

21 extracted references · 3 linked inside Pith

  1. [1]

    Generative agents: Interactive simulacra of human behavior,

    J. S. Park, J. O’Brien, C. J. Cai, M. R. Morris, P. Liang, and M. S. Bernstein, “Generative agents: Interactive simulacra of human behavior,” inProceedings of the 36th annual acm symposium on user interface software and technology, 2023, pp. 1–22

    2023

  2. [2]

    Re- flexion: Language agents with verbal reinforcement learning,

    N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao, “Re- flexion: Language agents with verbal reinforcement learning,”Advances in neural information processing systems, vol. 36, pp. 8634–8652, 2023

    2023

  3. [3]

    SpAIware: Uncovering a novel artificial intelligence attack vector through persistent memory in LLM applications and agents,

    M. Binhammad, M. Herrador, and J. Rehberger, “SpAIware: Uncovering a novel artificial intelligence attack vector through persistent memory in LLM applications and agents,”Future Generation Computer Systems, vol. 169, p. 107672, 2025

    2025

  4. [4]

    Security flaws within ChatGPT extensions allowed access to accounts on third-party websites and sensitive data,

    Salt Labs, “Security flaws within ChatGPT extensions allowed access to accounts on third-party websites and sensitive data,” Salt Security Blog, March 2024. [Online]. Avail- able: https://salt.security/blog/security-flaws-within-chatgpt-extensions- allowed-access-to-accounts-on-third-party-websites-and-sensitive-data

    2024

  5. [5]

    OW ASP top 10 for agentic applications 2026,

    OW ASP GenAI Security Project, “OW ASP top 10 for agentic applications 2026,” OW ASP Foundation, December 2025. [Online]. Available: https://genai.owasp.org/resource/owasp-top-10-for- agentic-applications-for-2026/

    2026

  6. [6]

    Agentpoison: Red- teaming llm agents via poisoning memory or knowledge bases,

    Z. Chen, Z. Xiang, C. Xiao, D. Song, and B. Li, “Agentpoison: Red- teaming llm agents via poisoning memory or knowledge bases,”Ad- vances in Neural Information Processing Systems, vol. 37, pp. 130 185– 130 213, 2024

    2024

  7. [7]

    A practical memory injection attack against llm agents,

    S. Dong, S. Xu, P. He, Y . Li, J. Tang, T. Liu, H. Liu, and Z. Xiang, “A practical memory injection attack against llm agents,”arXiv preprint arXiv:2503.03704, 2025

    arXiv 2025

  8. [8]

    Memorygraft: Persistent compromise of llm agents via poisoned experience retrieval,

    S. S. Srivastava and H. He, “Memorygraft: Persistent compromise of llm agents via poisoned experience retrieval,”arXiv preprint arXiv:2512.16962, 2025

    arXiv 2025

  9. [9]

    Ehragent: Code empowers large language models for few-shot complex tabular reasoning on electronic health records,

    W. Shi, R. Xu, Y . Zhuang, Y . Yu, J. Zhang, H. Wu, Y . Zhu, J. C. Ho, C. Yang, and M. D. Wang, “Ehragent: Code empowers large language models for few-shot complex tabular reasoning on electronic health records,” inProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024, pp. 22 315–22 339

    2024

  10. [10]

    Mem0: Building production-ready ai agents with scalable long-term memory,

    P. Chhikara, D. Khant, S. Aryan, T. Singh, and D. Yadav, “Mem0: Building production-ready ai agents with scalable long-term memory,” arXiv preprint arXiv:2504.19413, 2025

    Pith/arXiv arXiv 2025

  11. [11]

    React: Synergizing reasoning and acting in language models,

    S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. R. Narasimhan, and Y . Cao, “React: Synergizing reasoning and acting in language models,” inThe eleventh international conference on learning representations, 2022

    2022

  12. [12]

    Memorybank: En- hancing large language models with long-term memory,

    W. Zhong, L. Guo, Q. Gao, H. Ye, and Y . Wang, “Memorybank: En- hancing large language models with long-term memory,” inProceedings of the AAAI conference on artificial intelligence, vol. 38, no. 17, 2024, pp. 19 724–19 731

    2024

  13. [13]

    A-memguard: A proactive defense framework for llm-based agent memory,

    Q. Wei, T. Yang, Y . Wang, X. Li, L. Li, Z. Yin, Y . Zhan, T. Holz, Z. Lin, and X. Wang, “A-memguard: A proactive defense framework for llm-based agent memory,”arXiv preprint arXiv:2510.02373, 2025

    arXiv 2025

  14. [14]

    Mimic-iii, a freely accessible critical care database,

    A. E. Johnson, T. J. Pollard, L. Shen, L.-w. H. Lehman, M. Feng, M. Ghassemi, B. Moody, P. Szolovits, L. Anthony Celi, and R. G. Mark, “Mimic-iii, a freely accessible critical care database,”Scientific data, vol. 3, no. 1, pp. 1–9, 2016

    2016

  15. [15]

    Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies,

    M. Geva, D. Khashabi, E. Segal, T. Khot, D. Roth, and J. Berant, “Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies,”Transactions of the Association for Computational Linguistics, vol. 9, pp. 346–361, 2021

    2021

  16. [16]

    Rap: Retrieval-augmented planning with contextual memory for multimodal llm agents,

    T. Kagaya, T. J. Yuan, Y . Lou, J. Karlekar, S. Pranata, A. Kinose, K. Oguri, F. Wick, and Y . You, “Rap: Retrieval-augmented planning with contextual memory for multimodal llm agents,”arXiv preprint arXiv:2402.03610, 2024

    Pith/arXiv arXiv 2024

  17. [17]

    Poisonedrag: Knowledge corruption attacks to retrieval-augmented generation of large language models,

    W. Zou, R. Geng, B. Wang, and J. Jia, “Poisonedrag: Knowledge corruption attacks to retrieval-augmented generation of large language models,” in34th USENIX Security Symposium (USENIX Security 25), 2025, pp. 3827–3844

    2025

  18. [18]

    Poisoning re- trieval corpora by injecting adversarial passages,

    Z. Zhong, Z. Huang, A. Wettig, and D. Chen, “Poisoning re- trieval corpora by injecting adversarial passages,”arXiv preprint arXiv:2310.19156, 2023

    Pith/arXiv arXiv 2023

  19. [19]

    Watch out for your agents! investigating backdoor threats to llm-based agents,

    W. Yang, X. Bi, Y . Lin, S. Chen, J. Zhou, and X. Sun, “Watch out for your agents! investigating backdoor threats to llm-based agents,”Ad- vances in Neural Information Processing Systems, vol. 37, pp. 100 938– 100 964, 2024

    2024

  20. [20]

    Provenance-aware storage systems

    K.-K. Muniswamy-Reddy, D. A. Holland, U. Braun, and M. I. Seltzer, “Provenance-aware storage systems.” inUSENIX ATC, General Track, 2006, pp. 43–56

    2006

  21. [21]

    Taintdroid: an information- flow tracking system for realtime privacy monitoring on smartphones,

    W. Enck, P. Gilbert, S. Han, V . Tendulkar, B.-G. Chun, L. P. Cox, J. Jung, P. McDaniel, and A. N. Sheth, “Taintdroid: an information- flow tracking system for realtime privacy monitoring on smartphones,” ACM Transactions on Computer Systems (TOCS), vol. 32, no. 2, pp. 1–29, 2014

    2014