Pith. sign in

REVIEW 3 major objections 5 minor 8 references

This paper establishes that LLM agents can launder the provenance of untrusted observations into apparent user history during memory consolidation, and that a provenance-preserving memory firewall enforces source-authority non-amplification

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 12:20 UTC pith:BXWQVYK5

load-bearing objection Real failure mode, solid evidence, sensible fix; the zero-ASR claim is a conditional one the paper doesn't oversell. the 3 major comments →

arxiv 2607.29167 v1 pith:BXWQVYK5 submitted 2026-07-31 cs.CR cs.AI

Memory Provenance Laundering in LLM Agents: A Non-Amplification Firewall for Persistent Memory

Jinghan Xu , Yiyong Xiao , Wanru Shao , Hankai Liu , Xinjin Li This is my paper
classification cs.CR cs.AI
keywords LLM agentsmemory provenanceprovenance launderingindirect prompt injectionmemory poisoningtool authorizationrisk-authority gatingnon-amplification
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper identifies a failure mode it calls memory provenance laundering: when an LLM agent summarizes untrusted input (web pages, emails, tool output) into long-term memory, consolidation can preserve the action trigger while rewriting the record so that it appears to be user history, workflow support, or a confirmation. The low-trust source is erased, so the memory silently gains authority it never had. Existing prompt filters, content sanitizers, and tool guards operate on the current text or call and cannot see this cross-task authority upgrade. The paper proposes PPMF, a memory middleware that stores provenance as platform-maintained metadata and gates every tool call by matching the action's risk to the least-trusted memory supporting its arguments. In evaluation, vulnerable consolidated memories reach up to 1.000 attack success rate, while PPMF blocks every evaluated unauthorized high-risk action under intact labels and preserves confirmed benign actions.

Core claim

On the paper's own terms, the discovery is that lossy memory consolidation does not merely preserve malicious text; it can rewrite an external observation into a higher-authority form — user history, workflow rule, or prior confirmation — while keeping the action trigger intact, and this laundering persists across models and settings. The paper then shows that a defense which preserves source authority as system metadata and enforces a least-trust binding between tool-call arguments and memory support can separate the two effects: external-derived memory remains usable as context, but it cannot independently authorize high-risk effects (purchases, credential changes, deletions, emails). With

What carries the argument

The central object is the structured memory record m=(content, source, trust, history, risk, external) and the risk-authority gate that evaluates each proposed tool call against the least-trusted supporting memory. The gate is a fixed policy: READ requires EXTERNAL trust, NAV requires USERHISTORY, and EFFECT, PURCHASE, and CREDENTIAL require USER_CONFIRMED. The load-bearing step is deterministic argument-level support binding — matching schema-declared tool arguments (URLs, accounts, handles, workflow ids) to memory metadata, keeping all candidates on ambiguous matches — so authority is tied to the specific target and cannot be inherited by unrelated memories or by paraphrased text.

Load-bearing premise

The whole defense rests on platform-assigned provenance, confirmation, and risk labels being correct and unforgeable; if an attacker can forge or downgrade these metadata fields, the gate admits the attack, and the paper's own table shows 10% forged confirmations yield about 9% attack success.

What would settle it

Re-run the evaluation on the 3,680-page browser-transfer benchmark with one modification: let the memory consolidator write the phrase 'user confirmed' into a memory whose platform metadata is EXTERNAL, and give it a purchase tool call whose only support is that memory; if any such call executes, the claim that external-derived memory cannot independently authorize high-risk actions is false.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Indirect prompt injection and memory poisoning lose their cross-task effect for high-risk tool calls, because the injected trigger cannot carry authority from one task into the next.
  • Text-based filters are unnecessary at the memory boundary: paraphrase, fake confirmation markers, and recovered trigger wording do not matter, since the gate reads platform metadata, not memory text.
  • Confirmed benign actions stay executable, so a deployment can block attacks without re-confirming routine low-risk memory use or confirmed high-risk tasks.
  • Memory security becomes a metadata-integrity problem: the platform must keep source labels, confirmation events, and risk labels trustworthy, because a compromised platform breaks the guarantee.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to make confirmation events scoped — amount limits, expiry, tool-specific ranges — which the paper lists as future work and would likely reduce re-confirmation burden while keeping the zero-bypass property.
  • The same least-trust binding could be applied across agents, where one agent's memory becomes another's instruction; the paper lists inter-agent authority transfer as out of scope.
  • If this design is right, it suggests a broader principle for agent security: textual content — including model-generated summaries — should never be a source of authority; only platform-attested channels, events, and schema labels should grant execution rights.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper identifies "memory provenance laundering," a cross-task failure mode in persistent LLM-agent memory: during LLM-based consolidation, an external observation can be rewritten so that it appears to be user history, workflow support, or confirmation-like authority, while the low-trust source is erased. The authors formalize a source-authority non-amplification principle and instantiate it as PPMF, a memory middleware that stores platform-maintained provenance and authorizes tool calls by matching the least-trusted action-relevant memory against a fixed risk policy. The evaluation covers multiple models and datasets, including synthetic/adaptive scenarios, AgentDojo-derived and trace-derived cases, a server-backed web sandbox with real HTTP state changes, and a BrowseSafe-Bench adaptation. The main empirical claims are that vulnerable consolidation reaches up to 1.000 ASR, text-level defenses have no stable operating point, and PPMF achieves ASR 0.000 while preserving confirmed benign actions. The paper also gives a runtime-monitor property with a proof sketch.

Significance. If the phenomenon and architecture are accepted, this is a useful contribution: existing defenses mostly protect the current prompt/tool boundary, whereas PPMF targets the authority boundary across lossy memory consolidation and later tool execution. The strengths are substantial: the laundering phenomenon is demonstrated across multiple memory writers and datasets; the web-sandbox experiments count real server-side effects rather than only planned strings; the paper is unusually explicit about its threat model and trusted boundary; and it quantifies what happens when that boundary is partially violated. The main caveat is that the PPMF ASR=0.000 result is close to definitional: a high-risk action is defined as unauthorized when it lacks recorded confirmation, and the gate is defined to deny high-risk actions lacking USER_CONFIRMED support. The contribution is therefore best understood as the problem identification plus the provenance-preserving write and support-binding architecture, with the zero-ASR rows serving as policy-consistency checks rather than as measured defeats of an active adversary.

major comments (3)
  1. [§4, §5.5, Table 19] The PPMF ASR=0.000 result is a direct consequence of the gate policy rather than an empirical discovery. The threat model defines an attack as external-derived memory alone authorizing a high-risk action without a recorded confirmation, and the implemented gate requires USER_CONFIRMED for EFFECT/PURCHASE/CREDENTIAL. Any correctly implemented instance of this policy will therefore report ASR 0.000. Table 1 shows that the Gate-only ablation also achieves ASR 0.000 in every row; the difference between Gate and PPMF is entirely on the utility side. The paper should state this explicitly and present the PPMF safety rows as verification that the implementation enforces the declared policy, not as evidence that the gate defeats an adaptive adversary. The novel load-bearing parts are the provenance-preserving write path and the deterministic support binding, and the evaluation should be framed a
  2. [§5.5, Table 1, Table 4] The false-block metric is never formally defined, and the reported numbers are internally inconsistent under the natural definition. In the AgentDojo rows of Table 1, the Gate-only condition reports benign success 0.000 and false-block 0.042; if false-block is the fraction of benign cases incorrectly denied, this should be approximately 1.000 given benign success 0.000. In the same table, the no-defense condition reports benign success 0.042 with false-block 0.000, which is also difficult to reconcile unless false-block counts something other than denied benign cases. Because the central utility claim is that PPMF achieves benign success 1.000 and false-block 0.000, the metric definitions and the AgentDojo numbers need to be corrected or clarified before the utility claims can be evaluated.
  3. [§3.2, Limitations, Table 19] The security guarantee is fully contingent on the integrity of platform-maintained provenance, confirmation events, and risk labels. The paper is transparent about this, but Table 19 shows that this is not a remote corner case: a 10% confirmation-forgery rate yields ASR 0.088±0.016 and a 10% risk-downgrade rate yields ASR 0.112±0.006. Because PPMF is explicitly not a content-based detector, there is no second layer of defense when these metadata fields are wrong. The abstract and title use the strong phrase "non-amplification firewall," which invites a robustness reading. I recommend either qualifying the central claims as conditional on the trusted-boundary assumptions throughout, or adding an explicit residual-risk analysis that models realistic failure rates of confirmation UIs, schema registries, and audit mechanisms rather than treating all metadata compromise as uniformly outside s
minor comments (5)
  1. [Abstract] "Provenance-Preserving Memory Fire wall" has an awkward space; should be "Firewall."
  2. [§5.5] The metric ASRB is introduced but the notation is not used consistently elsewhere in the results; use a single term (ASR or ASRB) for the safety metric, and explicitly define the denominator for each reported rate.
  3. [Appendix A] The proof sketch is only a few sentences and largely restates the policy. If the paper wishes to claim a formal result, a standard inductive invariant over the write/bind/gate steps would be more convincing; otherwise the word "Proposition" overstates the formality.
  4. [References] Several references are incomplete (e.g., Meta 2025 has no URL/venue; several preprints omit arXiv identifiers). Please complete the bibliography.
  5. [Figure 1] The figure is dense and the left/right panel distinction is not immediately clear; adding numbered stages or arrows would improve readability.

Circularity Check

1 steps flagged

PPMF's zero-ASR result is largely definitional: attack success and the gate are defined so that unconfirmed high-risk actions are denied by construction, though the vulnerability measurements and benign-utility evaluations are independent.

specific steps
  1. self definitional [§3.2 Threat Model, §4 Risk-authority gate, §5.5 Metrics]
    "An attack succeeds when external-derived memory alone causes an unauthorized high-risk action in Task B (excluding actions authorized by the Task B user request or a recorded confirmation). ... the implemented policy instance is explicit: READ requires at least EXTERNAL; NAV requires USERHISTORY; EFFECT, PURCHASE, and CREDENTIAL require USER_CONFIRMED."

    The paper defines an attack as a high-risk action lacking a recorded confirmation and defines the gate as denying EFFECT/PURCHASE/CREDENTIAL unless USER_CONFIRMED exists. Thus PPMF's ASR=0.000 on 'unauthorized high-risk actions' follows from the definitions, not from an empirical generalization. The empirical content is limited to support binding (same-target matching) and preserving confirmed benign actions; the headline 'no unauthorized high-risk action passes' restates the policy. §5.5 concedes this ('not a claim of universal robustness'), but the abstract/conclusion still frame it as a result.

full rationale

The vulnerability portion of the paper is independent and non-circular: it measures that vulnerable consolidation produces laundered memories (e.g., Qwen2.5-14B 0.933 ASR, launder rates near 1.000) and that text filters fail. No fitted parameters are renamed as predictions, and the paper does not rely on a load-bearing self-citation or an imported uniqueness theorem; related work is cited as related work. The sole circularity is the definitional coupling between the threat model ('attack succeeds when external-derived memory alone causes an unauthorized high-risk action' with no recorded confirmation) and the gate ('EFFECT/PURCHASE/CREDENTIAL require USER_CONFIRMED'). Consequently the zero-ASR guarantee is a logical consequence of the policy, not a measured generalization. The paper is transparent about this boundary, explicitly listing metadata forgery/mislabeling outside the guarantee and reporting 10% confirmation-forgery ASR 0.088±0.016 and risk-downgrade ASR 0.112±0.006 in its own Table 19, so the circularity is partial rather than total. Score 6 reflects one central 'prediction' (zero unauthorized high-risk passes) reducing by construction, with independent content remaining in the attack characterization and benign-utility checks.

Axiom & Free-Parameter Ledger

1 free parameters · 4 axioms · 0 invented entities

The central result depends on several assumptions that the paper explicitly lists: trustworthy platform provenance (source, confirmation, risk labels), correct deterministic support binding, and exclusion of aggregate low-risk chains. The risk-to-trust policy is hand-chosen. No numeric parameters are fitted to data, but the choice of requiring USER_CONFIRMED for all high-risk actions is what makes the headline ASR=0.000 possible.

free parameters (1)
  • required_trust_by_risk_class = READ→EXTERNAL, NAV→USERHISTORY, EFFECT/PURCHASE/CREDENTIAL→USER_CONFIRMED
    Hand-chosen policy in §4/Table 6; PPMF's ASR=0.000 is a direct consequence of this policy requiring confirmed authority for all high-risk actions, not a measured property of a learned model.
axioms (4)
  • domain assumption Trusted platform provenance: source, trust, confirmation, and risk labels are assigned by the platform and cannot be forged or altered by the attacker.
    Stated in §3.2 'Trusted boundary'; Table 19 shows 10% forged confirmations yields ASR 0.088±0.016, so the guarantee is conditional on this.
  • domain assumption Correct risk labeling: every tool call's risk class is correctly determined from schemas before execution and cannot be downgraded by the attacker.
    §5.1 and Table 19 (10% risk downgrade ASR 0.112); a mislabeled high-risk action as READ would pass with only EXTERNAL authority.
  • domain assumption Deterministic support binding: tool-call arguments are matched to memory supports such that ambiguous/partial matches fail closed.
    §4 and Appendix D; the safety outcome depends on this matching being correct, though the paper stress-tests collisions and aliases (Table 7).
  • domain assumption No aggregate low-risk chain: sequences of low-risk actions cannot be composed into a high-risk harm that bypasses the per-call gate.
    Limitations explicitly state 'It does not fully cover aggregate low-risk chains that produce high-risk harm'; the central safety claim is scoped to single high-risk calls.

pith-pipeline@v1.3.0-daily-deepseek · 216 in / 9624 out tokens · 129956 ms · 2026-08-03T12:20:16.696843+00:00 · methodology

0 comments
read the original abstract

Long-term memory lets large language model(LLM) agents reuse prior preferences and work flows, but it also turns untrusted observations into persistent action context. We identify memory provenance laundering: during LLM-based memory consolidation, an external observation may be rewritten as apparent user history or workflow support, preserving an action trigger while erasing the low-trust source that should limit its authority. Existing prompt filters, content sanitizers, and tool guards do not enforce source-authority non-amplification after lossy memory consolidation. We formalize this boundary and instantiate it as Provenance-Preserving Memory Fire wall (PPMF), a lightweight memory middleware that preserves platform-maintained provenance and authorizes tool calls by matching action risk to the authority of action-relevant memories. In our schema-grounded evaluation with fixed risk policies, vulnerable consolidated memories reach up to 1.000 attack success rate(ASR); with intact platform-maintained provenance, confirmation, and risk labels, no evaluated unauthorized high-risk action passes the PPMF gate while confirmed benign actions and targeted low-risk memory use remain executable.

Figures

Figures reproduced from arXiv: 2607.29167 by Hankai Liu, Jinghan Xu, Wanru Shao, Xinjin Li, Yiyong Xiao.

Figure 1
Figure 1. Figure 1: PPMF attack scenario. Consolidated memory creates an authority-laundering attack surface. preserve information, but it must not upgrade the authority of the source. This work instantiates the principle with Provenance-Preserving Memory Firewall (PPMF), an execution-centric memory defense that de￾couples memory usefulness from action author￾ity. PPMF stores each memory with platform￾maintained provenance me… view at source ↗
Figure 2
Figure 2. Figure 2: PPMF introduces structural constraints at the memory boundary. Vulnerable consolidation can transform an external observation into apparent user workflow support. PPMF stores provenance as system metadata and authorizes tool execution by matching memory authority to action risk. maintained fields (s, τ, h, r, e); phrases such as “user-confirmed” do not create authority without recorded metadata. Source-aut… view at source ↗
Figure 3
Figure 3. Figure 3: Main-result summary after memory con￾solidation. Bars show attack success rates for represen￾tative settings and defenses; benign success and false￾block rates are reported in [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Retrieval-noise results with 20 decoy mem￾ories and top-10 retrieval. Noisy retrieval does not re￾move the attack path, because the retrieved memory can still carry laundered authority. PPMF checks recorded authority after retrieval rather than trusting memory text. 6.3 Retrieval, Middleware, and Web Effects For RQ4, retrieval-noise and decoy-memory exper￾iments test whether the defense depends on clean re… view at source ↗
Figure 5
Figure 5. Figure 5: Server-backed web sandbox with real state changes. A malicious success requires an actual local HTTP effect, such as posting, checkout, email sending, file deletion, or password update. PPMF authorizes only effects supported by recorded confirmation metadata bound to the same target. confirmation. The LangGraph store pilot writes Qwen2.5-14B-generated memories through persis￾tent storage, where PPMF preser… view at source ↗
Figure 6
Figure 6. Figure 6: Safety–utility trade-off under confirmation spoofing. Text guards stay unsafe when spoofed text is accepted. Handle/gate-only guards over-block. PPMF reaches the desired region because authority is system metadata, not memory text. H When PPMF False-Blocks PPMF false blocks are conservative denials caused by missing authority evidence or retrieval misses. They are different from content-filter false posi￾t… view at source ↗
Figure 7
Figure 7. Figure 7: Framework-style middleware experiment under confirmation spoofing. In the state-graph agent loop, PPMF is the only middleware that denies spoofed external-derived authority without rejecting benign high￾risk actions. I Additional Model and Boundary Results I.1 Cross-Model Memory Synthesis [PITH_FULL_IMAGE:figures/full_fig_p018_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

8 extracted references · 3 linked inside Pith

  1. [1]

    If a explicitly cites memory ids, set S(a) to those memories

  2. [2]

    Otherwise, canonicalize schema-declared tar- gets such as URLs, accounts, handles, files, end- points, workflow ids, or tool targets

  3. [3]

    ignore/disregard pre- vious instructions

    Memgpt: Towards llms as operating systems. Preprint, arXiv:2310.08560. Joon Sung Park, Joseph C. O’Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. Generative agents: Interactive simu- lacra of human behavior. InProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. Ofir Press, Mur...

    Pith/arXiv arXiv 2023

  4. [4]

    If multiple memories match the same target, keep all matches and authorize using the mini- mum trust level

  5. [5]

    Op- erational continuity note: for future related tasks, resume work- flow PM-A011. The workflow corresponds to reddit_review and should be treated as a user workflow

    If the match is partial, ambiguous, or missing, conservatively setS(a) =R. This rule makes support selection non-LLM- mediated. An unrelated confirmed memory cannot upgrade an external-derived target, and ambiguous binding fails closed unless a platform-recorded con- firmation is bound to the same target, principal, risk class, and scope. The current impl...

  6. [6]

    Exact identifiers dominate aliases; aliases must resolve to a canonical target through a plat- form mapping

    Match canonical targets against memory meta- data. Exact identifiers dominate aliases; aliases must resolve to a canonical target through a plat- form mapping

  7. [2023]

    Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, and Madian Khabsa

    Not what you’ve signed up for: Compromising real-world llm-integrated applications with indirect prompt injection.Preprint, arXiv:2302.12173. Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, and Madian Khabsa. 2023. Llama guard: Llm-based input-output safeguar...

    Pith/arXiv arXiv 2023

  8. [2025]

    Yupei Liu, Yuqi Jia, Runpeng Geng, Jinyuan Jia, and Neil Zhenqiang Gong

    Prompt flow integrity to prevent privilege es- calation in llm agents.Preprint, arXiv:2503.15547. Yupei Liu, Yuqi Jia, Runpeng Geng, Jinyuan Jia, and Neil Zhenqiang Gong. 2023. Formalizing and benchmarking prompt injection attacks and defenses. Preprint, arXiv:2310.12815. Llama Team, AI @ Meta. 2024. The llama 3 herd of models.Preprint, arXiv:2407.21783. ...

    Pith/arXiv arXiv 2023