{"id":"8d9facdb-c03c-4807-adfc-eb0361a100f5","arxiv_id":"2607.14651","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Write-time memory filters suppress direct single-record poisoning in LLM agents (L1 BCR ≈ 4.8%) but leave compositional (L2 ≈ 22.5%) and trigger-dormant (L3 ≈ 27.8%) poisoning substantially intact across ten model families.","lead":"MemPoison builds a benchmark of 1,227 hand-checked cases that poison the long-term memory of AI agents in three ways: direct overwrites, attacks split across several harmless-looking records, and dormant instructions that wake up on a specific trigger. Across ten AI models the paper finds that write-time filters block most direct attacks but keep missing the split and dormant ones, and proposes a diagnostic (MID) to show why.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"MID's interaction term has the wrong sign for its stated L2 mechanism: Table 3's L2 numbers imply individual fragments are nearly as influential as joint removal, undermining the compositional explanation.","rationale":"The reader's weakest_assumption focused on selection bias: the human-review filter requiring surface-benign injections may pre-ordain the qualitative L2/L3 frontier. That is a legitimate representativeness concern, but the more decisive problem is internal to the reported measurements. Even if the benchmark pack is representative, the MID statistics for L2 do not support the compositional mechanism they are used to establish. A positive Omega_g as defined by Eq. 17 indicates subadditivity or redundancy, not synergistic composition. Table 3's aggregate values force single-fragment effects to sum to 0.654 versus the joint effect 0.478, meaning individual L2 fragments already explain most of the score drop. This is the opposite of 'individually plausible fragments become harmful only when co-retrieved.' The reader's verdict of CONDITIONAL is appropriate: the empirical BCR frontier and benchmark construction are valuable, but the mechanistic explanation for L2 is internally inconsistent and must be corrected or restated. I therefore keep the reader's conditional disposition rather than moving to accept or reject, because the underlying defense-frontier numbers may survive reanalysis even if the MID interpretation does not.","tokens_in":35619,"tokens_out":5372,"duration_ms":60206,"concrete_test":"Recompute the L2 MID row in Table 3 per-case using Eq. 17: report the distribution and means of Delta_s_i and Delta_s_j for each designated fragment pair, and individual-fragment P2C rates. If mean(Delta_s_i + Delta_s_j) exceeds d(r, r_{-ij}) for a majority of cases (as Table 3's aggregate implies), the positive Omega_g is a redundancy signal, not a composition signal. Then re-run the L2 analysis with the corrected sign convention Omega_g = d(r, r_{-ij}) - (Delta_s_i + Delta_s_j); if this corrected term is negative in most cases, the claim that L2 is compositional is not supported by MID.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central mechanistic claim for L2 is that individually benign fragments become harmful only through joint retrieval and composition. That claim is directly contradicted by the paper's own MID equations and Table 3. Section 4.4 defines the interaction signal as Omega_g = Delta_s_i + Delta_s_j - d(r, r_{-ij}) (Eq. 2/17) and asserts that a positive Omega_g means 'corruption emerges through interaction rather than through a single dominant record.' But algebraically, a positive Omega_g means the joint-removal effect is smaller than the sum of single-removal effects -- i.e., the two fragments are partially redundant, so at least one individual fragment already has substantial influence on its own. Table 3 reports L2 Delta_g = 0.478 and Omega_g = 0.176. Substituting into Eq. 17 forces Delta_s_i + Delta_s_j = 0.654, so the mean single-fragment removal effect is about 0.327 -- larger than the L1 single-record effect of 0.266. Appendix B.7 confirms that positive Omega reflects 'overlap or non-additivity,' not synergy. Thus, the data imply that individual L2 fragments are not benign in isolation; they have single-record influence comparable to L1. If so, the claimed L1/L2 structural distinction collapses: write-time defenses are not failing because of deferred composition, but because individually harmful records pass the human-review filter and the tested defenses. This directly weakens the abstract's 'structural blind spots ... through joint retrieval composition' claim for L2.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces MemPoison, a benchmark and analysis framework for persistent memory poisoning in LLM agents. It contributes a curated pack of 1,227 hand-validated cases spanning a three-level taxonomy (L1 single-record corruption, L2 compositional multi-record corruption, L3 context-triggered dormant corruption), three injection channels, four attack types, and three memory substrates. Experiments on ten model families compare an undefended baseline against eleven defense configurations, including write-time consistency checks, anomaly filters, judge-based admission, retrieval-time source reweighting, and sanitization. The headline finding is a 'defense frontier': write-time defenses substantially suppress L1 but fail to reliably suppress L2 and L3. To explain this, the paper proposes Mechanistic Influence Decomposition (MID), a counterfactual diagnostic that attributes residual corruption to single-record influence, pairwise interaction, and trigger-conditioned activation.","tokens_in":35810,"tokens_out":3688,"duration_ms":44109,"significance":"If the claims hold, MemPoison would be a substantial empirical contribution: it is broader than most prior memory-poisoning studies, uses a paired clean/poisoned protocol, a deterministic contrastive evaluator, five repeated runs with reported standard deviations, disclosed defense calibration, and a family-capped, human-reviewed construction pipeline. The L1/L2/L3 taxonomy and the MID diagnostic are potentially reusable by the community, and the central policy suggestion — that pointwise write-time filtering is insufficient for securing persistent agent memory — is timely and consequential. However, as detailed in the major comments, the L2 mechanistic interpretation is contradicted by the paper's own interaction-sign algebra, and the qualitative 'structural blind spot' claim is partly pre-ordained by dataset construction choices. These issues do not invalidate the benchmark itself, but they require a substantial re-analysis and a more careful statement of what is empirical versus definitional.","major_comments":[{"comment":"The interpretation of Ω_g is algebraically wrong and undermines the L2 mechanistic claim. Eq. (2)/(17) defines Ω_g = Δs_i + Δs_j − d(r, r_{−ij}). A positive Ω_g means the joint-removal effect is smaller than the sum of single-removal effects — i.e., the two single-fragment effects overlap, so at least one fragment already has substantial influence alone. The text instead claims that positive Ω_g 'indicates that corruption emerges through interaction rather than through a single dominant record.' Appendix B.7 even states the opposite: positive Ω_g reflects 'overlap or non-additivity.' Table 3 reports Δ_g = 0.478 and Ω_g = 0.176; substituting into Eq. (17) forces Δs_i + Δs_j = 0.654, so the mean single-fragment effect is about 0.327 — larger than the L1 single-record effect of 0.266. This directly contradicts the assertion that L2 fragments are individually benign and that harm arises only","section":"§4.4, Eq. (2); §6, Table 3; App. B.7, Eq. (17)"},{"comment":"The qualitative 'write-time defenses miss L2/L3' finding is partly circular. L2 is defined as 'circumventing per-record write-time defenses' and L3 as 'structurally bypassing behavior write-time checks,' while the human-review filter retains only cases where the injected content 'appears benign on the surface' and the trigger is 'non-leaky.' Thus, by construction, every retained L2/L3 case must look benign at write time, guaranteeing that a write-time-only defense will miss it. What remains empirical is the magnitude (e.g., L2 BCR 22.54%, L3 27.80% under consistency checking) and the MID/pipeline quantities, not the existence of the frontier. The paper should distinguish the definitional statement from the empirical claim, and ideally analyze the rejected pool or a broader sample of plausible L2/L3 variants to estimate how much of the attack space the curated pack represents. As written,","section":"§3.2; App. A.1, human review criteria"},{"comment":"The headline BCR values, and therefore the defense frontier, inherit hand-set evaluator thresholds (clean/poisoned threshold 0.5, margin 0.1, ambiguous threshold 0.34) with no human-agreement study or threshold sensitivity analysis. The evaluator is deterministic, which is a strength, but the label boundaries are arbitrary and could change BCR levels and the L1/L2/L3 gaps. The authors should report inter-annotator agreement on a sample, and a sensitivity sweep over the thresholds (or at least show that the relative ordering of L1 vs. L2/L3 and the defense ranking is stable across a plausible range). This is important because the paper's central frontier claim is a comparative statement across difficulty levels, not just an absolute BCR.","section":"App. B.6, Eq. (10); §5.3"}],"minor_comments":[{"comment":"The text says 'defined in Appendix 5' but should refer to Appendix B.6 (or Eq. (8)–(10)).","section":"App. B.7, text before Eq. (13)"},{"comment":"The notation '0.532→0.266' for L1 is unclear: it appears to be a pre-removal vs. post-removal score, but the columns are not defined. Please clarify in the table caption or text.","section":"§6, Table 3"},{"comment":"In Table 3, the L1 row repeats Δ_s = 0.266 under both 'Score' and 'Interaction Signal' columns; this is confusing and should be marked as N/A or '—'.","section":"Table 3 and §4.4"},{"comment":"Several figure panels show garbled font substitutions instead of readable axis labels and legends (e.g., the 'uni0000…' sequences in the rendered text). The final version must regenerate figures with readable labels.","section":"Various figures (Fig. 2, Fig. 3, Fig. 4, Fig. 8)"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid empirical effort with a large, carefully constructed benchmark, but the main mechanistic narrative needs a substantive correction. The interaction-sign issue is not a minor typo: it undermines the paper's stated L2 explanation, although the benchmark data and the group-level effects may still support a weaker form of the defense-frontier claim if re-analyzed. I would be open to acceptance after the authors reconcile the MID interpretation, soften or re-scope the 'structural' conclusion given the selection circularity, and add evaluator robustness evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The MemPoison paper is a genuinely useful empirical resource: 1,227 hand-validated cases, a family-first construction pipeline, three substrates, ten model families, and a defense suite with disclosed calibration. The undefended BCR numbers and the defense frontier (consistency check: L1 4.77%, L2 22.54%, L3 27.80%) are the real content, and the paired clean/poisoned protocol with a deterministic evaluator is a good template. The dataset construction is unusually careful, with family caps and human gold review documented in detail.\n\nBut the mechanistic centerpiece, MID, has a sign error in its interaction term that undercuts the abstract's 'structural blind spot' claim for L2. A positive Omega_g is defined as the joint-removal effect being smaller than the sum of the single-removal effects. That is overlap or redundancy, not synergy. Table 3 reports L2 Delta_g = 0.478 and Omega_g = 0.176, which forces the two single-removal effects to sum to 0.654 — implying each L2 fragment has an average individual effect of 0.327, larger than the L1 single-record effect of 0.266. So the L2 fragments are not individually benign; they are individually influential and partially redundant. The claim that L2 harm emerges only through co-retrieval and composition is contradicted by the paper's own numbers. This is not a minor wording issue; it directly weakens the 'structural blind spot' argument.\n\nThere is also a circularity concern: L2 and L3 are defined as attacks that evade write-time plausibility checks, and the human review retains only surface-benign injections, so the finding that write-time defenses miss them is partly guaranteed by construction. The magnitudes remain empirical, but the qualitative frontier is less surprising than presented.\n\nReproducibility is promised but not shipped: no dataset URL, repository, or hash appears in the text. The token-overlap top-3 retriever is also a simplification; results may not transfer to production memory stacks.\n\nWho should read this? Researchers working on agent memory security will want the benchmark even if they discard the MID interpretation. It deserves a serious referee: the data collection is careful, and the flaws are fixable — fix or reinterpret the MID metric, add sensitivity analysis for the evaluator thresholds, and release the dataset. I would not cite the MID result as evidence for compositional harm until the sign issue is resolved.\n\nRecommendation: send to peer review with the expectation of heavy revision.","headline":"A serious benchmark with a load-bearing sign error: the MID 'interaction' metric is mislabeled, and the L2 compositional story collapses on the paper's own numbers.","tokens_in":36520,"tokens_out":4555,"would_cite":true,"duration_ms":49090,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Write-time memory filters stop direct attacks but miss composed and dormant poisonings in LLM agents.","keywords":["persistent memory poisoning","LLM agents","write-time defenses","compositional attacks","context-triggered dormant attacks","Mechanistic Influence Decomposition","defense frontier","behavioral corruption rate"],"falsifier":"Run the 1,227-case pack at top-1 retrieval instead of top-3 and check whether L2 Behavioral Corruption Rate collapses toward L1 levels; the paper's own top-k sweep predicts L2 is retrieval-completeness limited, so a production retriever that rarely co-retrieves all fragments would erase the L2 half of the frontier. Alternatively, in the MID analysis remove random non-designated records instead of the designated poison and confirm the score drop is near zero; if random removal matches designated removal, the causal attribution would be vacuous.","tokens_in":1471,"feed_emoji":"🧠","tokens_out":1589,"duration_ms":53363,"temperature":0.7,"pith_summary":"The paper claims that the security of an LLM agent's persistent memory cannot be decided at the moment a memory is written. It introduces a three-level threat ladder: a single harmful record (L1), a harmful conclusion split across several innocent-looking records that becomes dangerous only when retrieved together (L2), and a dormant instruction that activates only under a later natural trigger (L3). Across 1,227 hand-validated cases and ten model families, the paper measures that write-time defenses suppress L1 but leave L2 and L3 largely intact: the strongest pure write-time consistency check cut L1 corruption to about 5% while L2 and L3 stayed near 23% and 28%. A combined pipeline reduced overall corruption to about 11% but still left residual L2/L3 failures. If correct, agent-memory defenses must become retrieval- and context-aware rather than static filters at write time.","feed_headline":"Memory guards stop direct attacks but miss split and dormant ones","feed_subtitle":"On 1,227 curated cases, write-time filters cut direct corruption to 5% while composed and triggered attacks still hit 20-30%.","key_machinery":"Three named pieces carry the argument: the L1-L2-L3 taxonomy, which defines attack difficulty by the structure of the stored memory rather than by textual obfuscation; the benchmark protocol with paired clean/poisoned trigger tasks and Behavioral Corruption Rate (BCR) as the outcome metric; and MID, which removes the designated poison record or fragment set and measures the drop in poisoned-match score. MID's three statistics - single-record influence delta-s, pairwise interaction Omega-g, and ActivationShift - convert the abstract claim that harm can be deferred into measurable causal signatures. The pipeline audit then decomposes each poisoned case into write-blocked, admitted-but-not-retr","core_discovery":"On the paper's own terms, MemPoison establishes a defense frontier: write-time admission filters that check consistency, plausibility, or novelty can block directly harmful memory records (L1), but cannot reliably block attacks whose harm is deferred. L2 attacks partition the payload across individually benign records, so each write passes local checks; L3 records are inert until a natural trigger arrives. Mechanistic Influence Decomposition (MID), a counterfactual removal diagnostic over the retrieved memory set, shows why: L1 corruption is carried by a single record (poisoned-to-clean flip 95.7%), L2 by a non-additive joint effect between fragments (interaction signal 0.176), and L3 by a c","pith_inferences":["Editorial inference: the same structural argument likely applies to any pointwise sanitization of memory - content harmful only under composition or context cannot be detected by inspecting records in isolation - so the paper's frontier probably forms a ceiling for all write-time classifiers, not just the baselines tested.","Editorial inference: the MID signatures suggest a testable detection strategy: monitor pairwise interaction signals or activation shifts at retrieval time rather than at write time; if cheap to compute, such retrieval-aware scoring could close part of the L2/L3 gap.","Editorial inference: the benchmark's selection criteria (benign on the surface, non-leaky triggers) may understate real-world miss rates if attackers are sloppier, but may overstate them if production retrievers rarely co-retrieve all fragments; the paper's own top-k sweep is the lever to test this.","Editorial inference: extending MID to triple and higher-order fragment interactions would test whether L2's compositional signal saturates at pairs or grows with fragment count, which would inform how many fragments a future defense needs to track."],"forward_implications":["If the frontier holds, agent memory systems relying on write-time consistency checks will continue to be poisoned by attacks split across records or activated by later context; measured residual BCR for L2/L3 is in the 20-30% range even under the best pure write-time filter.","Defense evaluation for agent memory should report corruption by difficulty tier rather than only overall attack success, because averaging hides the L2/L3 residual.","Memory substrate matters: flat chunks preserve intact poisoned records and are most vulnerable; fact stores dilute injected content; hierarchical summaries partially attenuate but do not eliminate corruption.","Externally mediated write channels (tool outputs, inter-agent messages) consistently produce higher corruption than direct user input, so provenance-aware retrieval weighting is a natural complement to write-time filtering.","Retrieval breadth gates L2: the top-k sensitivity analysis shows L2 corruption rises sharply with larger retrieved context, meaning deployment choices about retriever coverage directly change the effective threat."],"fun_headline_variants":["Memory filters fail on split and dormant attack patterns","Write-time guards blind to composed memory attacks","Direct memory attacks blocked, deferred ones slip through","Memory defense frontier: filters stop only direct poison"],"cache_read_input_tokens":37504,"weakest_assumption_plain":"The benchmark retains only attacks that human reviewers judged benign on the surface with non-leaky triggers, so the qualitative finding that write-time defenses miss L2/L3 is partly baked into the selection; what is empirically measured is the size of the miss, and that size depends on the curated attack distribution.","fun_headline_variants_meta":{"raw":{"variants":["Memory filters fail on split and dormant attack patterns","Write-time guards blind to composed memory attacks","Direct memory attacks blocked, deferred ones slip through","Memory defense frontier: filters stop only direct poison"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000174,"raw_usage":{"total_tokens":1113,"prompt_tokens":732,"completion_tokens":381,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":476,"completion_tokens_details":{"reasoning_tokens":335}},"tokens_in":476,"tokens_out":381,"duration_ms":4680,"temperature":1.0,"reasoning_tokens":335,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T01:29:09.833773+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the 1,227-case pack at top-1 retrieval instead of top-3 and check whether L2 Behavioral Corruption Rate collapses toward L1 levels; the paper's own top-k sweep predicts L2 is retrieval-completeness limited, so a production retriever that rarely co-retrieves all fragments would erase the L2 half of the frontier. Alternatively, in the MID analysis remove random non-designated records instead of the designated poison and confirm the score drop is near zero; if random removal matches designated removal, the causal attribution would be vacuous.","supporting_citations":[],"review_version":1}