{"id":"6362f218-a824-4e6f-aae6-cc396fcbcb96","arxiv_id":"2608.07952","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Tool-augmented LLM agents retain injected preferences and instructions across many turns, while factual falsehoods are often corrected, and a name-binding registry is the main entry point.","lead":"Researchers show that tool-using AI agents can silently keep contaminated instructions and preferences alive across sessions and between agents. This matters because standard logging and debugging miss most of the contamination, leaving a hidden security risk.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 20–100% susceptibility headline relies on a keyword detector that the paper itself documents as having an 83% false-positive rate on factual-correction responses; judge-based per-model rates are needed before 'every tested model is susceptible' is established.","rationale":"The reader's verdict is CONDITIONAL, and this stress-test reinforces that condition rather than overturning it. The most load-bearing point is not the acknowledged §4.3 construction issue in the name-binding ablation, which the paper already qualifies as 'partly by construction'; it is that the central susceptibility claim is measured with a detector that the same paper discredits for a category covering four of the ten scenarios. Since the scaling sweep outputs are released, the fix is concrete and inexpensive: apply the judge protocol to the 1,100 runs. If all models remain susceptible under the reliable detector, the conditional verdict stands as is; if some model's interval includes zero, the abstract and security recommendations need substantial qualification. This is why I retain the reader's CONDITIONAL rather than moving to ACCEPT, and also why I do not move to REJECT: the concern is resolvable from the released artifacts, and the persistence and defense results use the more reliable judge-based detection. Agreement is partial because the reader's formal weakest-assumption focuses on the injection protocol, whereas I see the susceptibility detector as the first-order threat to the strongest empirical claim, even though the reader does flag the detector problem in the rationale.","tokens_in":32477,"tokens_out":5845,"duration_ms":68199,"concrete_test":"Re-run or re-label the 1,100 scaling-sweep outputs with the released judge (Gemini-2.0-Flash-Lite) or a human-labeled subset, reporting per-model contamination with Wilson intervals and a breakdown by contamination category. If every model's judge-based rate has a lower confidence bound above zero and the 20–100% spread survives, the concern is settled. If any model's interval includes zero (likely among the 20–30% rows) or factual categories contribute the bulk of positive calls, the headline susceptibility claim must be revised or explicitly qualified.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The susceptibility range is the paper's first headline and the basis for 'every tested model is susceptible.' Per Appendix C.1 and Table 20, the scaling sweep labels contamination with keyword matching plus cosine (cos>0.7), not the LLM-as-judge detector declared canonical for the ablation and defense experiments. But §3 and §C.6 document an 83% false-positive rate for this keyword channel on exactly the factual-correction responses that make up 4 of the 10 contamination scenarios in the panel, as in 'NOT Lyon, but Paris' containing 'lyon.' The released n=180 inter-detector comparison and the threshold sweep in §C.6 do not recover per-model, per-category rates; shifting the cosine threshold by ±8pp cannot repair a detector whose errors are categorical. A model that consistently corrects factual injections could be counted as contaminated in the panel even though the judge would label it clean, so the claimed 20–100% range, and the derived claim that no model is immune, is not established by the released evidence. This concern is distinct from the §4.3 name-binding circularity, which the paper already qualifies; the susceptibility claim is presented without an equivalent caveat.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper formalizes Persistent Semantic Entities (PSEs) as triples (N, T, P) capturing name binding, event triggering, and cross-boundary propagation in tool-augmented LLM agents, and reports a 14,293-run corpus across 24 models from 11 families. The headline claims are: (1) every tested model is susceptible (20–100% on the 20-model panel) with name binding as the necessary mechanism, since removing it gives 0% contamination; (2) persistence is type-dependent—preference and instruction contamination persist undecayed through turn 10, persona decays partially, and factual contamination is model-dependent; (3) context-isolated self-verification (CIV) reduces contamination by 20–79% (median 36.5%) without oracle references, keyword detection carries an 83% false-positive rate on factual corrections, and contamination compounds 1.9x along a four-stage pipeline (40% to 75%). The authors also report a 50pp observability improvement from enhanced logging. A notable feature is the paper's selective transparency: several earlier statistics (H1 ANOVA, the factorial interaction, operator utility and half-life columns, a secondary-judge check) are explicitly withdrawn or labeled descriptive, and the released artifacts include a SHA-256 manifest and a verification script that recomputes the run ledger.","tokens_in":32676,"tokens_out":17472,"duration_ms":177668,"significance":"If the susceptibility claim is re-established with judge-based per-model rates, this paper makes a useful contribution to the agent-security literature. It offers a crisp vocabulary for a real class of failure (state that persists through names and events rather than data flow), identifies the tool registry as an under-monitored attack surface, and produces falsifiable, type-dependent persistence patterns and a consistent defense ordering (external validation, CIV, no defense, on every model tested). The practical strengths are the shipped artifacts: a pinned release with a SHA-256 manifest, a verification script that recomputes all run counts, and explicit withdrawal of non-reproducible statistics rather than silent omission. The CIV defense is oracle-free and cheap. The formal sections are labeled interpretive by the authors themselves, which is an appropriate scope choice for an empirical paper. As reported, however, the first headline—the 20–100% susceptibility range and the no-immunity claim—rests on the non-canonical keyword-plus-cosine detector, and the name-binding claim is partly an artifact of the injection protocol; both issues are fixable within the paper's scope.","major_comments":[{"comment":"The 20-model susceptibility panel that supports the headline 'every tested model is susceptible (20–100%)' is scored with keyword matching plus cosine similarity (cos>0.7), the very channel that §3.3 and §C.6 document as having an 83% false-positive rate on factual-correction responses, which constitute 4 of the 10 scenarios (overall Cohen's kappa=0.22 against the LLM judge on n=180). Table 20 confirms that the scaling sweep uses keyword+cosine as its canonical labels, and the released n=180 inter-detector comparison is drawn from the temporal panel rather than from the scaling sweep, so it cannot characterize per-model detector error on the 20-model panel. The internal evidence illustrates the risk: Table 6 (judge-based) reports Llama-3.1-8B at 0% factual contamination because it self-corrects, while Table 12 lists the same model at 100% using a detector that flags corrective responses such as 'NOT Lyon, but Paris' as contaminated because the injected token appears. The 8-model judge-based defense panel (Table 7) does show 35–75% baseline rates on every model, so the qualitative no-immunity pattern may survive, but the 20–100% range and the 20-model claim are not established by the released evidence, and a ±8pp cosine-threshold shift cannot repair a categorical detector error. Because the 1,100-run scaling corpus is part of the released artifact, re-scoring it (or a per-model stratified sample) with the judge and reporting per-model, per-category rates is a feasible fix within the paper's scope.","section":"§3.3, §C.1, Table 12/Fig. 3"},{"comment":"The abstract and conclusion state 'name binding as the necessary and dominant mechanism: without it, contamination is 0%,' but §4.3 itself qualifies the ablation as 'partly by construction, since removing N removes the injection vector.' In this protocol, disabling N removes the only pathway by which contamination is introduced, so the 0% result follows from the injection design rather than from an independent measurement of the models' resistance. The experiment therefore supports 'name binding is the entry vector under this protocol,' not 'name binding is necessary and dominant in general.' The qualitative claim may hold, but the version in the abstract and §7 should carry the §4.3 qualification, and ideally the design should decouple the injection vector from N (for example, delivering contamination through event triggers or serialized state with N toggled independently) so the necessity claim can be tested rather than assumed.","section":"§4.3, Abstract, §7"}],"minor_comments":[{"comment":"The H2 observability gap (25% to 75%) is, as §4.2 states, fixed by construction—each logging regime yields a fixed visibility set—yet the §4.7 summary and the contribution list restate it as a finding ('enhanced logging improves observability by 50pp'); please present it as a definitional consequence rather than an empirical result.","section":"§4.2, §4.7"},{"comment":"The phrase 'every model probed' in the abstract's persistence claim covers Llama-3.1-8B at n=10 (Table 6) plus three models at n=5 with unanimous cells (Table 17), so the claim's strength exceeds what the sample sizes support; please state the per-model sample sizes and the wide Wilson intervals directly alongside the claim.","section":"§4.6, Abstract"},{"comment":"The Table 1 header 'Observ- bdry' and the PSE row entry 'Impl. ✓ 75% miss' appear partially corrupted; please repair the column so the observability contrast is legible.","section":"Table 1"},{"comment":"The indicator function in the contamination-rate estimator is typeset as '⊮', which is a font artifact that should be replaced by a standard 1[·] or bold-one notation.","section":"§3.2"},{"comment":"The 'RAG pollution' row of the run ledger (80 runs, 65% baseline) is not backed by any methodology or results subsection in the appendix; please add its configuration or remove it from the ledger.","section":"§2, Table 23"},{"comment":"The M1 re-anchoring formula 'C(h) = αh0 + (1−α)h' loses its subscripts and reads as garbled; please restore the time-indexed notation.","section":"Appendix D.2, Table 22"},{"comment":"The 20–79% reduction range is computed from point estimates in n=20 cells with wide Wilson intervals (for example, Gemini-Flash-Lite's 79% comes from 3/20 versus 14/20); the main text should state this interval-based caveat where the range is first quoted, as §C.2.5 already does for the ordering claim.","section":"§4.5, Table 5"},{"comment":"The primary judge (Gemini-2.0-Flash-Lite) is also the evaluated model in the H3 ablation arm, and the secondary-judge agreement check (kappa=0.88) is reported only descriptively without retained run-level output; please either retain and release that output or run a second judge on the Gemini-FL ablation cells.","section":"§3.3, Table 3"}],"recommendation":"major_revision","confidential_remarks":"This is a single-author paper whose review history appears to have substantially reshaped the claims: a nontrivial share of the originally planned analyses (the H1 significance test, the factorial interaction, the operator utility and half-life columns, the secondary-judge check, the SRV overhead figures) are now withdrawn, labeled descriptive, or labeled 'engineering guesses.' I regard the candor as a genuine strength, but the editor should be aware that the released corpus supports a trimmed claim set, and the one surviving headline that rests on unre-scored data—the 20-model susceptibility range—uses the non-canonical detector. The recommended major revision is therefore not about adding new experiments in general, but about re-scoring the existing released scaling corpus with the judge and aligning the abstract with the paper's own qualifications. The paper fits the venue's scope as an empirical security-phenomenology contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know: this paper ships real artifacts and admits its own weak spots, which puts it ahead of most of the agent-security literature. The persistent semantic entity (PSE) framing is mostly relabeling, but the empirical work behind it is largely genuine. The stand-out result is the type-dependent persistence pattern: preference and instruction contamination hold at ceiling through ten turns on Llama-3.1-8B, persona decays, factual contamination self-corrects on some models but not others. That asymmetry is new, practical, and supported by judge-based labels with cross-provider replication. The defense comparison is also useful: context-isolated self-verification gives a 20-79% reduction without oracle references, and the paper correctly notes the best case is not the typical case. The reproducibility package is exemplary - pinned GitHub release, SHA manifest, 14,293 released runs, and withdrawn statistics explicitly flagged. That is real evidence, and it should be credited. The soft spot is the headline susceptibility claim. The 20-model, 20-100% panel uses keyword-plus-cosine detection, and Appendix C.6 documents that this detector has an 83% false-positive rate on factual-correction responses. Factual scenarios are 4 of the 10 in the panel, so a model that consistently corrects factual injections could be counted as contaminated in the scaling sweep even though the judge would call it clean. The threshold sweep (+/-8pp) does not repair a categorical error of this kind. So 'every tested model is susceptible' is not established by the released evidence. This is a real flaw, but it is fixable: per-category, per-model judge-based labels would settle it. The name-binding-necessity claim is also partly by construction - the paper admits this in section 4.3 - but the conclusion still reads as a stronger finding than the injection protocol supports. Smaller issues: key cells are n=5-20, so many rates are weakly determined, and the secondary-judge agreement check was not retained, though the paper flags that honestly. Who is this for? Researchers working on LLM agent security and practitioners deploying tool-augmented agents. It deserves a serious referee, not a desk reject. I would accept peer review but send it back for per-category susceptibility labels, an independent judge or clearer non-independence caveats, and a toned-down statement about name binding. The persistence finding is likely to survive; the headline range may not.","headline":"An unusually transparent, reproducible agent-security paper whose main empirical highlight (type-dependent persistence) is solid, but whose headline 'every model is susceptible' rests on a detector the paper itself shows is unreliable for factual scenarios.","tokens_in":755,"tokens_out":1137,"would_cite":true,"duration_ms":38143,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Tool-augmented LLM agents hold hidden state that survives sessions and spreads between agents; name binding is its necessary mechanism, injected preferences and instructions persist undecayed, and external validation mitigates it best.","keywords":["persistent semantic entities","tool-augmented LLM agents","name binding","event triggering","cross-boundary propagation","contamination persistence","tool registry security"],"falsifier":"Run the §4.3 mechanism ablation with a name-free injection channel: seed contamination through a direct event subscription (a handler attached to a lifecycle event with no registry name) or through serialized memory state, keep the rest of the protocol identical, and check whether the no-name-binding arm stays at 0%; any nonzero rate in that arm would falsify the claim that name binding is necessary for PSE contamination.","tokens_in":32222,"feed_emoji":"⚠️","tokens_out":21491,"duration_ms":205533,"temperature":0.7,"pith_summary":"Tool-augmented LLM agents can carry implicit, semantic-level state that ordinary debugging cannot see: the paper formalizes this as Persistent Semantic Entities, constructs defined by name binding, event triggering, and cross-boundary propagation. Across 24 models from 11 families, every tested model is susceptible to contamination seeded through these mechanisms (20-100% on the 20-model susceptibility panel), and the mechanism ablation identifies name binding as the necessary and dominant lever: without it, contamination is 0%. Persistence tracks the type of injected content rather than model scale or deployment: preference and instruction contamination stay at 100% through ten turns on every model probed, persona-style injection decays partially, and factual contamination self-corrects on some models but not others. The practical stakes are that preference and instruction contamination persist without self-correction, evade keyword-based detection, and compound $1.9\\times$ along a four-stage agent pipeline, making tool registries a poorly monitored attack surface in deployed agent systems.","feed_headline":"Hidden state survives in every LLM agent tested","feed_subtitle":"Tool-name bindings carry injected preferences across 10 turns and agent boundaries, evading standard logs.","key_machinery":"The load-bearing object is the Persistent Semantic Entity, defined as a triple $(N, T, P)$: a name binding function $N: S \\to H$ mapping string identifiers to execution handlers, an event triggering function $T: V \\to 2^S$ mapping runtime events to the sets of bindings they activate, and a propagation function $P: S \\times C \\to 2^{S \\times C}$ mapping (binding, context) pairs to downstream activations. A PSE exhibits persistence when an activation in one context induces an activation in a distinct context without any explicit state transfer, and a resilient system must enforce name-binding integrity, event isolation, and bounded propagation chains. The name binding component carries the argument: in the mechanism ablation every configuration lacking it shows 0% contamination while name binding alone produces 45-95% (Cohen's $d = 3.26$ on one ablation model and 1.37 on the other), so the tool registry is identified as the primary attack surface. Contamination is operationalized as a measurable deviation from expected behavior caused by PSE state, and detection relies on an LLM-as-judge predicate because string matching cannot distinguish adoption of injected content from explicit correction of it.","core_discovery":"The paper's central claim is that a tool registry is not a passive lookup table but a durable semantic layer: an entity bound to a name outlives the session that created it, can be reactivated by runtime events without explicit invocation, and can propagate to other agents and future sessions through shared registries or serialized state. The author argues this by formalizing a Persistent Semantic Entity as a triple $(N, T, P)$ and then showing experimentally that contamination introduced through these mechanisms appears on every one of 24 tested models with no resolvable scale trend (20-100% across the 20-model panel; $R^2 = 0.25$, $p = 0.256$ on the matched scale sweep), that name binding alone reproduces the effect (45-95% in the ablation) while every configuration without it sits at 0%, and that persistence is governed by contamination type: preference and instruction injections hold at ceiling through turn 10 on every model probed, persona injection decays, and factual injection self-corrects on some models but not others. The paper further claims that defenses fail asymmetrically: self-reflection inside the contaminated context is unreliable and on one model increases contamination, context-isolated self-verification cuts contamination 20-79% without oracle references, and external validation at agent boundaries blocks the four-stage pipeline cascade entirely; it also shows that keyword-based detection systematically mislabels corrective responses as contaminated.","pith_inferences":["A test the paper did not run: because §4.3 admits that removing name binding also removes the injection vector, injecting contamination through a name-free channel (a direct event subscription, or serialized memory carrying no registry name) would decide whether name-binding necessity is a property of agent systems or only of this injection design.","The type-dependence asymmetry suggests a self-correction frontier worth mapping: contamination should persist when nothing in the model's parametric knowledge contradicts it and decay when it does; probing weakly held or fictional facts could trace that frontier, and the paper's own 20-turn sweep shows the frontier is model-specific.","If the model-collapse analogy runs deeper than an analogy, two checkable predictions follow from the four-stage cascade: the $1.9\\times$ amplification should grow with pipeline length, and even a weak per-boundary filter should flatten it."],"forward_implications":["Tool name registries become a security boundary: since contamination requires a name binding, registry integrity (authorized-only binding, explicit unbinding, whitelisting) is the primary architectural mitigation.","Observability must cover registry mutations, event subscriptions, and propagation edges, not just tool calls and I/O: enhanced logging of this state raises visibility of PSE-relevant activity from 25% to 75%.","Defensive effort should concentrate on preference and instruction contamination, the types that persist undecayed and face no parametric resistance, rather than on factual contamination, which at least some models self-correct.","Multi-agent deployments need validation at every agent boundary: without it, contamination compounds $1.9\\times$ along a four-stage pipeline (40% $\\to$ 75%), and external validation blocks the cascade outright.","Contamination detection should be semantically aware: judge-based evaluation distinguishes adoption from correction, whereas keyword matching systematically overestimates contamination on factual scenarios."],"supporting_citations":[{"why":"Grounds the name-binding mechanism: the agent framework keeps name-based tool registrations and a memory buffer that carries conversation history across chain invocations.","marker":"Chase, 2023"},{"why":"Anchors the §5 case reconstruction: its plugin system lets third-party code run in the agent process and register command handlers that serialized state can re-instate across restarts.","marker":"Significant Gravitas, 2023a"},{"why":"Grounds the cross-boundary propagation mechanism: the multi-agent framework coordinates agents through shared memory spaces that pass state between collaborating agents.","marker":"Moura, 2024"},{"why":"Defines indirect prompt injection, the single-context attack that PSEs are contrasted against: PSE adds the orthogonal axis of persistence and propagation across contexts.","marker":"Greshake et al., 2023"},{"why":"Establishes the prompt-injection vulnerability baseline whose in-context scope the paper extends with name binding, event triggering, and propagation.","marker":"Schulhoff et al., 2023"},{"why":"Supplies the model-collapse analogy used to frame compounding contamination along the multi-agent pipeline: contaminated output becoming the next stage's input.","marker":"Shumailov et al., 2024"},{"why":"Provides the Wilson score interval method used for all confidence intervals in the reported contamination rates.","marker":"Wilson, 1927"},{"why":"Characterizes persistence of memory poisoning in memory-based agents, the data-layer phenomenon that PSEs are defined as orthogonal to.","marker":"Devarangadi Sunil et al., 2026"}],"fun_headline_variants":["LLM agents hide persistent state in tool names that survives sessions","Tool-name injection persists in every LLM agent, evading standard logs","Preference contamination outlasts 10 turns in all tested LLM agents","Self-checking cannot fully stop persistent tool-name attacks in LLM agents","Tool registry becomes a hidden memory that propagates across agents"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the paper's injection protocol faithfully represents how contamination enters real agent systems: in the ablation, removing the name-bound injection vector also removes the contamination (as the paper itself notes in §4.3), so if real contamination can arrive through other channels, such as a direct event subscription or saved memory state that carries no registered tool name, the claim that name binding is necessary would not generalize.","fun_headline_variants_meta":{"raw":{"variants":["LLM agents hide persistent state in tool names that survives sessions","Tool-name injection persists in every LLM agent, evading standard logs","Preference contamination outlasts 10 turns in all tested LLM agents","Self-checking cannot fully stop persistent tool-name attacks in LLM agents","Tool registry becomes a hidden memory that propagates across agents"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000896,"raw_usage":{"total_tokens":3951,"prompt_tokens":1127,"completion_tokens":2824,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":743,"completion_tokens_details":{"reasoning_tokens":2732}},"tokens_in":743,"tokens_out":2824,"duration_ms":21014,"temperature":1.0,"reasoning_tokens":2732,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T00:37:32.635703+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the §4.3 mechanism ablation with a name-free injection channel: seed contamination through a direct event subscription (a handler attached to a lifecycle event with no registry name) or through serialized memory state, keep the rest of the protocol identical, and check whether the no-name-binding arm stays at 0%; any nonzero rate in that arm would falsify the claim that name binding is necessary for PSE contamination.","supporting_citations":[{"cited_title":"LangChain : Building applications with LLMs through composability","cited_arxiv_id":null,"evidence_quote":"Grounds the name-binding mechanism: the agent framework keeps name-based tool registrations and a memory buffer that carries conversation history across chain invocations."},{"cited_title":"AI models collapse when trained on recursively generated data","cited_arxiv_id":null,"evidence_quote":"Supplies the model-collapse analogy used to frame compounding contamination along the multi-agent pipeline: contaminated output becoming the next stage's input."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Wilson score interval method used for all confidence intervals in the reported contamination rates."}],"review_version":1}