{"id":"f0594f0c-e7c6-4e7d-b033-ee2d5ca56a68","arxiv_id":"2608.11295","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Defensive poisoning followed by unlearning erases most unknown backdoors in LLM tool-calling agents, while trigger recognition can outlive malicious execution.","lead":"This paper tests whether installing a known 'defensive' backdoor and then unlearning it can remove an unknown malicious backdoor hidden in an LLM tool-calling agent. Across 115 experiments on a prompt-injection benchmark, the strategy erases most original backdoors, and removing a single co-installed backdoor usually clears the rest.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"All decontamination results use a single fine-tuning schedule (lr=3e-5, 2 epochs), yet the paper's own Appendix D sweeps show that lr is highly consequential and specifically unstable at 3e-5; the 56% erasure and 'almost all survivors' rates may be artifacts of that schedule rather than robust…","rationale":"The reader's weakest assumption was external validity: all experiments use one student model (Qwen3-8B) and one harness (AgentDyn), and the 'never persist' universal is inferred from finite pairings in that single setting. That is a legitimate and important concern. The concern raised here is different and more internal: the paper's quantitative claims are all produced under a single fine-tuning schedule, and the paper itself provides direct evidence in Appendix D that lr=3e-5 is not a uniformly safe operating point. Because the decontamination mechanism under study is essentially catastrophic forgetting of a trigger-action mapping, the erasure rates and the 'never persist' regularity are exactly the kind of quantities that depend on schedule aggressiveness. A milder schedule might preserve more original backdoors, which would undercut the central claim that decontamination is a broadly effective baseline defense even within the reported model/harness. This does not change the verdict from CONDITIONAL; it adds a concrete internal-validity condition that the reader did not explicitly identify. The proposed lr/epoch sweep is feasible on a subset of pairs and would determine whether the headline rates are robust or schedule-bound. No issue is raised with the authors' conduct or the internal consistency of the framework; the concern is purely about the evidential weight of single-configuration empirical results.","tokens_in":17435,"tokens_out":13538,"duration_ms":127050,"concrete_test":"Re-run a stratified subset of the 115-pair matrix, specifically all same-trigger-type pairs plus 20 different-type pairs spanning the location/language/register families, through Phase 2 and Phase 3 at lr=1e-5 and lr=1e-4 in addition to the published 3e-5 arm, holding data, seeds, and epochs fixed. Also run one arm with 1 epoch and one with 4 epochs. If the erased fraction and the persist count remain within sampling error of 56% and 15/100, and no same-type persist appears at any alternate schedule, the schedule-dependence concern is answered. If erasure drops materially or any same-type persist appears, the central claim must be re-scoped as schedule-dependent.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Every Phase 2 and Phase 3 experiment runs at one configuration: full-weight or LoRA fine-tuning at lr=3e-5, 2 epochs, cosine schedule, with no lr/epoch sweep for the decontamination steps. The paper's own Appendix D (Figures 8 and 9) shows installation outcomes are sharply lr-dependent, and at exactly 3e-5 two full-weight combos (fr__dl12, ip2902__dl7) collapse benign utility non-monotonically (77%→4%→38% and 62%→0%→62% across 1e-5/3e-5/1e-4), with the authors noting single-seed instability cannot be ruled out. The headline quantities—~56% erasure from defensive poisoning and 'almost all survivors' erased after unlearning—are catastrophic-forgetting phenomena, which are known to be sensitive to learning rate and training length. If 3e-5 happens to be a destructively aggressive schedule, a milder schedule could leave materially more original backdoors persistent, and the same-type 'never persist' regularity could disappear. The paper explicitly declines to search the schedule axis for the oracle decontamination in §4, so the central rates are single-point estimates at one possibly unfavorable operating point. This is an internal-validity concern, not merely a generalization concern: the defense's effectiveness could be an artifact of the chosen schedule.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies the dynamics of a two-step backdoor defense for LLM tool-calling agents: after an attacker has implanted an unknown trigger–action backdoor A, the defender installs a known benign backdoor B (defensive poisoning) and then unlearns B (decontamination), and the authors measure whether A is erased, persists, partially survives, or is rerouted to B's payload. Using Qwen3-8B on the AgentDyn harness, they run 115 sequential install experiments with decoupled trigger, response, teacher, and fine-tuning axes, reporting that defensive poisoning alone erases roughly 56% of original backdoors, that subsequent unlearning erases almost all survivors, that same-trigger-type A/B pairs never persist, that co-installing up to four backdoors lowers erasure to about 36% while decontaminating one known co-resident clears 52/60 others, and that J-Lens reveals residual trigger awareness after decontamination.","tokens_in":17741,"tokens_out":8066,"duration_ms":75455,"significance":"The claimed empirical regularities are valuable if they reproduce: decontamination would be a cheap, no-knowledge baseline defense, and the recognition/execution dissociation is a useful behavioral target. The paper's strengths are its decoupled experimental design, direct trace-level measurements rather than fitted predictions, explicit threat model, disclosed outcome thresholds, reproducible seeds, and unusually candid appendix material, including learning-rate sweeps that expose fragility of the chosen operating point. The central rates are nonetheless single-point estimates on one model and one harness, and the most general statement ('never persist') is stronger than the finite sample supports.","major_comments":[{"comment":"All Phase-2 and Phase-3 fine-tuning is performed at a single configuration (full-weight or LoRA, lr=3e-5, 2 epochs; §F), and §4 and Appendix I explicitly state that the decontamination schedule is fixed and not searched. Appendix D then shows that installation outcomes are sharply learning-rate dependent, and specifically at lr=3e-5 two full-weight combos (fr__dl12, ip2902__dl7) collapse benign utility non-monotonically (77%→4%→38% and 62%→0%→62% across 1e-5/3e-5/1e-4), with the authors conceding that single-seed instability cannot be ruled out. Because decontamination erasure is a catastrophic-forgetting phenomenon, the headline rates (~56% erasure, 'almost all survivors' erased, 87% collateral coverage) are point estimates at one potentially destructive operating point, so the claim that decontamination is a broadly effective baseline is not yet internally robust. I request a learning-rate/epoch sensitivity analysis for at least a representative subset of Phase 2/3 runs, or a re-framing of the central claims as conditional on the training schedule.","section":"§4, §F, Appendix D (Figures 8–9), Appendix I"},{"comment":"The abstract and conclusion state that 'across 115 experiments, defensive poisoning alone erases ~56% of original backdoors,' but Appendix H reports 56 erased among the 100 pairs in which B installed, with 15 additional b_failed pairs in the 115; 56/115 is approximately 49%, not 56%. The denominator should be stated consistently, and it should be explicit whether b_failed pairs are excluded from the headline rate; as written, the same dataset yields two different percentages depending on which section is read.","section":"Abstract, §3, Appendix H (Figure 11)"},{"comment":"The abstract's statement that malicious backdoors 'never persist' when using different triggers of the same general type is a universal claim, but the evidence is a finite set of pairings on one model and one harness, and §7 explicitly says the authors' claims about predictors are tendencies, not laws. The regularity could also be an artifact of the single training schedule in the first major comment. Please either rephrase the claim as 'no persistence was observed in this sample,' or report the number of same-type pairs and a confidence interval, and test the regularity under at least one alternative schedule.","section":"§3 (Figure 3), §7 Limitations"}],"minor_comments":[{"comment":"The caption reads 'Joint poisoning makes backdoors increases resilience to defensive poisoning'; this should be 'increases' or 'makes backdoors more resilient'.","section":"Figure 4 caption"},{"comment":"The heading 'Resilience to Defensive Poisoining' contains a typo, and 'interestingrerouting' in §3 lacks a space.","section":"§4 heading, §3 text"},{"comment":"The 0% benign-utility bar for the dailylife benign-only fine-tune in Figure 6 is explained as a training artifact in Appendix D, but a reader encountering the figure has no such pointer; a sentence near Figure 6 would prevent misinterpretation.","section":"Figure 6 / Appendix D"},{"comment":"The outcome-classification thresholds (persist A→A>15, reroute A→B>15, erased A→A≤5, and the environment-metric equivalents) are disclosed but not stress-tested; a brief sensitivity analysis over threshold values would show how stable the 56/44 split and the co-install rates are.","section":"Appendix A"},{"comment":"The paper alternates between 'J-lens', 'J-Lens', and 'Jacobian lens'; the notation should be standardized throughout.","section":"§5 / Abstract"}],"recommendation":"major_revision","confidential_remarks":"The paper is an honest, well-executed measurement study, but the single-schedule issue is load-bearing because the headline defense rate could be schedule-induced. With a learning-rate/epoch sensitivity analysis, corrected denominators, and softer phrasing of the 'never persist' claim, the paper would be publishable; the authors' candid appendix disclosures strengthen trust in the underlying measurements."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read this if you work on LLM-agent security. It is the first systematic study of what happens when you install a known \"defensive\" backdoor over an unknown one and then unlearn it, on a realistic tool-calling agent benchmark. The framework is well thought out: trigger, response, teacher, and fine-tuning method are decoupled, and the trace-score metric cleanly separates rerouting from lingering trigger recognition. That pays off. The finding that same-type trigger pairings never persist — instead rerouting A's trigger to B's payload — is the kind of structural result that will get cited. The recognition/execution dissociation is also convincingly shown: under the \"refuse\" objective, ASR drops to zero while recognition stays at 100%. And the joint-poisoning results (36% erasure, 87% collateral coverage) are new and useful.\n\nThe soft spot is real, and it is internal validity, not just generalization. Every Phase 2 and Phase 3 experiment runs at lr=3e-5, 2 epochs. The paper's own Appendix D shows that lr=3e-5 is not a safe default: two full-weight combos collapse benign utility non-monotonically (77%→4%→38%, 62%→0%→62% across 1e-5/3e-5/1e-4), and for 4 of 7 combos the sweep favors 1e-5. The authors decline to search the schedule axis for decontamination, calling it out in the text. That matters because the headline numbers — ~56% erasure and \"almost all survivors\" erased — are catastrophic-forgetting phenomena. If 3e-5 is an aggressive schedule, a milder one could leave more original backdoors alive, and the \"never persist\" regularity could loosen or vanish. The paper itself walking it back to a \"tendency\" in Section 7 is the right instinct, but the abstract still states it as a universal. So my verdict is the same as yours: the observations are probably real, the central framework is sound, but the specific rates should be treated as a single operating point until swept.\n\nMinor points: the outcome classifications rely on hand-set thresholds (5%, 15%, 50%) that are plausible but arbitrary, and no code or data is shipped. The single model/harness limitation is stated honestly and is the right top-priority for follow-up.\n\nThis deserves a serious referee. I would send it out with a request for a schedule robustness check on a subset of pairs and for code release. For the reading group, yes — there is enough here to spark a good discussion about what counts as \"erased\" and how much we should trust fine-tuning as a generic defense.","headline":"A genuinely useful first map of backdoor decontamination dynamics in agents, but every headline number is a single-learning-rate point, and the paper's own appendix shows that rate is a risky choice.","tokens_in":18290,"tokens_out":1779,"would_cite":true,"duration_ms":18030,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Installing a known backdoor and then unlearning it erases most unknown backdoors in LLM agents, while trigger recognition and malicious execution come apart.","keywords":["backdoor attacks","LLM agents","defensive poisoning","machine unlearning","trigger recognition","tool calling","prompt injection","model safety"],"falsifier":"Re-run the same 115-pair decontamination protocol on a different model family (for example, a 70B or mixture-of-experts model) and a different agentic harness; if any same-trigger-type pairing shows full persistence, or the erasure rate falls well below 56%, the paper's central regularity is contradicted.","tokens_in":17242,"feed_emoji":"🛡️","tokens_out":9219,"duration_ms":83391,"temperature":0.7,"pith_summary":"This paper studies a two-step defense against hidden backdoors in fine-tuned LLM agents: first install a known backdoor with a benign payload, then unlearn it, hoping the unknown original backdoor is removed as a side effect. Across 115 controlled experiments, the defensive fine-tune alone erases about 56% of original backdoors, and the subsequent decontamination step drives almost all survivors to erasure. The paper also establishes that trigger recognition and malicious execution are behaviorally dissociable: a model can keep recognizing a trigger while refusing to perform the harmful action. If these dynamics generalize, decontamination becomes a credible baseline defense for unknown agent backdoors, and hidden backdoors are less durable than often assumed.","feed_headline":"Defensive poisoning erases 56% of hidden backdoors in AI agents","feed_subtitle":"Install a decoy backdoor, unlearn it, and most unknown triggers disappear.","key_machinery":"The load-bearing device is a decoupled backdoor design that treats trigger, response, teacher model, and fine-tuning method as independent axes, combined with a two-part measurement scheme: each trigger family emits a distinctive recognition phrase, so the paper can score trigger recognition (Rec) separately from malicious execution (ASR). The recognition phrase makes the behavioral dissociation visible — a model that recognizes the trigger but refuses to act has high Rec and zero ASR. The same-trigger-type regularity is explained as an overwrite of a shared recognition-to-task mapping: when A and B share a recognition phrase, B's fine-tune rewrites the mapping, so A's trigger inherits B's payload rather than keeping its own action.","core_discovery":"The central claim is that the two-step decontamination procedure — installing a known-trigger, benign-response backdoor B over an unknown backdoor A, then unlearning B — removes most A's as a side effect. In experiments with an 8-billion-parameter open-weight model on a tool-calling benchmark, defensive poisoning alone erases A in about 56% of valid pairs; unlearning B then clears almost all survivors; and same-trigger-type pairings never produce persistence, meaning A's trigger is either erased or rerouted to fire B's payload, but A's own attack never survives. The paper further shows that co-installing up to four backdoors makes them harder to remove (only about 36% erased by defensive poisoning), yet decontaminating one known co-resident still removes 87% of the others (52 of 60). Internal visualizations with a Jacobian lens show that even after decontamination restores benign responses, traces of original trigger awareness remain at intermediate layers.","pith_inferences":["If the decontamination dynamics transfer across model families, the same-trigger-type 'never persists' regularity could become a design rule for defender-chosen triggers: pick a trigger family you can guess, and you rule out full persistence.","The residual trigger-awareness traces suggest a targeted defense: instead of full unlearning, one could edit or mask the intermediate layers that encode recognition, potentially achieving more durable decontamination with less utility loss.","Because all results come from one model and one harness, the most informative next experiment is a multi-model, multi-harness replication; if the about-56% erasure rate moves sharply, the headline numbers should be read as properties of this task distribution rather than of agent backdoors generally.","The paper's teacher-dependence observation hints that backdoor durability may track the trajectory distribution, so a natural extension is to test whether decontamination succeeds better against teachers that produce shorter, less varied traces."],"forward_implications":["Defenders without any knowledge of the attacker's trigger can use defensive poisoning plus unlearning as a practical baseline, because it removes the majority of unknown backdoors and nearly all survivors.","The recognition/execution dissociation means safety can be measured behaviorally: an agent that recognizes a trigger but refuses to act is behaviorally clean even if internal trigger traces remain.","Because same-trigger-type pairings never persist, a defender who can guess the general trigger family gains a stronger guarantee: the original attack will not survive in its original form.","Attacker-side redundancy is not free: co-installing multiple backdoors raises resistance to about 36% erasure, but collateral decontamination still removes 87% of co-residents, so robustness against decontamination is limited.","The residual trace awareness found by the Jacobian lens implies that post-decontamination checkpoints may still be vulnerable to later fine-tuning that re-activates the original trigger."],"supporting_citations":[{"why":"Formulates backdoor attacks on LLM-based agents with triggers in queries or environment observations, establishing the threat model this paper defends against.","marker":"Yang et al. 2024"},{"why":"Shows fine-tuning on poisoned agent traces can implant harmful tool-use backdoors, motivating the study of post-installation dynamics.","marker":"Wang et al. 2024"},{"why":"AgentDojo provides the dynamic tool-calling environment that the paper's harness forks, supplying state-based success checks.","marker":"Debenedetti et al. 2024"},{"why":"AgentDyn is the evaluation harness used in all experiments, providing the indirect prompt-injection tasks.","marker":"Li et al. 2026"},{"why":"Introduces the defensive-poisoning procedure for instruction-tuned language models that this paper studies in agents.","marker":"Kim & Lee 2026"},{"why":"Provides the Jacobian-lens (J-Lens) method adapted to visualize post-decontamination internal representations.","marker":"Gurnee et al. 2026"},{"why":"Demonstrates that trigger-conditioned behaviors can persist through safety training, framing the persistence outcome measured here.","marker":"Hubinger et al. 2024"}],"fun_headline_variants":["Decoy backdoor unlearning wipes most hidden backdoors","Co-installed backdoors resist poisoning but unlearning clears 87%","Traces of hidden backdoors persist even after decontamination","Backdoor unlearning erases 56%, nearly all with decontamination"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"All headline numbers come from experiments on one 8-billion-parameter open-weight model and one tool-calling harness; if other models or agent frameworks behave differently, the reported rates and the 'same trigger type never persists' rule may not hold.","fun_headline_variants_meta":{"raw":{"variants":["Decoy backdoor unlearning wipes most hidden backdoors","Co-installed backdoors resist poisoning but unlearning clears 87%","Traces of hidden backdoors persist even after decontamination","Backdoor unlearning erases 56%, nearly all with decontamination"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001,"raw_usage":{"total_tokens":4269,"prompt_tokens":1021,"completion_tokens":3248,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":637,"completion_tokens_details":{"reasoning_tokens":3172}},"tokens_in":637,"tokens_out":3248,"duration_ms":22177,"temperature":1.0,"reasoning_tokens":3172,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:12:10.060076+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the same 115-pair decontamination protocol on a different model family (for example, a 70B or mixture-of-experts model) and a different agentic harness; if any same-trigger-type pairing shows full persistence, or the erasure rate falls well below 56%, the paper's central regularity is contradicted.","supporting_citations":[{"cited_title":"Merging Triggers, Breaking Backdoors: Defensive Poisoning for Instruction-Tuned Language Models","cited_arxiv_id":"2601.04448","evidence_quote":"Introduces the defensive-poisoning procedure for instruction-tuned language models that this paper studies in agents."},{"cited_title":"Verbalizable representations form a global workspace in language models","cited_arxiv_id":null,"evidence_quote":"Provides the Jacobian-lens (J-Lens) method adapted to visualize post-decontamination internal representations."}],"review_version":1}