REVIEW 2 major objections 5 minor 29 references
When Experience Becomes Instruction: Trajectory Poisoning in Self-Evolving Agent Skill Systems
T0 review · 2 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read An attacker who seeds 10% of the shared trajectory pool can get a self-evolving agent system to author a skill containing the attacker's chosen behavior.
desk verdict First clean attack on the evidence-promotion step of self-evolving skill systems, with strong ablations; the 91% headline is conditional on disabling optional quality gates, but the core mechanism holds up. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is PoisonedEvolution's trajectory transformation $\tau' = \mathcal{T}(\tau; s, b)$, which keeps the original task and outcome while inserting the target behavior $b$ at a plausible workflow locus and tying it to the outcome with a causal bridge: a constraint or requirement followed by $b$ and then successful completion, or a failed trace whose failure appears explained by the absence of $b$. The success criterion $\mathrm{PoisoningSuccess} = C_1 \wedge C_2 \wedge C_3$ makes the mechanism explicit, with $C_2$ Evolution Attribution carrying the load because the evolver must interpret the repeated pattern as reusable experience, not as a one-off accident or an unsafe detour. Recurrence is what makes the attribution work: a single poisoned record is weak, while two or three consistent records look exactly like the experience the pipeline is designed to distill.
What would settle it
Run the main SkillClaw evaluation at $n=30$, $k=3$ across the six evolvers and four behavior families with the session-quality judge and publish-time verifier enabled and see whether the skill embedding rate falls from 91.0% to near zero; a complementary test would spread the attacker's records across diverse contributor identities to see whether the pilot three-user provenance gate is actually evadable.
Extended reading notes
Core claim
The paper's central claim is that trajectory-grounded self-evolving skill systems are vulnerable to artifact poisoning through evidence alone. The attacker, modeled as a skill-visible black-box contributor with bounded input-side control, transforms a few of its own trajectories so that the target behavior appears causally relevant to the recorded outcome, recurs across the poisoned subset, and is encoded in actionable, domain-aligned language. Poisoning success is defined as the conjunction of three necessary conditions: Inclusion (poisoned records pass the filter and enter the evolution stream), Evolution Attribution (the evolver credits the behavior as a reusable, skill-worthy pattern rather than noise), and Realization (the behavior survives summarization and merge into the generated skill). The paper identifies Evolution Attribution as the distinctive bottleneck that separates this attack from retrieval-time poisoning, and its main empirical result is a 91.0% skill embedding rate at 10% attacker support across six evolvers and four security-effect families in SkillClaw, with a 61.5% rate on Trace2Skill.
Load-bearing premise
The headline success rate assumes that the pipeline's optional quality gates — the session-quality judge and publish-time verifier, which the main evaluation leaves disabled — would not reject the poisoned trajectories if they were turned on.
Editorial extensions
If this is right
- A system that accepts trajectories from untrusted or Sybil contributors can be made to author a poisoned skill with only a 10% adversarial share of the evidence pool.
- Raising attacker support from one to three consistent records in a 30-record batch lifts success from 5/25 to 25/25, so recurrence is the decisive input-side lever.
- The attack transfers across different evolution architectures, so defenses that check only final skill text will miss the same poisoning effect in a differently routed pipeline.
- Effective mitigation must happen at promotion time: provenance-aware checks that establish independent support before a pattern becomes shared instruction.
- Because a poisoned skill guides future agents, repeated evolution cycles could convert one injection into self-confirming evidence, which the paper's single-cycle scope does not cover.
Reading between the lines
- The attribution-bottleneck framing likely applies to any agent system that distills repeated, goal-linked observations into durable policy, such as memory consolidation or experience replay.
- An adaptive attacker could plausibly evade the pilot three-user provenance gate by spreading poisoned records across multiple identities; testing this evasion is a natural next step the paper does not run.
- Because the success ordering of the four behavior families reversed between SkillClaw and Trace2Skill, a detector tuned to one family or pipeline may miss the same effect when the behavior is framed differently.
- A longitudinal evaluation of multi-cycle evolution is the key unmeasured consequence: a poisoned skill may generate trajectories that make the pattern look independently validated, turning a single injection into a stable backdoor.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces PoisonedEvolution, a trajectory-poisoning attack on self-evolving skill (SES) systems that distill agent trajectories into persistent skills. The attack is modeled as a skill-visible black-box attacker who contributes a bounded number of transformed trajectories to a shared evidence pool. Success is decomposed into three necessary conditions: C1 Inclusion, C2 Evolution Attribution, and C3 Realization, with attribution identified as the distinctive bottleneck. The authors evaluate the attack on two pipelines: SkillClaw (six LLM evolvers, four security-effect families, n=30, k=3) and Trace2Skill (same budget), reporting 91.0% Skill Embedding Rate (SER) on SkillClaw and 61.5% on Trace2Skill. Ablations show that recurring support, causal framing, and domain-aligned encoding are key determinants, and a pilot provenance-diversity gate is explored as a defense. The paper claims artifact poisoning, not runtime compromise, and explicitly limits its scope to the evidence-to-artifact boundary.
Significance. The central claim is important: if a 10% attacker support reliably causes an SES pipeline to author a skill containing attacker-selected behavior, this identifies a new security boundary distinct from direct skill-file poisoning or RAG retrieval poisoning. The paper's main strengths are the coherent C1–C3 decomposition, the transparent reporting of embedded/completed trial counts, the cross-system transfer attempt, and the inclusion of condition ablations that support the attribution-bottleneck hypothesis. The pilot provenance-diversity gate is a useful concrete design implication. The principal weakness is that the headline result is obtained with two optional SkillClaw quality modules disabled, with no experiment showing how the attack behaves when they are enabled; this leaves the generalization of the 91.0% figure to a default deployment unsupported. The utility check also appears to use the same trajectory-collection split, which limits its validity. If these gaps are addressed, the paper would be a solid contribution to the security evaluation of agent skill pipelines.
major comments (2)
- [Empirical Evaluation, RQ1 (Table 1) and Background] The main SkillClaw evaluation disables SkillClaw's optional session-quality judge and publish-time verifier, and the paper states that these are 'configuration-dependent quality gates rather than source-trust checks' without providing any experiment with them enabled. Because the abstract and conclusion report the 91.0% SER without this caveat, the headline claim is not yet supported for a default SkillClaw deployment. If either module rejects poisoned trajectories at inclusion (breaking C1) or rejects the generated artifact before publication (breaking C3), the attack's success could drop substantially. Please run the main n=30, k=3 matrix, or at least a representative subset covering all six evolvers and four families, with both modules enabled, and report SER alongside C1 and C3 separately. The abstract's unconditional claim should be qualified until this experiment is provided.
- [Empirical Evaluation, RQ1 (Benign-task utility check)] The benign-task utility check is performed 'on the 100-task trajectory-collection split' with Qwen3.5-122B-A10B, which appears to be the same 100 SpreadsheetBench training tasks used to create the source corpus described at the start of the evaluation section. If the utility evaluation uses the same tasks from which poisoned trajectories were constructed, the comparison between poisoned-evolved skill (20.0% Hard) and no-skill (18.0% Hard) is not a valid held-out utility measurement; it may simply reflect overfitting to the poison distribution. Please evaluate on a held-out set of tasks disjoint from the poisoning corpus, or explicitly state and justify that the 100-task split is separate from the poisoning construction. This matters because the paper presents the utility check as evidence that poisoning 'coexists with the normal-task benefit of skill use.'
minor comments (5)
- [Empirical Evaluation, RQ1 (no-init diagnostic)] The no-init diagnostic is described as using 'four representative evolvers' but reports 490/600 trials. With 25 trials per model–family cell and four families, four evolvers would yield 400 trials, not 600. Please reconcile the denominator and specify which evolvers and how many trials per cell were used; the current text is arithmetically inconsistent.
- [Metrics] The SER detection relies on 'family-specific, diff-aware canary rules' but the exact rules are not provided and no code is released. To make the 546/600 and 369/600 counts reproducible, please include the full rule set in an appendix or release the evaluation code.
- [Empirical Evaluation, RQ1] The conversion of GLM-5 execution logs into SkillClaw-compatible sessions is not described. Please specify the conversion procedure, including how task instructions, tool calls, and outcomes are mapped to session fields, since the attack's C1 feasibility depends on this representation.
- [Figure 3] The C3-oriented comparison varies the behavioral encoding and the workflow locus simultaneously, and the text acknowledges this; the caption should state explicitly that this is a stress test rather than a single-variable causal ablation, to avoid readers misinterpreting the 0/25 result for the 'Auxiliary' variant.
- [Discussion] The distinction between artifact poisoning and runtime impact is well taken, but the paper could define an explicit 'artifact-only' scope note in the metrics subsection so that future readers do not confuse SER with execution-based attack success rates.
Circularity Check
No significant circularity: the attack success metric measures an empirical outcome, not an input tautology, and no load-bearing claim reduces to a self-citation or fitted parameter.
full rationale
The paper's derivation is self-contained as an empirical attack evaluation. PoisoningSuccess is defined as C1 ∧ C2 ∧ C3 (Eq. 3) and SER (Eq. 5) is an empirical frequency of newly embedded target behavior in evolved skill diffs, measured against inert canary rules. The attack construction (Eqs. 6–8, Algorithm 1) deliberately inserts the target behavior into attacker-owned trajectories, but that is the attack being evaluated, not a tautological prediction; the results vary substantially across evolvers (70/100 to 100/100), families (124/150 to 146/150), and representation ablations (0/25 to 25/25), so SER is not forced by construction. The attacker budget k=3 is a chosen operating point with reported sensitivity at k=1,2 and n=10,30,100, not a hidden tuning parameter fit to the headline. The paper cites SkillClaw (Ma et al. 2026) and Trace2Skill (Ni et al. 2026) as external systems, with no overlapping authors in this paper; no load-bearing self-citation or uniqueness theorem is invoked. The only notable caveats — the main matrix disables SkillClaw's optional session-quality judge and publish-time verifier, and the provenance-diversity gate is preliminary — are external-validity limitations, not circular steps. The paper even flags a potential feedback-loop circularity in deployed systems and explicitly does not measure it, which is an appropriate scope limitation rather than a circular derivation.
Assumptions & free parameters
free parameters (1)
- attacker support ratio rho = k/n =
10% (k=3, n=30)
assumptions (3)
- domain assumption Attackers can contribute trajectories to the shared pool in SkillClaw and similar multi-user systems.
- domain assumption Attackers can inspect the installed target skill (skill-visible black-box).
- domain assumption SkillClaw's optional session-quality judge and publish-time verifier, when disabled, would not have blocked the poisoning if enabled.
Cite this review
Pith. "Pith review of When Experience Becomes Instruction: Trajectory Poisoning in Self-Evolving Agent Skill Systems." pith.science (2026). https://pith.science/paper/5V24RV5H
@misc{pith2026260805563,
author = {Pith},
title = {Pith review of: When Experience Becomes Instruction: Trajectory Poisoning in Self-Evolving Agent Skill Systems},
year = {2026},
howpublished = {\url{https://pith.science/paper/5V24RV5H}},
note = {Machine review of arXiv:2608.05563}
}
read the original abstract
Self-evolving skill (SES) systems distill agent trajectories into persistent skills, allowing untrusted experience to become trusted instruction. We introduce PoisonedEvolution, a trajectory-poisoning attack on this promotion process. Our skill-visible black-box attacker can inspect a target skill and contribute bounded evidence, but cannot observe private pools or evolution logic or edit the skill bank. Artifact poisoning requires Inclusion, Evolution Attribution, and Realization. Attribution is the distinctive bottleneck: the target behavior must appear causally useful, recurrent, and generalizable before promotion. We evaluate four representative security-effect families using inert canary specifications. At 10% attacker support, across six mainstream LLM evolvers in SkillClaw, PoisonedEvolution embeds target behaviors in 546/600 trials (91.0% SER). On the structurally different Trace2Skill pipeline at the same ratio, it embeds target behaviors in 369/600 trials (61.5% SER), demonstrating transfer across evolution architectures. In a representative controlled study, three consistent attacker records suffice in a 30-record batch, whereas a single record is much weaker. Ablations identify recurring support, causal framing, and domain-aligned encoding as the main determinants of success. These findings expose evidence promotion as a security boundary for self-evolving agents.
Figures
Reference graph
Works this paper leans on
-
[1]
arXiv preprint arXiv:2604.08377 , year =
Ziyu Ma and Shidong Yang and Yuxiang Ji and Xucong Wang and Yong Wang and Yiming Hu and Tongwen Huang and Xiangxiang Chu , title =. arXiv preprint arXiv:2604.08377 , year =
-
[2]
arXiv preprint arXiv:2603.01145 , year =
Yutao Yang and Junsong Li and Qianjun Pan and Bihao Zhan and Yuxuan Cai and Lin Du and Jie Zhou and Kai Chen and Qin Chen and Xin Li and Bo Zhang and Liang He , title =. arXiv preprint arXiv:2603.01145 , year =
-
[3]
arXiv preprint arXiv:2603.25158 , year =
Jingwei Ni and Yihao Liu and Xinpeng Liu and Yutao Sun and Mengyu Zhou and Pengyu Cheng and Dexin Wang and Erchao Zhao and Xiaoxi Jiang and Guanjun Jiang , title =. arXiv preprint arXiv:2603.25158 , year =
-
[4]
arXiv preprint arXiv:2602.02474 , year =
Haozhen Zhang and Quanyu Long and Jianzhu Bao and Tao Feng and Weizhi Zhang and Haodong Yue and Wenya Wang , title =. arXiv preprint arXiv:2602.02474 , year =
-
[5]
arXiv preprint arXiv:2602.08234 , year =
Peng Xia and Jianwen Chen and Hanyang Wang and Jiaqi Liu and Kaide Zeng and Yu Wang and Siwei Han and Yiyang Zhou and Xujiang Zhao and Haifeng Chen and Zeyu Zheng and Cihang Xie and Huaxiu Yao , title =. arXiv preprint arXiv:2602.08234 , year =
-
[6]
Hanrong Zhang and Shicheng Fan and Henry Peng Zou and Yankai Chen and Zhenting Wang and Jiayu Zhou and Chengze Li and Wei-Chieh Huang and Yifei Yao and Kening Zheng and Xue Liu and Xiaoxiao Li and Philip S. Yu , title =. arXiv preprint arXiv:2604.01687 , year =
-
[7]
arXiv preprint arXiv:2604.10674 , year =
Hao Wang and Guozhi Wang and Han Xiao and Yufeng Zhou and Yue Pan and Jichao Wang and Ke Xu and Yafei Wen and Xiaohu Ruan and Xiaoxin Chen and Honggang Qi , title =. arXiv preprint arXiv:2604.10674 , year =
-
[8]
arXiv preprint arXiv:2605.27366 , year =
Huawei Lin and Peng Li and Jie Song and Fuxin Jiang and Tieying Zhang , title =. arXiv preprint arXiv:2605.27366 , year =
Show all 29 references
-
[9]
arXiv preprint arXiv:2602.12430 , year =
Renjun Xu and Yang Yan , title =. arXiv preprint arXiv:2602.12430 , year =
-
[10]
arXiv preprint arXiv:2604.09378 , year =
Guiyao Tie and Jiawen Shi and Pan Zhou and Lichao Sun , title =. arXiv preprint arXiv:2604.09378 , year =
-
[11]
Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security (AISec) , year =
Kai Greshake and Sahar Abdelnabi and Shailesh Mishra and Christoph Endres and Thorsten Holz and Mario Fritz , title =. Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security (AISec) , year =
-
[12]
arXiv preprint arXiv:2602.20156 , year =
David Schmotz and Luca Beurer-Kellner and Sahar Abdelnabi and Maksym Andriushchenko , title =. arXiv preprint arXiv:2602.20156 , year =
-
[13]
arXiv preprint arXiv:2603.22489 , year =
Charoes Huang and Xin Huang and Ngoc Phu Tran and Amin Milani Fard , title =. arXiv preprint arXiv:2603.22489 , year =
-
[14]
34th USENIX Security Symposium (USENIX Security 25) , pages =
Wei Zou and Runpeng Geng and Binghui Wang and Jinyuan Jia , title =. 34th USENIX Security Symposium (USENIX Security 25) , pages =
-
[15]
Advances in Neural Information Processing Systems , volume =
Zhaorun Chen and Zhen Xiang and Chaowei Xiao and Dawn Song and Bo Li , title =. Advances in Neural Information Processing Systems , volume =
-
[16]
Advances in Neural Information Processing Systems , year =
Shen Dong and Shaochen Xu and Pengfei He and Yige Li and Jiliang Tang and Tianming Liu and Hui Liu and Zhen Xiang , title =. Advances in Neural Information Processing Systems , year =
-
[17]
arXiv preprint arXiv:2604.02623 , year =
Wei Zou and Mingwen Dong and Miguel Romero Calvo and Shuaichen Chang and Jiang Guo and Dongkyu Lee and Xing Niu and Xiaofei Ma and Yanjun Qi and Jiarong Jiang , title =. arXiv preprint arXiv:2604.02623 , year =
-
[18]
arXiv preprint arXiv:2604.03081 , year =
Yubin Qu and Yi Liu and Tongcheng Geng and Gelei Deng and Yuekang Li and Leo Yu Zhang and Ying Zhang and Lei Ma , title =. arXiv preprint arXiv:2604.03081 , year =
-
[19]
arXiv preprint arXiv:2606.07943 , year =
Haochang Hao and Dehai Min and Zhifang Zhang and Yunbei Zhang and Miao Xu and Yingqiang Ge and Lu Cheng , title =. arXiv preprint arXiv:2606.07943 , year =
-
[20]
arXiv preprint arXiv:2604.22888 , year =
Wenjie Xiao and Xuehai Tang and Biyu Zhou and Songlin Hu and Jizhong Han , title =. arXiv preprint arXiv:2604.22888 , year =
-
[21]
Advances in Neural Information Processing Systems , year =
Zeyao Ma and Bohan Zhang and Jing Zhang and Jifan Yu and Xiaokang Zhang and Xiaohan Zhang and Sijia Luo and Xi Wang and Jie Tang , title =. Advances in Neural Information Processing Systems , year =. 2406.14991 , archivePrefix =
-
[22]
arXiv preprint arXiv:2404.13208 , year =
Eric Wallace and others , title =. arXiv preprint arXiv:2404.13208 , year =
-
[23]
arXiv preprint arXiv:2410.05451 , year =
Sizhe Chen and Arman Zharmagambetov and Saeed Mahloujifar and Kamalika Chaudhuri and David Wagner and Chuan Guo , title =. arXiv preprint arXiv:2410.05451 , year =
-
[24]
arXiv preprint arXiv:2503.18813 , year =
Edoardo Debenedetti and Ilia Shumailov and Tianqi Fan and Jamie Hayes and Nicholas Carlini and Daniel Fabian and Christoph Kern and Chongyang Shi and Andreas Terzis and Florian Tram\`er , title =. arXiv preprint arXiv:2503.18813 , year =
-
[25]
28th USENIX Security Symposium (USENIX Security 19) , pages =
Santiago Torres-Arias and Hammad Afzali and Trishank Karthik Kuppusamy and Reza Curtmola and Justin Cappos , title =. 28th USENIX Security Symposium (USENIX Security 19) , pages =
-
[26]
2025 , note =
Anthropic , title =. 2025 , note =
2025
-
[27]
2025 , note =
Vercel , title =. 2025 , note =
2025
-
[28]
2025 , note =
Smithery , title =. 2025 , note =
2025
-
[29]
C. J. Clopper and E. S. Pearson , title =. Biometrika , volume =
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.