Pith.

Cryptography and Security · arXiv:2611.10542 v1 · submitted 7 Nov 2026 · 55 references

When Experience Becomes a Backdoor: Poisoning Self-Evolving LLM Agents via Failure-Aware Task Optimization

Minhua Lin, Zongyu Wu, Hanqing Lu, Xianfeng Tang, Xiang Zhang, Qi He, Suhang Wang

The paper says

“Large language model (LLM) agents increasingly improve themselves through self-evolution, distilling execution experience into persistent, reusable skills. However, this persistence creates a new attack surface: malicious rules implanted into skills can compromise subsequent agent behavior.”

more of the abstract

Existing skill backdoor attacks typically assume direct manipulation of the victim's skills, which is impractical when skills are constructed internally by a trusted evolver. In this paper, we study a novel problem of task-level backdoor attacks, where the attacker can only inject poisoned evolution tasks, inducing the agent to implant backdoors into its own skills. Since the attacker controls neither skill induction nor downstream execution, a natural approach is to iteratively optimize poisoned tasks based on attack success. However, our preliminary analysis reveals two key challenges: (i) failures at different stages of skill evolution and downstream execution can produce indistinguishable outcomes, providing little guidance for targeted poisoned-task updates; and (ii) unsuccessful updates may undermine earlier progress, while simply discarding them may lead to repeated ineffective updates. To address these challenges, we propose BadEvolve, a failure-aware task-poisoning framework against self-evolving LLM agents. BadEvolve diagnoses attack failures across four stages, i.e., Induction, Activation, Following, and Effect, and directs updates toward the failing stage. It further employs rollback to preserve earlier progress and failure memory to avoid repeating ineffective updates. Extensive experiments across diverse self-evolution frameworks, benchmarks, and backbone LLMs demonstrate the effectiveness of BadEvolve. Our code is available at https://github.com/ventr1c/BadEvolve.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 10 Nov 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up. BadEvolve is a solidly executed red-teaming contribution that identifies a real vulnerability in self-evolving LLM agents and proposes a novel, empirically supported attack framework. This line is wrong

5 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 55 references say.

5 minor objections

The paper says, at §4.3.2, Eq. (11)

Pith says: minor objection 1 of 5

The failure memory stores records of rejected candidates, but the paper does not specify how the LLM optimizer reads or integrates these records into the next proposal (Eq. 12). Adding a short description of the prompt template or representation used to provide ℳ to q would improve reproducibility.

The paper says, at §5.1, Implementation Details

Pith says: minor objection 2 of 5

The paper reports K=5 self-evolution runs per candidate evaluation and L=5 rounds, but does not discuss computational cost (e.g., total LLM calls or wall-clock time). A brief note on cost would help practitioners assess feasibility.

show 3 more

The paper says, at §4.2, Eq. (8)

Pith says: minor objection 3 of 5

The stage-wise pass rates rely on inspecting the skill state for the malicious rule (Induction) and the solver's loaded skills (Activation). The paper should clarify that this inspection is performed on the surrogate agent's internal state, which the attacker controls; the victim's state is never inspected. This is already implicit from the threat model but could be stated explicitly to avoid confusion.

The paper says, at Appendix D.5, Table 7

Pith says: minor objection 4 of 5

The trigger for SpreadsheetBench includes an 'ARCHIVAL_AUDIT' sheet; please verify spelling ('Audit' vs. 'Audit' in the table; the text uses 'AdvitTrail' which may be a typo).

The paper says, at §3.2.2, Attacker's Knowledge and Capability

Pith says: minor objection 5 of 5

The threat model states the attacker knows the self-evolution mechanism and has access to the evolution set and a validation set. It also assumes the attacker can replicate self-evolution with a surrogate. The paper should note that this gray-box setting is realistic for open-source frameworks, which is a strength, but lso a limitation if the framework is proprietary.

What would settle it

Re-run the primary Payload Injection comparison (Table 2) for Trace2Skill on WikiTQ with Kimi-K2.5 using 5 additional random seeds for self-evolution. If the average ASR remains within ±10% of the reported 69.0% and CACC within 2 pp of CACC0, the result is robust; if variance is larger, the claim still holds but with noted sensitivity.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 10 Nov 2026

  1. Reader holds up confidence high
  2. Skeptic agrees 30 s · 24,939 tokens read, 2,740 written
  3. Editor wrote the verdict above 17 s · 25,722 tokens read, 1,303 written
  4. Referee accept: 5 minor 14 s · 25,280 tokens read, 1,558 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

Here's my take on 2611.10542.

The paper is worth your time if you care about security of self-evolving LLM agents. It defines a new threat model—task-level backdoor attack where the attacker only injects evolution tasks, never directly touches skills—and builds an optimization framework (BadEvolve) that actually works across four evolution frameworks, three datasets, and four backbones. The average payload-injection ASR of 79.6% with utility loss under 2 pp is real. The four-stage diagnosis (Induction, Activation, Following, Effect) and the rollback-with-memory gate are not just decoration; the ablation shows each piece pulls its weight. Code is released. The authors also test the load-bearing assumption—surrogate transfer—and report where ASR drops (e.g., 100% to 65% on WikiTQ with SkillOpt). They don't oversell.

Soft spots are minor. The stage detection rules (Eq. 8) are described at a high level; the implementation details are in the appendix but could be sharper for reproducibility. The optimizer is Claude Sonnet 4.6—proprietary—so the method has a closed-box dependency, though the general idea (LLM-as-optimizer with stage feedback) is clearly laid out and portable. The strongest results (100% ASR in Table 2) occasionally come from settings where CACC0 is already relatively low (e.g., 76% on SB with SkillOpt), which might mean the attack exploits an already weak agent; but the overall pattern holds.

For a reading group: maybe. It's a targeted security paper, not a broad method. But for anyone building or deploying self-evolving agents, this is relevant. I would cite it if I were working on agent safety. The paper shows clear thinking, honest experiments, and proper acknowledgment of limitations. A serious editor should send this to peer review without hesitation.

The strongest objection

The central claim—that BadEvolve implants backdoors into self-evolving LLM agents via poisoned evolution tasks—is supported by extensive experiments across multiple frameworks, backbones, and settings. The weakest point identified by the Reader (transferability of surrogate-optimized tasks to an unknown victim backbone) is openly tested in §5.3, and while ASR drops in some configurations (e.g., from 100% to 65% on WikiTQ with SkillOpt), the average ASR remains 84.6% and utility is preserved. The paper acknowledges this limitation and does not overclaim universal transfer; the experimental coverage (4 frameworks, 4 LLMs, 2 objectives) is substantial. No internal inconsistency, missing proof, or critical flaw was detected. The stage-wise detection method (Eq. 8) is not fully specified in implementation, but the case studies (Table 5, 8) and ablation (Fig.4) demonstrate its empirical value. Therefore, I find no load-bearing concern that would change the verdict.

The Reader's verdict ACCEPT with high confidence is appropriate. The weakest assumption (surrogate transfer) is correctly identified and experimentally investigated; the results show high average success with some variance, which is honestly reported. The paper's core contribution—a failure-aware optimization framework—is clearly described, ablated, and shown to outperform baselines. No new load-bearing concern emerged from a careful re-reading. The stage-wise detection implementation detail is secondary to the empirical validation. Therefore, no verdict adjustment is needed.

The referee's report

If the claims hold, this paper identifies a practically relevant attack surface in a growing paradigm (self-evolving LLM agents). The threat model is realistic—attackers can inject tasks through public or user-submitted content without accessing the agent's internals—and the proposed method is the first to succeed with only task-level poisoning. The evaluation is unusually thorough: four frameworks, four LLMs, two objectives, ablation studies, and explicit transferability tests. The paper also provides actionable analysis (stage-wise diagnosis) that could inform defenses. The results are significant for the security and AI safety communities.

Pith's reading of the paper (a machine's summary, not the authors')

This paper shows that self-evolving LLM agents, which improve themselves by distilling execution experience into reusable skills, are vulnerable to a new class of backdoor attack. Unlike prior work that assumed direct skill manipulation, the proposed BadEvolve attack works by injecting poisoned evolution tasks only—the agent's own evolver then writes the malicious rule into its skills. The key insight is to diagnose failures at four distinct stages (Induction, Activation, Following, Effect) and stabilize the optimization with a rollback and failure memory mechanism. Experiments across four evolution frameworks, three benchmarks, and several large language models show an average attack success rate of 79.6% for payload injection while keeping clean task accuracy within 1.1 percentage points of the unpoisoned baseline.

What carries the argument. The central mechanism is the BadEvolve framework, which consists of two components: (1) stage-wise feedback that decomposes the attack pipeline into Induction, Activation, Following, and Effect, each diagnosed from the evolved skills and execution trajectories; and (2) a failure-aware gate that decides whether to retain a candidate update based on end-to-end attack success and stage-wise progress, while recording rejected updates in a failure memory to guide future proposals. Together, these mechanisms steer an LLM-based attack optimizer to iteratively refine the poisoned instruction shared across poisoned tasks.

Core claim. BadEvolve is a failure-aware task-poisoning framework that solves two fundamental challenges of outcome-driven optimization for backdoor attacks on self-evolving LLM agents. First, it decomposes attack success into four sequential stages—whether the malicious rule is written into skills (Induction), loaded on trigger-attached tasks (Activation), followed by the solver (Following), and produces the intended outcome (Effect)—and provides the optimizer with per-stage pass rates to target updates precisely. Second, it uses a failure-aware gate that combines rollback (to preserve earlier progress when an update regresses) and failure memory (to avoid repeating ineffective updates). With only 20 poisoned tasks, BadEvolve achieves an average attack success rate of 79.6% for Payload Injection and up to 80.0% for Answer Corruption, while maintaining task utility across diverse self-evolution frameworks, benchmarks, and backbone LLMs.

The premise it rests on. The attack assumes that the attacker can run a surrogate agent with the same self-evolution framework as the victim and that the poisoned tasks optimized on that surrogate will transfer to the victim, even when the victim's backbone LLM is unknown.

If the paper is right (Pith's inferences, not the authors')

Self-evolving LLM agents that rely on skill-based learning are vulnerable to backdoor attacks even when the attacker cannot directly modify skills or observe the victim's internals.

The attack generalizes across different skill-evolution frameworks (Trace2Skill, SkillGrad, SkillOpt, A-Evolve) and backbone LLMs (Kimi-K2.5, GPT-OSS-120B, Claude Sonnet 4.6, Qwen3-235B).

The attack succeeds in the gray-box setting where the victim's backbone is unknown, with average ASR of 84.6% across surrogate–victim pairs.

Both Payload Injection (adding hidden behavior) and Answer Corruption (making agent produce wrong answers) are achievable without degrading clean-task accuracy.

This work motivates defenses that verify the content and provenance of evolution tasks, or that detect anomalous skill rules induced by poisoned experience.

The failure-aware optimization approach may generalize to other discrete optimization problems in LLM agent security, such as poisoning prompts, memory, or tool configurations, where ambiguous outcome signals and coupled effects pose similar challenges.

The transferability results show that the attack sometimes works better with a different surrogate backbone than the victim's own, suggesting that the attacker's choice of surrogate is a critical variable that deserves deeper study.

A direct defense would be to sanitize the evolution task set by filtering tasks that contain explicit instructions to condition behavior on a trigger, or by running a separate inspection of induced skills for spurious rules.

The paper considers only skill-based self-evolution; agents that evolve by updating other harness components (e.g., prompts, reflection templates) might require different attack strategies, and their vulnerability remains an open question.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Large language model (LLM) agents increasingly improve themselves through self-evolution, distilling execution experience into persistent, reusable skills. However, this persistence creates a new attack surface: malicious rules implanted into skills can compromise subsequent agent behavior. Existing skill backdoor attacks typically assume direct manipulation of the victim's skills, which is impractical when skills are constructed internally by a trusted evolver. In this paper, we study a novel problem of task-level backdoor attacks, where the attacker can only inject poisoned evolution tasks, inducing the agent to implant backdoors into its own skills. Since the attacker controls neither skill induction nor downstream execution, a natural approach is to iteratively optimize poisoned tasks based on attack success. However, our preliminary analysis reveals two key challenges: (i) failures at different stages of skill evolution and downstream execution can produce indistinguishable outcomes, providing little guidance for targeted poisoned-task updates; and (ii) unsuccessful updates may undermine earlier progress, while simply discarding them may lead to repeated ineffective updates. To address these challenges, we propose BadEvolve, a failure-aware task-poisoning framework against self-evolving LLM agents. BadEvolve diagnoses attack failures across four stages, i.e., Induction, Activation, Following, and Effect, and directs updates toward the failing stage. It further employs rollback to preserve earlier progress and failure memory to avoid repeating ineffective updates. Extensive experiments across diverse self-evolution frameworks, benchmarks, and backbone LLMs demonstrate the effectiveness of BadEvolve. Our code is available at https://github.com/ventr1c/BadEvolve.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-11-10

The authors' main claim, as the classifier picked it out 0.99

“Extensive experiments across diverse self-evolution frameworks, benchmarks, and backbone LLMs demonstrate the effectiveness of BadEvolve.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.21 · states limitation 0.27 · human participants 0.03 · controlled comparison 0.17 · randomized assignment 0.02 · states quantitative result 0.04 · compares with prior results 0.48 · mentions code or data availability 0.98

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Large language model (LLM) agents increasingly improve themselves through self-evolution, distilling execution experience into persistent, reusable skills. background 1.00 0.06 0.03 asserted 1.00
However, this persistence creates a new attack surface: malicious rules implanted into skills can compromise subsequent agent behavior. background 0.90 0.11 0.08 asserted 0.98
Existing skill backdoor attacks typically assume direct manipulation of the victim's skills, which is impractical when skills are constructed internally by a trusted evolver. gap 0.78 0.05 0.03 asserted 1.00
In this paper, we study a novel problem of task-level backdoor attacks, where the attacker can only inject poisoned evolution tasks, inducing the agent to implant backdoors into its own skills. contribution 0.90 0.28 0.10 asserted 0.98
Since the attacker controls neither skill induction nor downstream execution, a natural approach is to iteratively optimize poisoned tasks based on attack success. method 0.66 0.14 0.04 asserted 0.64
However, our preliminary analysis reveals two key challenges: (i) failures at different stages of skill evolution and downstream execution can produce indistinguishable outcomes, providing little guidance for targeted poisoned-task updates; and (ii) unsuccessful updates may undermine earlier progress, while simply discarding them may lead to repeated ineffective updates. gap 0.85 0.90 0.03 asserted 0.92
To address these challenges, we propose BadEvolve, a failure-aware task-poisoning framework against self-evolving LLM agents. contribution 0.88 0.86 0.11 asserted 1.00
BadEvolve diagnoses attack failures across four stages, i.e., Induction, Activation, Following, and Effect, and directs updates toward the failing stage. method 0.94 0.88 0.10 asserted 1.00
It further employs rollback to preserve earlier progress and failure memory to avoid repeating ineffective updates. method 0.97 0.87 0.06 asserted 1.00
Extensive experiments across diverse self-evolution frameworks, benchmarks, and backbone LLMs demonstrate the effectiveness of BadEvolve. result 0.96 0.95 0.57 asserted 1.00
Our code is available at https://github.com/ventr1c/BadEvolve. contribution 0.81 0.50 0.02 asserted 0.97

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.99

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-10: 1 request, 8,247 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (6)
Figure 1 of the paper
Figure 1. Overview of task-level backdoor attacks against self-evolving LLM agents and the corresponding challenges. Despite this promise, the persistence and reusability of skills also expose a distinct attack surface. Since skills carry guidance across tasks, an attacker who embeds malicious rules in them can influence the agent’s subsequent behavior. A particularly concerning threat is the backdoor attack, where an agent behaves normally on benign tasks but exhibits attacker-chosen behavior on trigger-attached tasks [5, 9, 14, 35, 39, 47]. Such attacks can be especially harmful as a single…
Figure 2 of the paper
Figure 2. Cumulative stage-wise pass rates of TextGrad-Poison on trigger-attached validation tasks. GPT-OSS-120B is the victim agent’s backbone. Orange and green areas: tasks that fail at Activation and Following, respectively. 3.3 Problem Statement To achieve the attacker’s goal, we formulate this problem as a task-level backdoor attack against self-evolving LLM agents. We first define attack success on trigger-attached target tasks. Let 𝒟target ⊆ 𝒟test denote the target task set, and let 𝑎(𝑥, 𝛿) denote the operation that attaches trigger 𝛿 to task 𝑥. Given the solver’s outcome 𝑜 on the…
Figure 3 of the paper
Figure 3. An overview of proposed BadEvolve. 4.1 Task-Level Backdoor Attack Framework 4.1.1 Poisoned Task Construction We first construct the poisoned evolution tasks 𝒟poison. Since the attacker cannot modify the skills directly, the attacker-chosen behavior can arise only from a malicious rule 𝑟 that the evolver 𝒜evo writes into the skills. The rule, in turn, can only be learned from the execution experience of solving the poisoned tasks. Specifically, each poisoned task 𝑥 𝜙 𝑏 is obtained from a clean task 𝑥𝑏 ∈ 𝒟evolve by appending a poisoned instruction 𝜙 to it, where 𝜙 describes the…
Figure 4 of the paper
Figure 4. Ablation studies on Kimi-K2.5 as the backbone. 1 5 10 20 30 40 Number of Poisoned Tasks B 0 25 50 75 100 Rate (%) ASR CACC TACC CACC0 (a) WikiTQ × Kimi-K2.5 1 5 10 20 30 40 Number of Poisoned Tasks B 0 25 50 75 100 Rate (%) ASR CACC TACC CACC0 (b) SB × Claude Sonnet 4.6
Figure 5 of the paper
Figure 5. Impact of the poisoning budget 𝐵 with Trace2Skill. BadEvolve\RB and BadEvolve\FM. For example, the ASR of BadEvolve\RB drops to 33.3% on WikiTQ with Trace2Skill, and that of BadEvolve\FM drops to 0.0% on OfficeQA with SkillGrad. This implies that rollback and the failure memory are complementary: rollback prevents harmful updates from erasing earlier progress, while the failure memory prevents the optimizer from repeating ineffective updates. (iii) BadEvolve and all variants keep CACC and TACC close to CACC0, showing that the components improve attack effectiveness without…
Figure 6 of the paper
Figure 6. Cumulative stage-wise pass rates on SpreadsheetBench with Trace2Skill and Kimi-K2.5. Orange / green areas: tasks failing at Activation / Following. Solid / hollow markers: retained / rolled-back candidates. contrast, the stage-wise pass rates of BadEvolve show that Activation and Following already reach 100%, which localizes the failure at Induction (𝑝ˆ𝐼 = 0.4). By focusing its updates on Induction, BadEvolve raises 𝑝ˆ𝐼 from 0.4 to 0.8 and the ASR from 40.0% to 79.5% in round 2. More details are in Appendix F.1. Failure-aware gate stabilizes the optimization

Figures are the authors', reproduced from the paper.

References (55)
  1. [1] Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, et al. gpt-oss-120b & gpt-oss-20b model card.arXiv preprint arXiv:2508.10925, 2025 on PithPith's reviewarXiv
  2. [2] Huan ang Gao, Jiayi Geng, Wenyue Hua, Mengkang Hu, Xinzhe Juan, Hongzhang Liu, Shilong Liu, Jiahao Qiu, Xuan Qi, Qihan Ren, Yiran Wu, Hongru WANG, Han Xiao, Yuhang Zhou, Shaokun Zhang, Jiayi Zhang, Jinyu Xiang, Yixiong Fang, Qiwen Zhao, Dongrui Liu, Cheng Qian, Zhenhailong Wan... on Pith
  3. [3] Anthropic. Claude sonnet 4.6 system card, Feb 2026. URL https://www-cdn.anthropic.com/ bbd8ef16d70b7a1665f14f306ee88b53f686aa75.pdf on Pith
  4. [4] Jialuo Chen, Lingqi Jiang, Xinhao Deng, Xiaohu Du, Jianan Ma, Yunhao Feng, Yuqi Qing, Zhihao Yuan, Linkang Du, and Jingyi Wang. When experience becomes instruction: Trajectory poisoning in self-evolving agent skill systems. arXiv preprint arXiv:2608.05563, 2026 on PithPith's reviewarXiv
  5. [5] Zhaorun Chen, Zhen Xiang, Chaowei Xiao, Dawn Song, and Bo Li. Agentpoison: Red-teaming llm agents via poisoning memory or knowledge bases.Advances in Neural Information Processing Systems, 37:130185–130213, 2024 on Pith
  6. [6] Enyan Dai, Minhua Lin, Xiang Zhang, and Suhang Wang. Unnoticeable backdoor attacks on graph neural networks. InProceedings of the ACM Web Conference 2023, pages 2263–2273, 2023 on Pith
  7. [7] Shen Dong, Shaochen Xu, Pengfei He, Yige Li, Jiliang Tang, Tianming Liu, Hui Liu, and Zhen Xiang. Memory injection attacks on llm agents via query-only interaction.Advances in Neural Information Processing Systems, 38: 46697–46731, 2026 on Pith
  8. [8] Jinyuan Fang, Yanwen Peng, Xi Zhang, Yingxu Wang, Xinhao Yi, Guibin Zhang, Yi Xu, Bin Wu, Siwei Liu, Zihao Li, et al. A comprehensive survey of self-evolving ai agents: A new paradigm bridging foundation models and lifelong agentic systems.arXiv preprint arXiv:2508.07407, 2025 on PithPith's reviewarXiv
  9. [9] Yunhao Feng, Yifan Ding, Yingshui Tan, Boren Zheng, Yanming Guo, Xiaolong Li, Kun Zhai, Yishan Li, and Wenke Huang. Skilltrojan: Backdoor attacks on skill-based agent systems.arXiv preprint arXiv:2604.06811, 2026 on PithPith's reviewarXiv
  10. [10] Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. Swe- bench: Can language models resolve real-world github issues? InInternational Conference on Learning Representations, volume 2024, pages 54107–54157, 2024 on Pith
  11. [11] Doyun Kim, Chanwoo Kim, Sugyeong Eo, Yeo-Chan Yoon, and Chanjun Park. Evoskill injection: Red-teaming autonomous skill generation and evolution in self-evolving agents.arXiv preprint arXiv:2608.30429, 2026 on PithPith's reviewarXiv
  12. [12] Team Kimi, Tongtong Bai, Yifan Bai, Yiping Bao, SH Cai, Yuan Cao, Y Charles, HS Che, Cheng Chen, Guanduo Chen, et al. Kimi k2.5: Visual agentic intelligence.arXiv preprint arXiv:2602.02276, 2026 on PithPith's reviewarXiv
  13. [13] Xiangyi Li, Yimin Liu, Wenbo Chen, Bingran You, Zonglin Di, Yifeng He, Shenghan Zheng, Kyoung Whan Choe, Jiankai Sun, Shuyi Wang, et al. Skillsbench: Benchmarking how well agent skills work across diverse tasks.arXiv preprint arXiv:2602.12670, 2026 on PithPith's reviewarXiv
  14. [14] Yige Li, Hanxun Huang, Yunhan Zhao, Xingjun Ma, and Jun Sun. Backdoorllm: A comprehensive benchmark for backdoor attacks and defenses on large language models.Advances in neural information processing systems, 38, 2026 on Pith
  15. [15] Minhua Lin, Zhiwei Zhang, Enyan Dai, Zongyu Wu, Yilong Wang, Xiang Zhang, and Suhang Wang. Are you using reliable graph prompts? trojan prompt attacks on graph neural networks. InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, pages 1... on Pith
  16. [16] Minhua Lin, Zhicheng Gao, Yilong Wang, Hanqing Lu, Xiang Zhang, and Suhang Wang. Trojaning the alignment: Stealthy backdoor attacks against graph foundation models.arXiv preprint arXiv:2608.20991, 2026 on PithPith's reviewarXiv
  17. [17] MinhuaLin,HanqingLu,ZhanShi,BingHe,RuiMao,ZhiweiZhang,ZongyuWu,XianfengTang,HuiLiu,Zhenwei Dai, et al. Position: Agentic evolution is the path to evolving llms.arXiv preprint arXiv:2602.00359, 2026 on PithPith's reviewarXiv
  18. [18] Minhua Lin, Zijun Wang, Zewen Liu, Zhaotian Weng, Bing He, Zhan Shi, Benoit Dumoulin, Xiang Zhang, Suhang Wang, and Hanqing Lu. Language models learn to improve language models.arXiv preprint arXiv:2610.23344, 2026. 14 on PithPith's reviewarXiv
  19. [19] Minhua Lin, Juncheng Wu, Zijun Wang, Zhan Shi, Yisi Sang, Bing He, Zewen Liu, Tianxin Wei, Zongyu Wu, Zhiwei Zhang, et al. Harness updating is not harness benefit: Disentangling evolution capabilities in self-evolving llm agents.arXiv preprint arXiv:2605.30621, 2026 on PithPith's reviewarXiv
  20. [20] Minhua Lin, Zhiwei Zhang, Hanqing Lu, Hui Liu, Xianfeng Tang, Qi He, Xiang Zhang, and Suhang Wang. Memma: Coordinating the memory cycle through multi-agent reasoning and in-situ self-evolution.arXiv preprint arXiv:2603.18718, 2026 on PithPith's reviewarXiv
show 35 more
  1. [21] Zewen Liu, Zhan Shi, Yisi Sang, Bing He, Minhua Lin, Tianxin Wei, Dakuo Wang, Benoit Dumoulin, Wei Jin, and Hanqing Lu. Adaptive auto-harness: Sustained self-improvement for agentic system deployment on open-ended task streams.arXiv preprint arXiv:2606.01770, 2026 on PithPith's reviewarXiv
  2. [22] YuyangLuo,HaoranWang,andKaiShu. Query-onlybackdoorattacksonself-evolvingskillsviatrajectorypoisoning. arXiv preprint arXiv:2608.08303, 2026 on PithPith's reviewarXiv
  3. [23] Zeyao Ma, Bohan Zhang, Jing Zhang, Jifan Yu, Xiaokang Zhang, Xiaohan Zhang, Sijia Luo, Xi Wang, and Jie Tang. Spreadsheetbench: Towards challenging real world spreadsheet manipulation.Advances in Neural Information Processing Systems, 37:94871–94908, 2024 on Pith
  4. [24] Jingwei Ni, Yihao Liu, Xinpeng Liu, Yutao Sun, Mengyu Zhou, Pengyu Cheng, Dexin Wang, Erchao Zhao, Xiaoxi Jiang, and Guanjun Jiang. Trace2skill: Distill trajectory-local lessons into transferable agent skills.arXiv preprint arXiv:2603.25158, 2026 on PithPith's reviewarXiv
  5. [25] Krista Opsahl-Ong, Arnav Singhvi, Jasmine Collins, Ivan Zhou, Cindy Wang, Ashutosh Baheti, Owen Oertell, Jacob Portes, Sam Havens, Erich Elsen, et al. Officeqa pro: An enterprise benchmark for end-to-end grounded reasoning. arXiv preprint arXiv:2603.08655, 2026 on PitharXiv
  6. [26] Panupong Pasupat and Percy Liang. Compositional semantic parsing on semi-structured tables. InProceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Paper... on Pith
  7. [27] Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. Toolllm: Facilitating large language models to master 16000+ real-world apis. InInternational Conference on Learning Representations, volume 2024, pages 9695... on Pith
  8. [28] Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving language understanding by generative pre-training. 2018 on Pith
  9. [29] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. 2019 on Pith
  10. [30] Rohan Rao and Vineeth Kalluru. Skillopt: Trajectory-derived, verifier-grounded compilation of llm-agent skills. In First Workshop on Agent Skills, 2026 on Pith
  11. [31] Samuel Schmidgall, Rojin Ziaei, Carl Harris, Eduardo Reis, Jeffrey Jopling, and Michael Moor. Agentclinic: a multimodal agent benchmark to evaluate ai in simulated clinical environments.arXiv preprint arXiv:2405.07960, 2024 on PithPith's reviewarXiv
  12. [32] Wenqi Shi, Ran Xu, Yuchen Zhuang, Yue Yu, Jieyu Zhang, Hang Wu, Yuanda Zhu, Joyce C Ho, Carl Yang, and May Dongmei Wang. Ehragent: Code empowers large language models for few-shot complex tabular reasoning on electronic health records. InProceedings of the 2024 Conference on E... on Pith
  13. [33] Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning.Advances in neural information processing systems, 36:8634–8652, 2023 on Pith
  14. [34] Xiangru Tang, Anni Zou, Zhuosheng Zhang, Ziming Li, Yilun Zhao, Xingyao Zhang, Arman Cohan, and Mark Gerstein. Medagents: Large language models as collaborators for zero-shot medical reasoning. InFindings of the Association for Computational Linguistics: ACL 2024, pages 599–621, 2024 on Pith
  15. [35] Guiyao Tie, Jiawen Shi, Pan Zhou, and Lichao Sun. Badskill: Backdoor attacks on agent skills via model-in-skill poisoning.arXiv preprint arXiv:2604.09378, 2026 on PithPith's reviewarXiv
  16. [36] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, SoumyaBatra, PrajjwalBhargava, ShrutiBhosale, etal. Llama2: Open foundation and fine-tunedchat models.arXiv preprint arXiv:2307.09288, 2023 on PithPith's reviewarXiv
  17. [37] GuanzhiWang,YuqiXie,YunfanJiang,AjayMandlekar,ChaoweiXiao,YukeZhu,LinxiFan,andAnimaAnandkumar. Voyager: An open-ended embodied agent with large language models.arXiv preprint arXiv:2305.16291, 2023. 15 on PithPith's reviewarXiv
  18. [38] Hanyu Wang, Yifan Lan, Bochuan Cao, Lu Lin, and Jinghui Chen. Skillgrad: Optimizing agent skills like gradient descent.arXiv preprint arXiv:2605.27760, 2026 on PithPith's reviewarXiv
  19. [39] Yifei Wang, Dizhan Xue, Shengjie Zhang, and Shengsheng Qian. Badagent: Inserting and activating backdoor attacks in llm agents. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9811–9827, 2024 on Pith
  20. [40] Zijun Wang, Zewen Liu, Minhua Lin, Zhaotian Weng, Zhan Shi, Bing He, Yisi Sang, Dakuo Wang, Benoit Dumoulin, Wei Jin, et al. Language models as ai research world models.arXiv preprint arXiv:2610.12235, 2026 on PithPith's reviewarXiv
  21. [41] Zora Zhiruo Wang, Jiayuan Mao, Daniel Fried, and Graham Neubig. Agent workflow memory. InForty-second International Conference on Machine Learning, 2025. URLhttps://openreview.net/forum?id=NTAhi2JEEE on Pith
  22. [42] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models.Advances in neural information processing systems, 35:24824–24837, 2022 on Pith
  23. [43] Tianxin Wei, Zhan Shi, Minhua Lin, Bing He, Zewen Liu, Yisi Sang, Yuanchen Bei, Xuying Ning, Jiaru Zou, Ting-Wei Li, et al. Evo-harness: Context-to-harness skill compilation for self-evolving agents.arXiv preprint arXiv:2608.15071, 2026 on PithPith's reviewarXiv
  24. [44] Wujiang Xu, Zujie Liang, Kai Mei, Hang Gao, Juntao Tan, and Yongfeng Zhang. A-mem: Agentic memory for llm agents.Advances in Neural Information Processing Systems, 38:17577–17604, 2026 on Pith
  25. [45] An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025 on PithPith's reviewarXiv
  26. [46] Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V Le, Denny Zhou, and Xinyun Chen. Large language models as optimizers. InInternational Conference on Learning Representations, volume 2024, pages 12028–12068, 2024 on Pith
  27. [47] Wenkai Yang, Xiaohan Bi, Yankai Lin, Sishuo Chen, Jie Zhou, and Xu Sun. Watch out for your agents! investigating backdoor threats to llm-based agents.Advances in Neural Information Processing Systems, 37:100938–100964, 2024 on Pith
  28. [48] Zonghao Ying, Xiangfan Wu, Huiyu Wu, Xing Zheng, Huangsheng Cheng, Xiaorong Shi, and Jing Guo. Skilljack: Persistent skill backdoors in self-evolving agents.arXiv preprint arXiv:2608.03509, 2026 on PithPith's reviewarXiv
  29. [49] Yangyang Yu, Haohang Li, Zhi Chen, Yuechen Jiang, Yang Li, Jordan W Suchow, Denghui Zhang, and Khaldoun Khashanah. Finmem: A performance-enhanced llm trading agent with layered memory and character design.IEEE Transactions on Big Data, 2025 on Pith
  30. [50] MertYuksekgonul, FedericoBianchi, JosephBoen, ShengLiu, ZhiHuang, CarlosGuestrin, andJamesZou. Textgrad: Automatic" differentiation" via text.arXiv preprint arXiv:2406.07496, 2024 on PithPith's reviewarXiv
  31. [51] Wentao Zhang, Lingxuan Zhao, Haochong Xia, Shuo Sun, Jiaze Sun, Molei Qin, Xinyi Li, Yuqing Zhao, Yilei Zhao, Xinyu Cai, et al. A multimodal foundation agent for financial trading: Tool-augmented, diversified, and generalist. InProceedings of the 30th acm sigkdd conference on ... on Pith
  32. [52] Zhiwei Zhang, Minhua Lin, Enyan Dai, and Suhang Wang. Rethinking graph backdoor attacks: A distribution- preserving perspective. InProceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining, pages 4386–4397, 2024 on Pith
  33. [53] Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. Expel: Llm agents are experiential learners. InProceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 19632–19642, 2024 on Pith
  34. [54] Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, et al. Webarena: A realistic web environment for building autonomous agents. InInternational Conference on Learning Representations, volume 2024, pag... on Pith
  35. [55] Tianyu Zhou, Pinqiao Wang, Yilin Wu, and Hongyang Yang. Finrobot: Ai agent for equity research and valuation with large language models.arXiv preprint arXiv:2411.08804, 2024. 16 Appendix A Additional Details of Related Works Self-Evolution of LLM Agents.Self-evolving LLM agent... on PithPith's reviewarXiv

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment