Pith. sign in

REVIEW 2 major objections 4 minor 30 references

One policy that solves tasks and rewrites a shared skill document learns transferable skills across related tasks, beating multi-stage pipelines and independent RL.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-30 21:11 UTC pith:NLPXQWX2

load-bearing objection Clean end-to-end recipe for cross-task skill RL with real gains and a coherent test-time scaling story; main limit is family-metadata sequence construction, already flagged. the 2 major comments →

arxiv 2607.26784 v1 pith:NLPXQWX2 submitted 2026-07-29 cs.LG cs.AI

SkillRise: Agentic Reinforcement Learning for Cross-Task Skill Evolution

Zhiyuan Yao , Yuxin Chen , Zhengxi Lu , Zishan Xu , Yueqing Sun , Yifu Guo , Yuquan Lu , Zhengzhou Cai
show 8 more authors
Kangning Zhang Zhuowen Han Zi-Han Wang Ziang Ye Qi Gu Xunliang Cai Weiwen Liu Yongliang Shen
This is my paper
classification cs.LG cs.AI
keywords agentic reinforcement learningcross-task skill learningskill document curationdecoupled credit assignmentLLM agentstest-time scalingALFWorldWebShop
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

LLM agents keep meeting related tasks that share solution patterns, yet ordinary agentic RL treats every task as a fresh episode and skill systems often bury extraction, retrieval, and execution in separate stages. SkillRise instead chains related instances from simple to hard and lets a single policy alternate between solving the current task with an evolving skill document and rewriting that document for the next task. Solving is trained on the immediate outcome; curation is trained on discounted later outcomes, so the document is judged by whether it actually helps downstream tasks. On ALFWorld, WebShop, and ScienceWorld this yields the strongest Pass@1 among the compared methods, still works when the same task is retried, and improves as related-task sequences get longer even when each task is tried only once—evidence that the agent is reusing transferable skills rather than just resampling one instance. The same end-to-end loop also cuts the runtime cost of multi-stage skill pipelines while keeping performance.

Core claim

Cross-task skill learning can be cast as a single reinforcement-learning problem over ordered sequences of related tasks: one policy both solves each task conditioned on a skill document and curates that document from the trajectory, with credit split so solving gets the current reward and curation gets discounted rewards from later tasks. That design produces stronger Pass@1 than independent RL and multi-stage skill methods, generalizes to within-task retries, and shows test-time gains as related sequences lengthen with one attempt per task.

What carries the argument

Decoupled cross-task credit assignment on an evolving skill document: after each solve, the same policy rewrites a single textual skill document that is the only channel to the next task; solving advantages use the current task reward, curation advantages use the discounted sum of subsequent task rewards, and group-relative advantages are computed within the same sequence position and phase.

Load-bearing premise

Related tasks must already be grouped by family metadata and ordered from simple to hard; without that hand-built relatedness the cross-task training signal may not appear.

What would settle it

On the same held-out ALFWorld, WebShop, and ScienceWorld splits, run SkillRise versus the strongest baseline with matched task plays: if Pass@1 gains disappear, or if lengthening related sequences with one attempt per task no longer raises success while same-task resampling does, the transfer claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Agents can improve on later related tasks by refining one shared skill document rather than storing and retrieving from a multi-stage memory bank.
  • Training on distinct related instances can still produce a curation policy that helps repeated attempts on one held-out task.
  • Longer related-task sequences at test time can raise success even when each task is attempted only once.
  • End-to-end solve-and-curate training can match or beat multi-stage skill pipelines at a fraction of the runtime.
  • Separating current-task credit from downstream credit is enough to jointly train execution and transferable skill writing in one policy.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If family labels are unavailable, the same loop would need an automatic relatedness or curriculum stage before the reported transfer can be expected in open task streams.
  • The skill document acts like a compressed, editable procedural memory whose only success metric is later-task reward, which may extend to other long-horizon agent settings with verifiable outcomes.
  • Larger backbones may amplify the method more than task-independent RL because capacity is spent jointly on execution and cross-task abstraction, as the 1.7B-to-4B gap already hints.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. SkillRise proposes an end-to-end agentic RL framework for cross-task skill learning. Related task instances from the same family are ordered into progressive sequences; a single policy alternates between solving the current task conditioned on an evolving skill document and curating that document for the next task. Decoupled credit assignment (Eq. 6) gives the current-task reward to solving and a discounted sum of later-task rewards to curation, with role- and position-aware group-relative advantages (Eqs. 7–8). On ALFWorld, WebShop, and ScienceWorld with Qwen3-1.7B/4B, SkillRise reports the best Pass@1 among prompting, standard RL (PPO/RLOO/GRPO/GiGPO), and LaMer baselines (gains of 2.3–8.5 pp over GiGPO), plus within-task Pass@k transfer of a cross-task-trained curator, cross-task test-time scaling with sequence length K when each task is attempted once (Fig. 2), γ robustness and a no-curation ablation (Fig. 3), and lower runtime than multi-stage skill pipelines (Fig. 4).

Significance. If the results hold, the paper offers a clean and practical alternative to both independent-episode agentic RL and multi-stage skill banks: one policy, one sequence-local document, and temporally aligned supervision. The matched task-play budget (384 plays/update), no-curation ablation, within-task generalization of a cross-task curator, and K-scaling without repeated sampling of the same instance are concrete strengths that support a transferable skill-learning mechanism rather than mere extra sampling. Public code is a further plus. The main external-validity limit—reliance on family metadata and hand-designed simple-to-hard ordering—is already stated in Limitations and does not erase the contribution within that scope. The work is a solid incremental advance for LLM agent skill evolution.

major comments (2)
  1. [§4.2, Table 1; §5.1, Figure 2] Tables 1–3 and Figs. 2–4 report point estimates only (no seeds, standard errors, or confidence intervals). The headline margins over GiGPO are 2.3 pp (ALFWorld), 7.1 pp (WebShop), and 8.5 pp (ScienceWorld). Without multi-seed variance, it is hard to judge whether the smaller ALFWorld gap and several per-family ranks (e.g., Clean/Heat where SkillRise is not first) are stable. At minimum, report mean±std over ≥3 seeds for the main Pass@1 comparison and for Fig. 2’s K-scaling curve, or justify single-run reporting with a clear limitation.
  2. [§3.1; Limitations; Abstract] Sequence construction (§3.1) uses environment family metadata and attribute-based simple-to-hard ordering. Evaluation sequences are likewise built from related held-out instances. The central claim is cross-task skill transfer under this construction; the Limitations section acknowledges automatic discovery as future work. To keep the claim precise, the abstract/intro should state more explicitly that gains are demonstrated under oracle (or metadata-given) relatedness and progressive ordering, not under arbitrary task streams. A small stress test—random within-family order, or shuffled/unrelated sequences—would show how much of the gain depends on the progressive curriculum versus any related-task grouping.
minor comments (4)
  1. [Table 1] In Table 1, SkillRise is not uniformly best per ALFWorld family (e.g., Clean 88.5 vs GiGPO 96.4; Heat 81.0 vs GiGPO 83.3) while leading on average. A short discussion of when curation helps or hurts would strengthen interpretation.
  2. [§4.1; §5.3] Implementation details fix K=3, N=8, γ=0.6, and 150 updates. Fig. 3 shows γ robustness; a brief note on sensitivity to K during training (not only test-time K) would help reproducibility.
  3. [§6] Related work cites Evolving-RL and SkillOS as concurrent/similar skill-evolution lines. A tighter paragraph contrasting single-policy end-to-end curation vs frozen executor + separate curator would clarify novelty relative to those works.
  4. [Title; Figure 1; Appendix A] Minor polish: “AGENTICREINFORCEMENTLEARNING” spacing in the title block; ensure Fig. 1 panel labels match the caption; define admissible-action formatting consistently across Appendix prompts.

Circularity Check

0 steps flagged

No significant circularity: external environment rewards and held-out Pass@1 evaluate an empirical RL method, not a self-defined quantity.

full rationale

SkillRise is an empirical agentic RL framework. Its load-bearing claims are comparative Pass@1 (and Pass@k) success rates on ALFWorld, WebShop, and ScienceWorld, measured by environment-provided outcome rewards R(τ) on held-out task instances. Decoupled credit assignment (Eq. 6) and role-aware group-relative advantages (Eqs. 7–8) are optimization design choices that assign existing external rewards to solve vs. curate phases; they do not redefine the evaluation target as a fitted internal constant. Sequence construction uses family metadata and simple-to-hard ordering as a training curriculum assumption (stated in §3.1 and Limitations), not as a quantity that is later reported as a derived prediction. Cross-task test-time scaling (Fig. 2) and the no-curation ablation compare the same external success metric under controlled protocol changes. There is no self-definitional loop, no fitted-input-called-prediction, no load-bearing uniqueness theorem imported from overlapping authors, and no renaming of a known closed-form result. Minor related-work self-citations are normal background and do not force the empirical claims. Score 0 is appropriate.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 2 invented entities

Load-bearing premises are standard RL/LLM-agent assumptions plus paper-specific design choices (family metadata sequences, textual skill document as sole channel, phase-decoupled returns, group-relative advantages). No physical constants; free parameters are training hyperparameters chosen by the authors.

free parameters (3)
  • cross-task discount γ = 0.6
    Weights how strongly later task rewards supervise curation (Eq. 6). Default 0.6; ablated in {0.3,0.4,0.6,0.7}.
  • sequence length K and trials N = K=3, N=8, batch=16 sequences
    Training uses K=3 tasks per sequence and N=8 trials; batch of 16 sequences. Chosen to match baseline interaction budget, not derived.
  • PPO-style clip ε and actor lr = lr=1e-6; ε as standard clip range (not numerically highlighted)
    Optimization hyperparameters for the clipped importance-ratio objective; lr given as 1e-6.
axioms (5)
  • domain assumption Related instances from the same environment task family share transferable interaction routines that a short textual document can carry.
    Stated in §3.1 as the reason sequences yield a skill-transfer learning signal.
  • ad hoc to paper Task-family metadata and attribute-based difficulty ordering are available to build progressive sequences.
    §3.1 construction and Limitations: open-ended discovery of relatedness is left future work.
  • domain assumption Verifiable outcome reward R(τ) is a sufficient training signal for both solving and curation quality.
    Preliminaries and method use environment success only; no process rewards.
  • domain assumption Group-relative advantages within (sequence position, phase) are stable enough for joint policy improvement of solve and curate heads of one model.
    §3.3 optimization follows GRPO-style comparisons without a learned critic.
  • standard math Standard clipped policy-gradient / importance-ratio RL on LLM tokens is a valid optimizer for multi-turn agent trajectories.
    Uses the usual clipped objective form (Eq. 8) common to GRPO-like LLM RL.
invented entities (2)
  • Evolving skill document S_i as sole cross-task information channel independent evidence
    purpose: Compress prior trajectories into text that conditions the next solve and receives downstream-return credit for curation.
    Central mechanism (§3.2); not a physical entity but a new training-state artifact relative to independent-episode RL.
  • Decoupled phase-level returns G_{i,solve} vs G_{i,curate} independent evidence
    purpose: Align supervision with temporal roles of solving vs skill transfer.
    Defined in Eq. 6; design choice rather than discovered natural object.

pith-pipeline@v1.2.0-daily-grok45 · 20454 in / 3452 out tokens · 62975 ms · 2026-07-30T21:11:33.395249+00:00 · methodology

0 comments
read the original abstract

Large language model agents often encounter related yet distinct tasks that share reusable solution patterns. Yet standard agentic reinforcement learning treats tasks as independent episodes, while existing approaches to skill learning either focus on repeated attempts of one task or use pipelines with multiple stages that entangle extraction, retrieval, and execution. We introduce SkillRise, a unified reinforcement learning framework for learning skills across tasks. SkillRise organizes related instances into progressively challenging sequences and uses a single policy to alternate between task solving and curating an evolving skill document passed directly to the next task. Decoupled credit assignment across tasks supervises solving with the current task outcome and curation with discounted downstream outcomes. Experiments on ALFWorld, WebShop, and ScienceWorld show that SkillRise achieves the strongest Pass@1 performance among the compared methods, with gains over the strongest baseline ranging from 2.3 to 8.5 percentage points. Although trained across distinct tasks, its learned curation policy remains effective for repeated attempts on the same task. Further analysis reveals scaling at test time across tasks: performance improves with longer sequences of related tasks even when each task is attempted only once. This trend suggests that SkillRise reuses transferable skills across tasks rather than benefiting from repeated sampling of the same task. SkillRise further retains strong performance while substantially reducing the runtime overhead of skill learning pipelines with multiple stages. Together, these results provide a simple and efficient training paradigm for LLM agents to extract, refine, and reuse transferable skills across tasks.

Figures

Figures reproduced from arXiv: 2607.26784 by Kangning Zhang, Qi Gu, Weiwen Liu, Xunliang Cai, Yifu Guo, Yongliang Shen, Yueqing Sun, Yuquan Lu, Yuxin Chen, Zhengxi Lu, Zhengzhou Cai, Zhiyuan Yao, Zhuowen Han, Ziang Ye, Zi-Han Wang, Zishan Xu.

Figure 1
Figure 1. Figure 1: Overview of SkillRise. (a) Related task instances are organized from simple to chal [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Cross-task test-time scaling on ALF [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Training dynamics on ALFWorld in terms of training reward (%). Thin lines show the raw [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Comparison of skill-learning pipelines on ALFWorld. Left: average success rate. Right: [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: SkillRise task-solving prompt for ALFWorld. [PITH_FULL_IMAGE:figures/full_fig_p013_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: SkillRise task-solving prompt for WebShop. [PITH_FULL_IMAGE:figures/full_fig_p013_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: SkillRise task-solving prompt for ScienceWorld. [PITH_FULL_IMAGE:figures/full_fig_p014_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: SkillRise skill-curation prompt for ALFWorld. [PITH_FULL_IMAGE:figures/full_fig_p014_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: SkillRise skill-curation prompt for WebShop. [PITH_FULL_IMAGE:figures/full_fig_p015_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: SkillRise skill-curation prompt for ScienceWorld. [PITH_FULL_IMAGE:figures/full_fig_p016_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Case 1: a rule recorded after failure. After a failure in which only one of two required objects is placed, the curation output records a repeat-for-each-instance rule. The following task is conditioned on this document and successfully places both instances of a different object. B CASE STUDY We show selected actions and abridged curation outputs from the first two tasks of two ALFWorld training rollouts… view at source ↗
Figure 12
Figure 12. Figure 12: Case 2: an incremental skill revision. After a successful task that requires opening the destination, curation adds an “open if closed” condition to the existing placement workflow. The added condition uses generic object and receptacle terms. Case 2: incremental skill evolution. Both displayed tasks succeed. Task 1 places an object on a destination that requires no open action, and skill v1 contains a ba… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

30 extracted references · 24 linked inside Pith

  1. [3]

    Yao Fu, Dong-Ki Kim, Jaekyeom Kim, Sungryull Sohn, Lajanugen Logeswaran, Kyunghoon Bae, and Honglak Lee

    URLhttps://arxiv.org/abs/2505.10978. Yao Fu, Dong-Ki Kim, Jaekyeom Kim, Sungryull Sohn, Lajanugen Logeswaran, Kyunghoon Bae, and Honglak Lee. Autoguide: Automated generation and selection of context-aware guide- lines for large language model agents.Advances in Neural Information Processing Systems, 37: 119919–119948,

    Pith/arXiv arXiv

  2. [5]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948,

    Pith/arXiv arXiv

  3. [6]

    Hierarchy-of-groups policy opti- mization for long-horizon agentic tasks.arXiv preprint arXiv:2602.22817,

    Shuo He, Lang Feng, Qi Wei, Xin Cheng, Lei Feng, and Bo An. Hierarchy-of-groups policy opti- mization for long-horizon agentic tasks.arXiv preprint arXiv:2602.22817,

    arXiv

  4. [7]

    Meta-rl induces explo- ration in language agents.arXiv preprint arXiv:2512.16848,

    Yulun Jiang, Liangze Jiang, Damien Teney, Michael Moor, and Maria Brbic. Meta-rl induces explo- ration in language agents.arXiv preprint arXiv:2512.16848,

    arXiv

  5. [8]

    Self-distilled agentic reinforcement learning.arXiv preprint arXiv:2605.15155, 2026a

    Zhengxi Lu, Zhiyuan Yao, Zhuowen Han, Zi-Han Wang, Jinyang Wu, Qi Gu, Xunliang Cai, Weim- ing Lu, Jun Xiao, Yueting Zhuang, et al. Self-distilled agentic reinforcement learning.arXiv preprint arXiv:2605.15155, 2026a. Zhengxi Lu, Zhiyuan Yao, Jinyang Wu, Chengcheng Han, Qi Gu, Xunliang Cai, Weiming Lu, Jun Xiao, Yueting Zhuang, and Yongliang Shen. Skill0: ...

    Pith/arXiv arXiv

  6. [9]

    Trace2skill: Distill trajectory-local lessons into transferable agent skills.arXiv preprint arXiv:2603.25158,

    Jingwei Ni, Yihao Liu, Xinpeng Liu, Yutao Sun, Mengyu Zhou, Pengyu Cheng, Dexin Wang, Er- chao Zhao, Xiaoxi Jiang, and Guanjun Jiang. Trace2skill: Distill trajectory-local lessons into transferable agent skills.arXiv preprint arXiv:2603.25158,

    Pith/arXiv arXiv

  7. [10]

    Siru Ouyang, Jun Yan, I Hsu, Yanfei Chen, Ke Jiang, Zifeng Wang, Rujun Han, Long T Le, Samira Daruki, Xiangru Tang, et al

    URLhttps://openai.com/index/ introducing-gpt-5/. Siru Ouyang, Jun Yan, I Hsu, Yanfei Chen, Ke Jiang, Zifeng Wang, Rujun Han, Long T Le, Samira Daruki, Xiangru Tang, et al. Reasoningbank: Scaling agent self-evolving with reasoning memory. arXiv preprint arXiv:2509.25140,

    Pith/arXiv arXiv

  8. [11]

    Skillos: Learning skill curation for self-evolving agents.arXiv preprint arXiv:2605.06614,

    Siru Ouyang, Jun Yan, Yanfei Chen, Rujun Han, Zifeng Wang, Bhavana Dalvi Mishra, Rui Meng, Chun-Liang Li, Yizhu Jiao, Kaiwen Zha, et al. Skillos: Learning skill curation for self-evolving agents.arXiv preprint arXiv:2605.06614,

    Pith/arXiv arXiv

  9. [12]

    Webrl: Training llm web agents via self-evolving online curriculum reinforcement learning

    10 Zehan Qi, Xiao Liu, Iat Long Iong, Hanyu Lai, Xueqiao Sun, Jiadai Sun, Xinyue Yang, Yu Yang, Shuntian Yao, Wei Xu, et al. Webrl: Training llm web agents via self-evolving online curriculum reinforcement learning. InInternational Conference on Learning Representations, volume 2025, pp. 79791–79821,

    2025

  10. [13]

    Autorefine: From trajectories to reusable expertise for continual llm agent refinement

    Libin Qiu, Zhirong Gao, Junfu Chen, Yuhang Ye, Weizhi Huang, Xiaobo Xue, Wenkai Qiu, and Shuo Tang. Autorefine: From trajectories to reusable expertise for continual llm agent refinement. arXiv preprint arXiv:2601.22758,

    arXiv

  11. [14]

    Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347,

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347,

    Pith/arXiv arXiv

  12. [16]

    Skill1: Unified evolution of skill-augmented agents via reinforcement learning

    Yaorui Shi, Yuxin Chen, Zhengxi Lu, Yuchun Miao, Shugui Liu, Qi Gu, Xunliang Cai, Xiang Wang, and An Zhang. Skill1: Unified evolution of skill-augmented agents via reinforcement learning. arXiv preprint arXiv:2605.06130,

    Pith/arXiv arXiv

  13. [20]

    Milestone-guided policy learning for long-horizon language agents.arXiv preprint arXiv:2605.06078, 2026b

    Zixuan Wang, Yuchen Yan, Hongxing Li, Teng Pan, Dingming Li, Ruiqing Zhang, Weiming Lu, Jun Xiao, Yueting Zhuang, and Yongliang Shen. Milestone-guided policy learning for long-horizon language agents.arXiv preprint arXiv:2605.06078, 2026b. Zora Zhiruo Wang, Jiayuan Mao, Daniel Fried, and Graham Neubig. Agent workflow memory. arXiv preprint arXiv:2409.07429,

    Pith/arXiv arXiv

  14. [21]

    Rong Wu, Xiaoman Wang, Jianbiao Mei, Pinlong Cai, Daocheng Fu, Cheng Yang, Licheng Wen, Xuemeng Yang, Yufan Shen, Yuxin Wang, et al

    URLhttps://arxiv.org/abs/ 2502.18449. Rong Wu, Xiaoman Wang, Jianbiao Mei, Pinlong Cai, Daocheng Fu, Cheng Yang, Licheng Wen, Xuemeng Yang, Yufan Shen, Yuxin Wang, et al. Evolver: Self-evolving llm agents through an experience-driven lifecycle.arXiv preprint arXiv:2510.16079,

    Pith/arXiv arXiv

  15. [22]

    Agentgym-rl: Training llm agents for long-horizon decision making through multi-turn reinforcement learning.arXiv preprint arXiv:2509.08755,

    Zhiheng Xi, Jixuan Huang, Chenyang Liao, Baodai Huang, Honglin Guo, Jiaqi Liu, Rui Zheng, Jun- jie Ye, Jiazheng Zhang, Wenxiang Chen, et al. Agentgym-rl: Training llm agents for long-horizon decision making through multi-turn reinforcement learning.arXiv preprint arXiv:2509.08755,

    Pith/arXiv arXiv

  16. [23]

    Skillrl: Evolving agents via recursive skill-augmented reinforcement learning.arXiv preprint arXiv:2602.08234,

    11 Peng Xia, Jianwen Chen, Hanyang Wang, Jiaqi Liu, Kaide Zeng, Yu Wang, Siwei Han, Yiyang Zhou, Xujiang Zhao, Haifeng Chen, et al. Skillrl: Evolving agents via recursive skill-augmented reinforcement learning.arXiv preprint arXiv:2602.08234,

    Pith/arXiv arXiv

  17. [24]

    Qwen3 technical report.arXiv preprint arXiv:2505.09388,

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388,

    Pith/arXiv arXiv

  18. [25]

    Skillopt: Executive strategy for self-evolving agent skills.arXiv preprint arXiv:2605.23904,

    Yifan Yang, Ziyang Gong, Weiquan Huang, Qihao Yang, Ziwei Zhou, Zisu Huang, Yan Li, Xuemei Gao, Qi Dai, Bei Liu, et al. Skillopt: Executive strategy for self-evolving agent skills.arXiv preprint arXiv:2605.23904,

    Pith/arXiv arXiv

  19. [26]

    Webshop: Towards scalable real-world web interaction with grounded language agents.Advances in Neural Information Pro- cessing Systems, 35:20744–20757, 2022a

    Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. Webshop: Towards scalable real-world web interaction with grounded language agents.Advances in Neural Information Pro- cessing Systems, 35:20744–20757, 2022a. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in lan...

    Pith/arXiv arXiv

  20. [27]

    Look before you leap: Autonomous exploration for llm agents.arXiv preprint arXiv:2605.16143,

    Ziang Ye, Wentao Shi, Yuxin Liu, Yu Wang, Zhengzhou Cai, Yaorui Shi, Qi Gu, Xunliang Cai, and Fuli Feng. Look before you leap: Autonomous exploration for llm agents.arXiv preprint arXiv:2605.16143,

    Pith/arXiv arXiv

  21. [28]

    The landscape of agentic reinforcement learning for llms: A survey.arXiv preprint arXiv:2509.02547,

    Guibin Zhang, Hejia Geng, Xiaohang Yu, Zhenfei Yin, Zaibin Zhang, Zelin Tan, Heng Zhou, Zhongzhi Li, Xiangyuan Xue, Yijiang Li, et al. The landscape of agentic reinforcement learning for llms: A survey.arXiv preprint arXiv:2509.02547,

    Pith/arXiv arXiv

  22. [29]

    Memskill: Learning and evolving memory skills for self-evolving agents.arXiv preprint arXiv:2602.02474, 2026a

    Haozhen Zhang, Quanyu Long, Jianzhu Bao, Tao Feng, Weizhi Zhang, Haodong Yue, and Wenya Wang. Memskill: Learning and evolving memory skills for self-evolving agents.arXiv preprint arXiv:2602.02474, 2026a. Xiaoying Zhang, Zichen Liu, Yipeng Zhang, Xia Hu, and Wenqi Shao. Retroagent: From solving to evolving via retrospective dual intrinsic feedback.arXiv p...

    Pith/arXiv arXiv

  23. [30]

    Lifelongagentbench: Evaluating llm agents as lifelong learners.arXiv preprint arXiv:2505.11942,

    Junhao Zheng, Xidi Cai, Qiuke Li, Duzhen Zhang, ZhongZhi Li, Yingying Zhang, Le Song, and Qianli Ma. Lifelongagentbench: Evaluating llm agents as lifelong learners.arXiv preprint arXiv:2505.11942,

    Pith/arXiv arXiv

  24. [2017]

    Deepseekmath: Pushing the limits of mathemati- cal reasoning in open language models.arXiv preprint arXiv:2402.03300,

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathemati- cal reasoning in open language models.arXiv preprint arXiv:2402.03300,

    Pith/arXiv arXiv

  25. [2020]

    Reinforcement learning for self-improving agent with skill library

    Jiongxiao Wang, Qiaojing Yan, Yawei Wang, Yijun Tian, Soumya Smruti Mishra, Zhichao Xu, Megha Gandhi, Panpan Xu, and Lin Lee Cheong. Reinforcement learning for self-improving agent with skill library. InProceedings of the 64th Annual Meeting of the Association for Com- putational Linguistics (Volume 1: Long Papers), pp. 1529–1550, 2026a. Ruoyao Wang, Pete...

    2022

  26. [2022]

    Ragen: Understanding self-evolution in llm agents via multi-turn reinforcement learning.arXiv preprint arXiv:2504.20073,

    Zihan Wang, Kangrui Wang, Qineng Wang, Pingyue Zhang, Linjie Li, Zhengyuan Yang, Xing Jin, Kefan Yu, Minh Nhat Nguyen, Licheng Liu, et al. Ragen: Understanding self-evolution in llm agents via multi-turn reinforcement learning.arXiv preprint arXiv:2504.20073,

    Pith/arXiv arXiv

  27. [2023]

    Alfworld: Aligning text and embodied environments for interactive learning.arXiv preprint arXiv:2010.03768,

    Mohit Shridhar, Xingdi Yuan, Marc-Alexandre C ˆot´e, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. Alfworld: Aligning text and embodied environments for interactive learning.arXiv preprint arXiv:2010.03768,

    Pith/arXiv arXiv 2010

  28. [2024]

    A survey of self-evolving agents: What, when, how, and where to evolve on the path to artificial super intelligence.arXiv preprint arXiv:2507.21046,

    Huan-ang Gao, Jiayi Geng, Wenyue Hua, Mengkang Hu, Xinzhe Juan, Hongzhang Liu, Shilong Liu, Jiahao Qiu, Xuan Qi, Yiran Wu, et al. A survey of self-evolving agents: What, when, how, and where to evolve on the path to artificial super intelligence.arXiv preprint arXiv:2507.21046,

    Pith/arXiv arXiv

  29. [2025]

    Yuxin Chen, Yu Wang, Yi Zhang, Ziang Ye, Zhengzhou Cai, Yaorui Shi, Qi Gu, Hui Su, Xunliang Cai, Xiang Wang, et al

    URLhttps://www.anthropic.com/ news/claude-sonnet-4-5. Yuxin Chen, Yu Wang, Yi Zhang, Ziang Ye, Zhengzhou Cai, Yaorui Shi, Qi Gu, Hui Su, Xunliang Cai, Xiang Wang, et al. Learning to self-verify makes language models better reasoners.arXiv preprint arXiv:2602.07594,

    arXiv

  30. [2026]

    Evolving- rl: End-to-end optimization of experience-driven self-evolving capability within agents.arXiv preprint arXiv:2605.10663,

    Zhiyuan Fan, Wenwei Jin, Feng Zhang, Bin Li, Yihong Dong, Yao Hu, and Jiawei Li. Evolving- rl: End-to-end optimization of experience-driven self-evolving capability within agents.arXiv preprint arXiv:2605.10663,

    Pith/arXiv arXiv