Pith. sign in

REVIEW 3 major objections 4 minor 67 references

Skill-α trains an agent to write agent skills by testing each edit on real downstream execution, and reports consistent gains over pipelines that hand-design skill generation.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 23:00 UTC pith:O5HAJLGT

load-bearing objection A real, clearly described new method for learning skill-editing policies, but the CL-Bench headline gains share a judge between training reward and final metric, and the paper doesn't address it. the 3 major comments →

arxiv 2608.01678 v1 pith:O5HAJLGT submitted 2026-08-03 cs.LG cs.CL

Progressive Agent Skill Generation via Reinforcement Learning

Junhao Shen , Zhanqiu Zhang , Yiwen Guo , Hong Cheng This is my paper
classification cs.LG cs.CL
keywords skill generationreinforcement learningLLM agentsprogressive editingrollback rewardGRPOdocument-to-skillexperience-to-skill
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that high-quality agent skills can be learned, not hand-designed or pipeline-compressed: it frames skill generation as a sequence of local edit decisions on a single SKILL.md artifact—create, update, merge, prune, or no-op—and trains the editor with a rollback reward that runs the worker on the same anchored query under the old and edited skill and rewards only edits that improve the downstream outcome. If right, one learned editing policy can generate reusable skills from documents and from execution experience alike, outperforming heuristic and pipeline baselines on held-out tasks. The reported effect is a 3.3-point average success-rate gain on CL-Bench and 6.7 points on tau2-bench over the strongest baselines with a GPT-4o worker, with transfer to another worker backbone.

Core claim

Skill-α's central claim is that the missing supervision signal for skill quality—which cannot be judged by relevance or factual correctness—can be replaced by an execution-grounded local comparison. The generator conditions on the current skill and a batch of evidence, samples an edit action from {CREATE, UPDATE, MERGE, PRUNE, NOOP}, applies it, then has the fixed worker answer the same anchored query under both the original and edited skill; a verifier decides which outcome is better, and GRPO maximizes the probability of edits whose edited rollout beats the control. The paper argues this turns skill generation from source-specific pipelines into a unified decision problem, and it reports t

What carries the argument

Rollback reward: for each local edit, the fixed worker executes the same anchored query once under the control skill and once under the edited skill; the edit is rewarded 1 only if the edited outcome is verifiably better than the control (or, for NOOP, only if no sampled edit beats the control). The paper shows that under independent branch rollouts this reward is a Bernoulli sample of the edited skill's pairwise win probability, and that under a calibrated verifier it preserves ideal-preference ordering and, for a binary verifier, exact success-probability ordering. Progressive editing is the companion mechanism: building the skill one evidence batch at a time gives each edit its own local

Load-bearing premise

That a single pairwise comparison of worker outcomes on one anchored query, using a verifier whose errors are assumed independent of the edit and calibrated enough to preserve ordering, is a sufficient training signal for local edits—and that locally rewarded edits compose into better final skills.

What would settle it

Train Skill-α exactly as described but replace the anchored-query verifier's rollback comparison with a random or deliberately miscalibrated signal; if downstream success rates stay near the reported levels, the rollback reward is not carrying the result. Alternatively, take a set of trained edits, measure whether the rollback reward's preference agrees with full held-out task success more often than chance on the same edits; agreement near chance would refute the central credit-assignment claim.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • A single trained editing policy can cover both document-to-skill and experience-to-skill, removing the need for evidence-source-specific pipeline designs.
  • Skill artifacts become optimizable: each edit has an execution-grounded reward, so skill quality can improve with more RL data rather than with better prompt engineering.
  • Gains transfer across worker backbones: skills built from GPT-4o trajectories also improve a Claude-Sonnet-4.5 worker.
  • The action space matters: removing MERGE/PRUNE or NOOP degrades results, so consolidation and abstention are functional parts of skill maintenance, not optional extras.
  • Evidence granularity is load-bearing: a moderate batch of four evidence units per step outperforms both smaller and larger batches, indicating progressive editing depends on controlling what the editor sees at each step.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If rollback reward is truly the binding constraint, then the method should transfer to any environment with a reliable, cheap verifier; benchmarks with weak or absent verifiers are the hard boundary, and future work could train on self-consistency or learned reward models.
  • The local pairwise signal suggests a possible scaling law: per-edit comparisons are expensive, so variance reduction—more anchors, better verifier calibration—may matter more than more RL steps.
  • The framework may apply beyond text SKILL.md skills—to multimodal or executable skill artifacts—provided the edit action space is extended and rollback remains executable.
  • A testable extension: use the same rollback reward to edit an agent's system prompt or tool-use policy, not just skills, since the mechanism is agnostic to artifact type.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. Skill-α proposes a reinforcement-learning framework for generating agent skills by formulating skill construction as a sequence of local edit decisions (CREATE, UPDATE, MERGE, PRUNE, NOOP). The skill-editing policy is trained with GRPO using a "rollback reward" that compares the fixed worker's outcome under the original skill against the edited skill on the same anchored query, with benchmark-specific verifiers. The method is evaluated in two settings: document-to-skill (CL-Bench) and experience-to-skill (SpreadsheetBench and tau2-bench), using GPT-4o as the same-worker backbone and Claude-Sonnet-4.5 as a transfer worker. The paper reports average downstream pass-rate improvements over the strongest baselines (+3.3 points on CL-Bench, +6.7 points on tau2-bench under GPT-4o), ablations supporting the role of rollback reward and progressive generation, and analyses of evidence order and batch size.

Significance. If the empirical claims hold, the paper makes a useful contribution: it unifies document-to-skill and experience-to-skill generation under a single learned editing policy, and it proposes an execution-grounded credit-assignment signal for local skill edits. The held-out evaluation protocol is carefully described, including the rule that no test query or test rubric is used for skill construction or reward design. The cross-worker transfer results are a plus, and the ablations (especially SFT-only and w/o rollback reward) are informative. The formal lemmas in Appendix A correctly identify the exact quantity estimated by the rollback comparison under the stated assumptions. However, the central empirical claim is currently supported by point estimates without variance, and for CL-Bench the training reward and the headline metric share the same GPT-5.5 rubric judge. These issues make the magnitude and even the direction of the claimed gains uncertain, especially given the small task counts behind several headline numbers.

major comments (3)
  1. [§4.2, §5.1, Appendix D.2] CL-Bench rollback reward and final evaluation use the same GPT-5.5 judge with the same official task-specific rubrics. §4.2 defines V_t for CL-Bench as a GPT-5.5 rubric judge, and §5.1/Appendix D.2 state that final CL-Bench pass/fail is also produced by GPT-5.5 under the same rubric protocol. The RL objective is therefore scored by the same verifier that defines the headline metric. Appendix A's Assumption A.3 assumes candidate-independent false-positive/false-negative rates, but the paper provides no evidence that the GPT-5.5 judge satisfies this assumption, nor any independent judge, human validation, or correlation check. If the judge has surface-level biases (formatting, rubric keywords, answer length), the policy can exploit them on anchored queries and the same biases are counted as held-out success. I request an independent-judge evaluation of the final skills on CL-Bench (e.g., h
  2. [§5.2, Tables 2–4, Appendix C] All headline numbers are single point estimates with no variance, no multiple seeds, and no significance tests. Several reported gains are very small counts: on SpreadsheetBench Skill-α is 27.5 vs. 26.0 for Skill-Creator (200 held-out tasks, i.e., 3 tasks), on tau2 Telecom 22.5 vs. 20.0 (40 held-out tasks, i.e., 1 task), and on tau2 Airline 65.0 vs. 55.0 (20 held-out tasks, i.e., 2 tasks); Retail is a tie at 80.0. The +6.7 average tau2 gain is driven by such counts. Given the noise inherent in LLM workers and the single anchored-query comparison admitted in Appendix E, these differences could easily be within run-to-run variation. Please report variance over independent seeds or task-level bootstrap intervals and, where possible, significance tests, for Tables 2–5.
  3. [Appendix A, Assumption A.3] The theoretical support for rollback reward is a conditional monotonicity result rather than an independent justification. Lemma A.4 assumes the verifier has candidate-independent error rates with η+ + η− < 1, and the conclusion E[R_i] > E[R_j] ⇔ p⋆_i > p⋆_j follows from the affine transform. Lemma A.5 assumes an exact binary verifier. No experiment checks whether the GPT-5.5 rubric judge or benchmark environment verifier satisfies these assumptions. Since the abstract claims that rollback reward 'evaluates each edit by comparing downstream execution,' the reader needs at least a calibration or correlation study on labeled pairs, or a clear statement that Assumption A.3 is an idealized condition whose violation does not undermine the empirical method.
minor comments (4)
  1. [§4] Several typos: 'It is a for progressive skill generation framework' and 'how decompose skill generation' should be corrected. Figure 1 also contains 'Controled' instead of 'Controlled'.
  2. [§5.4, Table 5] The evidence-order and batch-size analysis reports single-run numbers without variance. The claim that 'Skill-α is fairly robust to evidence order' and the batch-size interpretation would be more convincing with repeated runs or at least a clear statement that these are exploratory single-run observations.
  3. [§5.1, Appendix D.2] The main text should state explicitly in Section 5.1 that CL-Bench final evaluation uses the same GPT-5.5 judge as the rollback verifier; currently this coupling is only clear from Appendix D.2. This is not merely a presentation issue because it directly affects how readers interpret the CL-Bench results.
  4. [§5.3, Table 4] The claim that removing NOOP still leaves a 'relatively strong model' and that the full model is 'consistently better' is not supported without variance: the difference is 10.38 vs. 9.55 on CL-Bench, 27.50 vs. 22.00 on SpreadsheetBench, and 55.83 vs. 53.33 on tau2-bench. A single run cannot establish consistency.

Circularity Check

0 steps flagged

No definitional circularity; central claim is empirically grounded on held-out tasks and cross-worker transfer, though CL-Bench rollback reward and final evaluation share the same GPT-5.5 judge.

full rationale

The paper's derivation chain is: train a skill-editing policy with GRPO using rollback reward (Eq. 5), where the reward compares verifier outputs on an anchored query under control and edited skills; then evaluate the resulting skills on held-out tasks. Appendix A's lemmas are conditional identities (e.g., Lemma A.4 rewrites Assumption A.3; Lemma A.5 rewrites Assumption A.1 in the binary case). They are not circular because the paper does not use them to derive the empirical gains, and it explicitly states in Appendix E that a single rollback comparison is a noisy binary signal and that local updates are not guaranteed to improve the final skill. The headline empirical claim is tested on held-out tasks, with cross-worker transfer to Claude-Sonnet-4.5, and the SpreadsheetBench and tau2-bench results use direct benchmark-side environment feedback, which is independent of the GPT-5.5 rubric judge. The only notable coupling is on CL-Bench, where Appendix D.2 states that the GPT-5.5 judge is used both for final evaluation and for rollback comparison during RL; this creates a risk of optimizing the judge's surface preferences rather than genuine skill quality, but it is not a definitional equivalence because the anchored training queries and the held-out evaluation queries differ, and no fitted parameter is renamed as a prediction. Self-citations in Related Work are not load-bearing. No Eq. X is defined in terms of Eq. Y by construction. Score 2 reflects the CL-Bench judge coupling and the conditional nature of the Appendix A justification, without elevating them to circularity.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

The central claim rests on the reliability of the rollback reward and verifiers, plus the assumption that source and held-out tasks share a task family. The paper introduces no fitted constants in the sense of physics or econometrics, but the evidence batch size and anchor-selection threshold are hand-tuned choices that affect the reported numbers. The most fragile inputs are Assumptions A.1 and A.3, which are stated rather than validated.

free parameters (3)
  • Evidence batch size = 4 evidence units per step
    Main results use a batch of 4; Section 5.4 shows 1, 2, 8 all perform worse, so the headline number depends on a hand-tuned granularity.
  • Rollback anchor selection threshold = score below one or unsuccessful final status
    Experience benchmarks select anchored queries only from unsuccessful/partial trajectories, which shapes the reward distribution and is an unvalidated design choice.
  • GRPO group size G = 8
    Standard GRPO choice; the central claim is unlikely to hinge on it, but it is a hand-set hyperparameter in the training loop.
axioms (4)
  • domain assumption Assumption A.1: control and edited rollouts are independent and the skill condition is the only systematic difference.
    Underlies every expected-value lemma in Appendix A; violated in practice by stochastic LLM workers and shared prompt caches.
  • ad hoc to paper Assumption A.3: the verifier has candidate-independent false positive and false negative rates with eta+ + eta- < 1.
    Introduced to make rollback reward preserve ideal-preference ordering; no evidence is provided that GPT-5.5 rubric judging or environment feedback satisfies it.
  • domain assumption Task-family stationarity: source evidence x1:T and held-out target queries q are drawn from the same task family with a shared teacher distribution p*_M.
    Eq. 2 defines the ideal objective over this distribution; if held-out tasks differ systematically from source evidence, the learned editor may not transfer.
  • domain assumption Benchmark verifiers (GPT-5.5 rubrics, SpreadsheetBench/tau2 environment feedback) measure the intended downstream success and are not gameable by skill text.
    The entire reward and evaluation loop assumes verifier fidelity; on CL-Bench the same judge is used for training and final scoring, which weakens this assumption.

pith-pipeline@v1.3.0-daily-deepseek · 23213 in / 16479 out tokens · 146326 ms · 2026-08-04T23:00:39.797360+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Progressive Agent Skill Generation via Reinforcement Learning." pith.science (2026). https://pith.science/paper/O5HAJLGT

@misc{pith2026260801678,
  author       = {Pith},
  title        = {Pith review of: Progressive Agent Skill Generation via Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/O5HAJLGT}},
  note         = {Machine review of arXiv:2608.01678}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Existing skill generation methods largely rely on heuristics or pipeline-style consolidation, which must be specially designed for different evidence sources. In contrast, learning-based approaches offer a more unified way to model skill generation across heterogeneous sources. However, learning-based skill generation remains challenging because skills lack a natural supervision signal based on relevance or correctness; their value can largely be determined only by whether they improve the behavior of the agent on downstream tasks. To address this challenge, we propose Skill-$\alpha$, a reinforcement learning method for progressively generating high-quality agent skills. Specifically, we formulate skill generation as a sequential editing process that decomposes skill construction into individually evaluable edits, and introduce a novel rollback reward that evaluates each edit by comparing downstream execution under the original and edited skills on an anchored query. Extensive experiments show that Skill-$\alpha$ generates more effective skills than methods based on heuristics or pipelines in both document-to-skill and experience-to-skill settings. Under the main GPT-4o worker, Skill-$\alpha$ improves average downstream success rates over the strongest skill-generation baseline by 3.3 points on CL-Bench and 6.7 points on tau2-bench. Further ablations validate the importance of rollback reward and progressive generation.

Figures

Figures reproduced from arXiv: 2608.01678 by Hong Cheng, Junhao Shen, Yiwen Guo, Zhanqiu Zhang.

Figure 1
Figure 1. Figure 1: An overview of Skill-α. Left: during inference, Skill-α reads evidence sequentially and applies a sequence of local edit actions to progressively generate the skill from the initial state z0 to the final skill zT . Right: during training, the skill generator samples a group of candidate actions from the current skill state, constructs the corresponding edited skills, and evaluates them on the same evidence… view at source ↗
Figure 2
Figure 2. Figure 2: Training dynamics and rollout action distribution under the main ablations. Left: training [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

67 extracted references · 31 canonical work pages · 1 internal anchor

  1. [3]

    Frontiers of Computer Science , volume=

    A Survey on Large Language Model based Autonomous Agents , author=. Frontiers of Computer Science , volume=

    work page

  2. [4]

    Multi-Step Reasoning with Large Language Models, a Survey , journal =

    Aske Plaat and Annie Wong and Suzan Verberne and Joost Broekens and Niki van Stein and Thomas B. Multi-Step Reasoning with Large Language Models, a Survey , journal =

    work page

  3. [6]

    arXiv preprint arXiv:2509.09677 , year=

    The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs , author=. arXiv preprint arXiv:2509.09677 , year=

    arXiv

  4. [18]

    IEEE Transactions on Cognitive and Developmental Systems , year=

    Speci: Skill prompts based hierarchical continual imitation learning for robot manipulation , author=. IEEE Transactions on Cognitive and Developmental Systems , year=

    work page

  5. [19]

    Skill Creator , year =

    work page

  6. [20]

    2025 , month = oct, howpublished =

    Claude. 2025 , month = oct, howpublished =

    work page 2025

  7. [23]

    arXiv preprint arXiv:2605.13527 , year=

    MMSkills: Towards Multimodal Skills for General Visual Agents , author=. arXiv preprint arXiv:2605.13527 , year=

    Pith/arXiv arXiv

  8. [24]

    Proceedings of the 42nd International Conference on Machine Learning , year =

    Zora Zhiruo Wang and Jiayuan Mao and Daniel Fried and Graham Neubig , title =. Proceedings of the 42nd International Conference on Machine Learning , year =

    work page

  9. [25]

    Proceedings of the 43rd International Conference on Machine Learning , year =

    Victor Barres and Honghua Dong and Soham Ray and Xujie Si and Karthik Narasimhan , title =. Proceedings of the 43rd International Conference on Machine Learning , year =

    work page

  10. [26]

    Advances in Neural Information Processing Systems 37 , address=

    Spreadsheetbench: Towards challenging real world spreadsheet manipulation , author=. Advances in Neural Information Processing Systems 37 , address=

    work page

  11. [28]

    Proceedings of the 38th AAAI Conference on Artificial Intelligence , pages=

    ExpeL: LLM Agents Are Experiential Learners , author=. Proceedings of the 38th AAAI Conference on Artificial Intelligence , pages=. 2024 , address=

    work page 2024

  12. [32]

    2026 , journal=

    From Raw Experience to Skill Consumption: A Systematic Study of Model-Generated Agent Skills , author=. 2026 , journal=

    work page 2026

  13. [36]

    Nature , volume=

    DeepSeek-R1 Incentivizes Reasoning in LLMs through Reinforcement Learning , author=. Nature , volume=

    work page

  14. [38]

    Advances in Neural Information Processing Systems 38 , address=

    Semi-off-policy Reinforcement Learning for Vision-language Slow-thinking Reasoning , author=. Advances in Neural Information Processing Systems 38 , address=

    work page

  15. [39]

    Proceedings of the 14th International Conference on Learning Representations , address=

    Achieving Olympia-Level Geometry Large Language Model Agent via Complexity Boosting Reinforcement Learning , author=. Proceedings of the 14th International Conference on Learning Representations , address=

    work page

  16. [49]

    OpenAI , url =. Hello

    work page

  17. [50]

    Introducing Claude Sonnet 4.5 , author=

    work page

  18. [51]

    Introducing

    OpenAI , url =. Introducing

  19. [52]

    Claude S kills

    Anthropic . Claude S kills. https://claude.com/blog/skills, October 2025. Accessed: 2026-05-27

    work page 2025

  20. [53]

    Introducing claude sonnet 4.5, 2025

    Anthropic. Introducing claude sonnet 4.5, 2025. URL https://www.anthropic.com/news/claude-sonnet-4-5

    work page 2025

  21. [54]

    Skill creator

    Anthropic . Skill creator. https://github.com/anthropics/skills/blob/main/skills/skill-creator/SKILL.md, March 2026. Accessed: 2026-05-27

    work page 2026

  22. [55]

    ^2 -bench: Evaluating conversational agents in a dual-control environment

    Victor Barres, Honghua Dong, Soham Ray, Xujie Si, and Karthik Narasimhan. ^2 -bench: Evaluating conversational agents in a dual-control environment. In Proceedings of the 43rd International Conference on Machine Learning, Seoul, South Korea, 2026

    work page 2026

  23. [56]

    Agent-r1: Training powerful llm agents with end-to-end reinforcement learning

    Ming Cheng, Jinhua Ouyang, Shulin Yu, Ruochen Yan, Yuchen Luo, Zihan Liu, and Enhong Chen. Agent-r1: Training powerful llm agents with end-to-end reinforcement learning. arXiv preprint arXiv:2511.14460, 2025

    Pith/arXiv arXiv 2025

  24. [57]

    DeepSeek-AI , Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai ...

    work page 2025

  25. [58]

    Deepseek-v4: Towards highly efficient million-token context intelligence

    DeepSeek-AI, Anyi Xu, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, Chenchen Ling, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chengyu Hou, Chenhao Xu, Chenze Shao, Chong Ruan, Conner Sun, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Donghao Li, Dongjie Ji, Erhang Li, Fang Wei, Fangyun Lin, Fangzhou Yuan...

    arXiv 2026

  26. [59]

    Cl-bench: A benchmark for context learning

    Shihan Dou, Ming Zhang, Zhangyue Yin, Chenhao Huang, Yujiong Shen, Junzhe Wang, Jiayi Chen, Yuchen Ni, Junjie Ye, Cheng Zhang, et al. Cl-bench: A benchmark for context learning. arXiv preprint arXiv:2602.03587, 2026

    arXiv 2026

  27. [60]

    Understanding the planning of llm agents: A survey

    Xu Huang, Weiwen Liu, Xiaolong Chen, Xingmei Wang, Hao Wang, Defu Lian, Yasheng Wang, Ruiming Tang, and Enhong Chen. Understanding the planning of llm agents: A survey. arXiv preprint arXiv:2402.02716, 2024

    Pith/arXiv arXiv 2024

  28. [61]

    From raw experience to skill consumption: A systematic study of model-generated agent skills

    Zisu Huang, Jingwen Xu, Yifan Yang, Ziyang Gong, Qihao Yang, Muzhao Tian, Xiaohua Wang, Changze Lv, Xuemei Gao, Qi Dai, Bei Liu, Kai Qiu, Xue Yang, Dongdong Chen, Xiaoqing Zheng, and Chong Luo. From raw experience to skill consumption: A systematic study of model-generated agent skills. arXiv preprint arXiv:2605.23899, 2026

    Pith/arXiv arXiv 2026

  29. [62]

    Search-r1: Training llms to reason and leverage search engines with reinforcement learning

    Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-r1: Training llms to reason and leverage search engines with reinforcement learning. arXiv preprint arXiv:2503.09516, 2025

    Pith/arXiv arXiv 2025

  30. [63]

    Skillsbench: Benchmarking how well agent skills work across diverse tasks

    Xiangyi Li, Yimin Liu, Wenbo Chen, Shenghan Zheng, Xiaokun Chen, Yifeng He, Yubo Li, Bingran You, Haotian Shen, Jiankai Sun, Shuyi Wang, Binxu Li, Qunhong Zeng, Di Wang, Xuandong Zhao, Yuanli Wang, Roey Ben Chaim, Zonglin Di, Yipeng Gao, Junwei He, Yizhuo He, Liqiang Jing, Luyang Kong, Xin Lan, Jiachen Li, Songlin Li, Yijiang Li, Yueqian Lin, Xinyi Liu, X...

    Pith/arXiv arXiv 2026

  31. [64]

    Torl: Scaling tool-integrated rl

    Xuefeng Li, Haoyang Zou, and Pengfei Liu. Torl: Scaling tool-integrated rl. arXiv preprint arXiv:2503.23383, 2025

    Pith/arXiv arXiv 2025

  32. [65]

    Graph of skills: Dependency-aware structural retrieval for massive agent skills

    Dawei Liu, Zongxia Li, Hongyang Du, Xiyang Wu, Shihang Gui, Yongbei Kuang, and Lichao Sun. Graph of skills: Dependency-aware structural retrieval for massive agent skills. arXiv preprint arXiv:2604.05333, 2026

    Pith/arXiv arXiv 2026

  33. [66]

    Yu, and Ming Zhang

    Junyu Luo, Weizhi Zhang, Ye Yuan, Yusheng Zhao, Junwei Yang, Yiyang Gu, Bohan Wu, Binqi Chen, Ziyue Qiao, Qingqing Long, Rongcheng Tu, Xiao Luo, Wei Ju, Zhiping Xiao, Yifan Wang, Meng Xiao, Chenwu Liu, Jingyang Yuan, Shichang Zhang, Yiqiao Jin, Fan Zhang, Xian Wu, Hanqing Zhao, Dacheng Tao, Philip S. Yu, and Ming Zhang. Large language model agent: A surve...

    Pith/arXiv arXiv 2025

  34. [67]

    Spreadsheetbench: Towards challenging real world spreadsheet manipulation

    Zeyao Ma, Bohan Zhang, Jing Zhang, Jifan Yu, Xiaokang Zhang, Xiaohan Zhang, Sijia Luo, Xi Wang, and Jie Tang. Spreadsheetbench: Towards challenging real world spreadsheet manipulation. In Advances in Neural Information Processing Systems 37, pp.\ 94871--94908, Vancouver, Canada, 2024

    work page 2024

  35. [68]

    Skill-pro: Learning reusable skills from experience via non-parametric ppo for llm agents

    Qirui Mi, Zhijian Ma, Mengyue Yang, Haoxuan Li, Yisen Wang, Haifeng Zhang, and Jun Wang. Skill-pro: Learning reusable skills from experience via non-parametric ppo for llm agents. arXiv preprint arXiv:2602.01869, 2026

    Pith/arXiv arXiv 2026

  36. [69]

    Trace2skill: Distill trajectory-local lessons into transferable agent skills

    Jingwei Ni, Yihao Liu, Xinpeng Liu, Yutao Sun, Mengyu Zhou, Pengyu Cheng, Dexin Wang, Erchao Zhao, Xiaoxi Jiang, and Guanjun Jiang. Trace2skill: Distill trajectory-local lessons into transferable agent skills. arXiv preprint arXiv:2603.25158, 2026

    Pith/arXiv arXiv 2026

  37. [70]

    Hello GPT-4o , 2024

    OpenAI. Hello GPT-4o , 2024. URL https://openai.com/index/hello-gpt-4o/

    work page 2024

  38. [71]

    Introducing GPT‑5.5 , 2026

    OpenAI. Introducing GPT‑5.5 , 2026. URL https://openai.com/index/introducing-gpt-5-5/

    work page 2026

  39. [72]

    Skillos: Learning skill curation for self-evolving agents

    Siru Ouyang, Jun Yan, Yanfei Chen, Rujun Han, Zifeng Wang, Bhavana Dalvi Mishra, Rui Meng, Chun-Liang Li, Yizhu Jiao, Kaiwen Zha, et al. Skillos: Learning skill curation for self-evolving agents. arXiv preprint arXiv:2605.06614, 2026

    Pith/arXiv arXiv 2026

  40. [73]

    Multi-step reasoning with large language models, a survey

    Aske Plaat, Annie Wong, Suzan Verberne, Joost Broekens, Niki van Stein, and Thomas B \" a ck. Multi-step reasoning with large language models, a survey. ACM Computing Surveys , 58 0 (6): 0 160:1--160:35, 2026

    work page 2026

  41. [74]

    Webrl: Training llm web agents via self-evolving online curriculum reinforcement learning

    Zehan Qi, Xiao Liu, Iat Long Iong, Hanyu Lai, Xueqiao Sun, Wenyi Zhao, Yu Yang, Xinyue Yang, Jiadai Sun, Shuntian Yao, et al. Webrl: Training llm web agents via self-evolving online curriculum reinforcement learning. arXiv preprint arXiv:2411.02337, 2024

    Pith/arXiv arXiv 2024

  42. [75]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    Pith/arXiv arXiv 2017

  43. [76]

    Semi-off-policy reinforcement learning for vision-language slow-thinking reasoning

    Junhao Shen, Haiteng Zhao, Yuzhe Gu, Songyang Gao, Kuikun Liu, Haian Huang, Jianfei Gao, Dahua Lin, Wenwei Zhang, and Kai Chen. Semi-off-policy reinforcement learning for vision-language slow-thinking reasoning. In Advances in Neural Information Processing Systems 38, San Diego, CA, 2025

    work page 2025

  44. [77]

    Dynamic skill lifecycle management for agentic reinforcement learning

    Junhao Shen, Teng Zhang, Xiaoyan Zhao, and Hong Cheng. Dynamic skill lifecycle management for agentic reinforcement learning. arXiv preprint arXiv:2605.10923, 2026

    Pith/arXiv arXiv 2026

  45. [78]

    Llm with tools: A survey

    Zhuocheng Shen. Llm with tools: A survey. arXiv preprint arXiv:2409.18807, 2024

    Pith/arXiv arXiv 2024

  46. [79]

    From context to skills: Can language models learn from context skillfully? arXiv preprint arXiv:2604.27660, 2026

    Shuzheng Si, Haozhe Zhao, Yu Lei, Qingyi Wang, Dingwei Chen, Zhitong Wang, Zhenhailong Wang, Kangyang Luo, Zheng Wang, Gang Chen, et al. From context to skills: Can language models learn from context skillfully? arXiv preprint arXiv:2604.27660, 2026

    Pith/arXiv arXiv 2026

  47. [80]

    Agentic reasoning and tool integration for llms via reinforcement learning

    Joykirat Singh, Raghav Magazine, Yash Pandya, and Akshay Nambi. Agentic reasoning and tool integration for llms via reinforcement learning. arXiv preprint arXiv:2505.01441, 2025

    Pith/arXiv arXiv 2025

  48. [81]

    Skillx: Automatically constructing skill knowledge bases for agents

    Chenxi Wang, Zhuoyun Yu, Xin Xie, Wuguannan Yao, Runnan Fang, Shuofei Qiao, Kexin Cao, Guozhou Zheng, Xiang Qi, Peng Zhang, and Shumin Deng. Skillx: Automatically constructing skill knowledge bases for agents. arXiv preprint arXiv:2604.04804, 2026 a

    Pith/arXiv arXiv 2026

  49. [82]

    Voyager: An open-ended embodied agent with large language models

    Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291, 2023

    Pith/arXiv arXiv 2023

  50. [83]

    Skill-sd: Skill-conditioned self-distillation for multi-turn llm agents

    Hao Wang, Guozhi Wang, Han Xiao, Yufeng Zhou, Yue Pan, Jichao Wang, Ke Xu, Yafei Wen, Xiaohu Ruan, Xiaoxin Chen, et al. Skill-sd: Skill-conditioned self-distillation for multi-turn llm agents. arXiv preprint arXiv:2604.10674, 2026 b

    Pith/arXiv arXiv 2026

  51. [84]

    Reinforcement learning for self-improving agent with skill library

    Jiongxiao Wang, Qiaojing Yan, Yawei Wang, Yijun Tian, Soumya Smruti Mishra, Zhichao Xu, Megha Gandhi, Panpan Xu, and Lin Lee Cheong. Reinforcement learning for self-improving agent with skill library. arXiv preprint arXiv:2512.17102, 2025 a

    Pith/arXiv arXiv 2025

  52. [85]

    A survey on large language model based autonomous agents

    Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, and Ji-Rong Wen. A survey on large language model based autonomous agents. Frontiers of Computer Science, 18 0 (6): 0 186345, 2024

    work page 2024

  53. [86]

    Mem- : Learning memory construction via reinforcement learning

    Yu Wang, Ryuichi Takanobu, Zhiqi Liang, Yuzhen Mao, Yuanzhe Hu, Julian McAuley, and Xiaojian Wu. Mem- : Learning memory construction via reinforcement learning. arXiv preprint arXiv:2509.25911, 2025 b

    Pith/arXiv arXiv 2025

  54. [87]

    Webxskill: Skill learning for autonomous web agents

    Zhaoyang Wang, Qianhui Wu, Xuchao Zhang, Chaoyun Zhang, Wenlin Yao, Fazle Elahi Faisal, Baolin Peng, Si Qin, Suman Nath, Qingwei Lin, et al. Webxskill: Skill learning for autonomous web agents. arXiv preprint arXiv:2604.13318, 2026 c

    Pith/arXiv arXiv 2026

  55. [88]

    Effiskill: Agent skill based automated code efficiency optimization

    Zimu Wang, Yuling Shi, Mengfan Li, Zijun Liu, Jie M Zhang, Chengcheng Wan, and Xiaodong Gu. Effiskill: Agent skill based automated code efficiency optimization. arXiv preprint arXiv:2603.27850, 2026 d

    arXiv 2026

  56. [89]

    Agent workflow memory

    Zora Zhiruo Wang, Jiayuan Mao, Daniel Fried, and Graham Neubig. Agent workflow memory. In Proceedings of the 42nd International Conference on Machine Learning, Vancouver, Canada, 2025 c

    work page 2025

  57. [90]

    Skillrl: Evolving agents via recursive skill-augmented reinforcement learning

    Peng Xia, Jing Chen, Hao Wang, Jia Liu, Kai Zeng, Yixiao Wang, and Haoran Yao. Skillrl: Evolving agents via recursive skill-augmented reinforcement learning. arXiv preprint arXiv:2602.08234, 2026

    Pith/arXiv arXiv 2026

  58. [91]

    Speci: Skill prompts based hierarchical continual imitation learning for robot manipulation

    Jingkai Xu and Xiangli Nie. Speci: Skill prompts based hierarchical continual imitation learning for robot manipulation. IEEE Transactions on Cognitive and Developmental Systems, 2025

    work page 2025

  59. [92]

    Memory-r1: Enhancing large language model agents to manage and utilize memories via reinforcement learning

    Sikuan Yan, Xiufeng Yang, Zuchao Huang, Ercong Nie, Zifeng Ding, Zonggen Li, Xiaowen Ma, Jinhe Bi, Kristian Kersting, Jeff Z Pan, et al. Memory-r1: Enhancing large language model agents to manage and utilize memories via reinforcement learning. arXiv preprint arXiv:2508.19828, 2025

    Pith/arXiv arXiv 2025

  60. [93]

    Qwen3 technical report

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, ...

    Pith/arXiv arXiv 2025

  61. [94]

    Autoskill: Experience-driven lifelong learning via skill self-evolution

    Yutao Yang, Junsong Li, Qianjun Pan, Bihao Zhan, Yuxuan Cai, Lin Du, Jie Zhou, Kai Chen, Qin Chen, Xin Li, Bo Zhang, and Liang He. Autoskill: Experience-driven lifelong learning via skill self-evolution. arXiv preprint arXiv:2603.01145, 2026

    arXiv 2026

  62. [95]

    Dapo: An open-source llm reinforcement learning system at scale

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wang, Hongli Yu, Yuxuan Song, Xiangpeng Wei, Hao Zhou, Jingjing Liu, W...

    Pith/arXiv arXiv 2025

  63. [96]

    Group of Skills: Group-Structured Skill Retrieval for Agent Skill Libraries

    Kun Zeng, Yu Huo, Siyu Zhang, Zi Ye, Yuecheng Zhuo, Haoyue Liu, Yuquan Lu, Junhao Wen, and Xiaoying Tang. Group of skills: Group-structured skill retrieval for agent skill libraries. arXiv preprint arXiv:2605.06978, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  64. [97]

    Expel: Llm agents are experiential learners

    Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. Expel: Llm agents are experiential learners. In Proceedings of the 38th AAAI Conference on Artificial Intelligence, pp.\ 19632--19642, Vancouver, Canada, 2024

    work page 2024

  65. [98]

    Achieving olympia-level geometry large language model agent via complexity boosting reinforcement learning

    Haiteng Zhao, Junhao Shen, Yiming Zhang, Songyang Gao, Kuikun Liu, Tianyou Ma, Fan Zheng, Dahua Lin, Wenwei Zhang, and Kai Chen. Achieving olympia-level geometry large language model agent via complexity boosting reinforcement learning. In Proceedings of the 14th International Conference on Learning Representations, Rio de Janeiro, Brazil, 2026

    work page 2026

  66. [99]

    Skillrouter: Skill routing for llm agents at scale

    YanZhao Zheng, ZhenTao Zhang, Chao Ma, YuanQiang Yu, JiHuai Zhu, Yong Wu, Tianze Xu, Baohua Dong, Hangcheng Zhu, Ruohui Huang, et al. Skillrouter: Skill routing for llm agents at scale. arXiv preprint arXiv:2603.22455, 2026

    Pith/arXiv arXiv 2026

  67. [100]

    Skillgenbench: Benchmarking skill generation pipelines for llm agents

    Yifan Zhou, Zhentao Zhang, Ziming Cheng, Shuo Zhang, Qizhen Lan, Zhangquan Chen, Zhi Yang, Ronghao Chen, Huacan Wang, Sen Hu, et al. Skillgenbench: Benchmarking skill generation pipelines for llm agents. arXiv preprint arXiv:2605.18693, 2026

    Pith/arXiv arXiv 2026