Artificial Intelligence · arXiv:2610.17860 v1 · submitted 13 Oct 2026 · 51 references
Socratic Skill Learning: Interleaving Teaching and Practice for Transferable Agent Skills
The paper says
“Agent Skills improve LLM agents by making task knowledge reusable across sessions. Learning transferable Skills requires evidence about a procedure and its conditions of use, which a successful task attempt can leave untested.”
more of the abstract
We introduce Socratic Skill Learning (SSL), which interleaves expert teaching with Student practice. An Educator guides investigation while the Student remains the sole environment operator and Skill author, able to question, test, and adapt advice. To test reuse beyond the learning task, we construct SkillTransferBench, with 35 task families and 408 executable cases, pairing public variation specifications with held-out evaluation in fresh solver sessions. On the main 22-family, 155-case cohort, Fixed-Window SSL reaches 0.801 utility, versus 0.723 without a Skill, 0.761 for adapted Trace2Skill (trajectory distillation), and 0.751 for adapted SKILL-KD (contrastive distillation). Across three model pairs on shared families, interleaved guidance outperforms expert authoring and post-hoc teaching. Process analysis links guidance to Student writing and package revision; a case study shows the Student testing and refining a proposed procedure for later reuse. As demand for new lessons falls, the cost of fixed-window review grows. Issue-Guided SSL tracks unresolved questions across Skill revisions to select and focus review, reaching 0.805 utility with 61.7% fewer Educator tokens and 16.9% fewer total acquisition tokens than Fixed-Window SSL on the main cohort.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 14 Oct 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
The passes disagreed. Reader: holds up. Skeptic: agrees with the reader. Referee: major revision.
Solid paper: interleaved teaching clearly beats post-hoc methods for agent skill transfer, but the efficiency claim (Issue-Guided SSL preserves utility with fewer tokens) lacks a reported confidence interval for the utility difference, which the authors should fix. This line is wrong
2 major objections · 3 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 51 references say.
The paper says, at §5.4, Table 17 and Figure 5
The review did not quote this passage.
Pith says: Objection 1 of 2 · major
The central efficiency claim (RQ4) that Issue-Guided SSL preserves transfer utility while reducing Educator tokens rests on a utility comparison of 0.805 versus 0.801 for Fixed-Window SSL (g=10). This 0.004 difference is reported without a confidence interval or statistical test, unlike the headline comparisons in Table 8 (e.g., Fixed-Window SSL vs.
more of objection 1
Trace2Skill, which has a 95% CI of [1.45, 6.49]). The bootstrap procedure used elsewhere (family resampling) could straightforwardly be applied here. Without uncertainty quantification, the reader cannot assess whether the utility is statistically unchanged (supporting 'preserves transfer') or whether there is a meaningful (possibly negative) shift. The paper's strongest cost-efficiency argument depends on this being a true preservation. I request that the authors compute and report the 95% CI (or an equivalent test) for the Issue-Guided SSL versus Fixed-Window SSL comparison on the main cohort.
The paper says, at §5.2, Table 13
The review did not quote this passage.
Pith says: Objection 2 of 2 · major
The headline claim that 'interleaved guidance outperforms expert authoring and post-hoc teaching' is supported on the main Qwen cohort with narrow CIs. However, on the shared 10-family subset for DeepSeek and Kimi, the 95% paired family-bootstrap intervals for Fixed-Window SSL versus Expert Authoring include zero (DeepSeek: [-1.93, 7.15]; Kimi: [-2.33, 7.98]).
more of objection 2
The intervals for Issue-Guided SSL versus Fixed-Window SSL also include zero across all three pairs. The paper should explicitly discuss the limited statistical power of these cross-model comparisons, particularly for the lite subset, and temper the claim accordingly.
3 minor objections
The paper says, at Abstract and §5.4
Pith says: minor objection 1 of 3
The abstract states 'Issue-Guided SSL reaches 0.805 utility with 61.7% fewer Educator tokens and 16.9% fewer total acquisition tokens than Fixed-Window SSL on the main cohort.' The 16.9% acquisition-token reduction is specific to Qwen; Table 18 shows that for DeepSeek, total acquisition tokens increase by 1.101M per run. Please qualify the token-savings claim as model-specific, or state the qualification already present in the supplementary cost analysis.
The paper says, at §5.3, Figure 4 caption
Pith says: minor objection 2 of 3
The caption says 'Panels show (a) anchor scores and teaching rates at regular reviews, (b) teaching–silence differences ...' but the panels are labelled 1, 2, 3 in the figure, not a, b, c. Ensure the caption matches the sub-figure labels.
show 1 more
The paper says, at §3.1, Eq. (1)
Pith says: minor objection 3 of 3
The definition of transfer utility U_f uses an expectation over D_f, but the evaluation (Eq. 2) averages over a fixed set of held-out cases with equal weight. Clarify that D_f is the empirical distribution over the available test cases.
What would settle it
Compute the 95% paired family-bootstrap confidence interval for the difference in family-macro Avg@3 utility between Issue-Guided SSL and Fixed-Window SSL (g=10) on the Qwen 22-family main cohort, using the same resampling procedure as in Table 8 (resample whole families with replacement, 10,000 repetitions). Report the interval and the proportion of resamples where Issue-Guided SSL is worse than Fixed-Window SSL. If the interval contains zero and the one-tailed p-value for superiority is >0.05, the claim of preserved utility is supported.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 14 Oct 2026
- Reader holds up
- Skeptic partly agrees
- Editor wrote the verdict above
- Referee major revision: 2 major objections, 3 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
This paper makes a real contribution. The Socratic Skill Learning framework—especially the idea that an Educator guides during student-led practice rather than after—is well-motivated and backed by clear empirical work on three model pairs. SkillTransferBench is a useful new resource with 35 families and held-out evaluation. The main result is solid: Fixed-Window SSL beats Trace2Skill and SKILL-KD on the 22-family cohort with a statistically significant 3.99-point gain (CI [1.45, 6.49]). The process analysis linking lessons to file edits and package revisions is a nice addition, and the code and data release is thorough.
Now the soft spot, which the stress-test note nails. RQ4 claims that Issue-Guided SSL reaches 0.805 utility versus 0.801 for Fixed-Window SSL (g=10) while cutting Educator tokens 61.7%. That 0.004 difference has no confidence interval or significance test, unlike every other headline contrast in the paper. The bootstrap procedure they use for family-level CIs would work here. This is a load-bearing gap for the efficiency argument because the claim of "preserving utility" depends on that 0.004 not being noise. It doesn't affect the main interleaving-vs-post-hoc findings, which have proper CIs, but it needs to be addressed. The authors acknowledge the single-anchor limitation, which is fine, and the CIs for the shared 10-family subset are wide, but the main cohort comparisons are solid.
Who this is for: anyone working on LLM agent skill acquisition, curriculum learning, or human-in-the-loop agent training. The paper deserves a serious referee. I'd ask the authors to add CIs for the Issue-Guided vs Fixed-Window comparison and to discuss what magnitude of utility change would count as meaningful degradation. After that, publish.
The strongest objection
The paper's central efficiency result (RQ4) states that Issue-Guided SSL reaches 0.805 utility versus 0.801 for Fixed-Window SSL (g=10) while reducing Educator tokens by 61.7%. This 0.004 difference (0.5 percentage points) is small, yet no confidence interval or statistical test is reported for this comparison, unlike the headline contrasts in Table 8. The bootstrap procedure used elsewhere (resampling families) could straightforwardly be applied here. Without it, the reader cannot assess whether the utility is statistically unchanged (supporting 'preserves transfer') or if there is a meaningful (possibly negative) shift. The paper's strongest efficiency claim depends on this being a true preservation, not a degradation hidden by noise. The omission is notable because the paper does report a narrow CI for Fixed-Window SSL vs. Trace2Skill. This is a load-bearing methodological gap for the efficiency part of the argument, though it does not affect the main interleaving-versus-post-hoc comparison.
The reader's weakest assumption focused on generalization from a single anchor, which is acknowledged as a limitation. My concern is different: the statistical rigor around the primary efficiency comparison is incomplete. However, this concern does not undermine the core claim that interleaved guidance beats post-hoc teaching, which is well-supported across three model pairs with reported intervals. The efficiency claim is important but secondary; the missing CI is a flaw in presentation but not fatal. Therefore the reader's ACCEPT verdict remains appropriate. The agreement is partial because the reader identified a different weakest point (single-anchor generalization), while I identify a statistical support gap for the efficiency claim. Both are load-bearing but neither changes the overall verdict.
The referee's report
If the results hold, SSL provides a principled way to combine expert guidance with student-led practice for learning transferable agent skills, reducing expert supervision cost while maintaining or improving transfer utility. The construction of SkillTransferBench with public variation specifications is itself a contribution that enables future work on guided skill acquisition. The process analysis linking teaching to student writing and revision offers empirical evidence for the role of interactive guidance.
Pith's reading of the paper (a machine's summary, not the authors')
Socratic Skill Learning (SSL) is a method for acquiring transferable agent skills by interleaving expert teaching with student practice. Unlike prior approaches that offer feedback after task attempts or distill static trajectories, SSL allows an Educator to guide investigation while the Student remains the sole environment operator and skill author. The paper evaluates SSL on a new benchmark, SkillTransferBench (35 task families, 408 cases), and shows that both Fixed-Window SSL and Issue-Guided SSL outperform direct expert authoring, post-hoc teaching, and adapted trajectory-distillation methods across three model pairs. Issue-Guided SSL further reduces Educator token use by 62 percent while maintaining transfer utility, by tracking unresolved learning issues to selectively trigger reviews.
What carries the argument. Two core mechanisms carry the argument. First, Socratic Skill Learning (SSL): during ongoing student practice, an Educator observes recent work and the live skill, then explains, challenges, diagnoses, or remains silent; the Student operates the environment and authors the package, able to question, test, and adapt advice. Second, Issue-Guided SSL: a persistent tree of learning issues tracks unresolved questions across revisions; a review is triggered only when an issue opens, changes, or reopens, and the Educator receives focused context around that issue. This selective review reduces unnecessary expert calls while retaining the ability to redirect investigation when new evidence contradicts prior conclusions.
Core claim. The central discovery is that interleaving expert guidance with student practice during skill acquisition produces more transferable skills than providing the same guidance after the student completes its work. The Educator reviews ongoing work, challenges assumptions, and suggests investigations, while the Student tests, adapts, and decides what to retain. This process is instantiated in Fixed-Window SSL (review every g tool calls) and Issue-Guided SSL (review on unresolved issue transitions). Across experiments on 22 task families with 155 held-out cases, Fixed-Window SSL achieves 0.801 utility (compared to 0.723 without a skill, 0.761 for Trace2Skill, and 0.751 for SKILL-KD). Issue-Guided SSL matches this performance at 0.805 utility while using 61.7 percent fewer Educator tokens and 16.9 percent fewer total acquisition tokens than Fixed-Window SSL at g equals 10.
The premise it rests on. The paper assumes that learning from a single anchor practice task per family, guided by an Educator, yields skills that generalize to unseen variants described by the family specification, an assumption the authors acknowledge as a limitation.
If the paper is right (Pith's inferences, not the authors')
If SSL is correct, agent skill learning systems can more efficiently use expert knowledge by providing timely guidance during exploration, reducing the need for costly post-hoc revisions.
The method may extend to other human-in-the-loop or intelligent tutoring scenarios where transferable procedures need to be learned from limited practice.
Issue-Guided SSL's adaptive review strategy offers a concrete path to lowering supervision costs in large language model interactions: by tracking unresolved issues, the system can focus expert attention where it matters most.
The SkillTransferBench benchmark (35 families, 408 cases) with public variation specifications and held-out evaluation provides a standardized testbed for future research on transferable agent skills.
The effectiveness of SSL may partly stem from the generation effect: students who construct skills themselves retain knowledge better than those who receive static instructions, and the guidance ensures productive exploration directions.
Issue-Guided SSL's issue tree could naturally extend to continual learning settings, where past unresolved issues from previous tasks could inform review priorities in a lifelong agent.
If the student model lacks sufficient capacity to benefit from Socratic dialogue (e.g., weaker models cannot effectively test and adapt advice), the advantage of interleaving may diminish, suggesting a dependency on student model capability.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Agent Skills improve LLM agents by making task knowledge reusable across sessions. Learning transferable Skills requires evidence about a procedure and its conditions of use, which a successful task attempt can leave untested. We introduce Socratic Skill Learning (SSL), which interleaves expert teaching with Student practice. An Educator guides investigation while the Student remains the sole environment operator and Skill author, able to question, test, and adapt advice. To test reuse beyond the learning task, we construct SkillTransferBench, with 35 task families and 408 executable cases, pairing public variation specifications with held-out evaluation in fresh solver sessions. On the main 22-family, 155-case cohort, Fixed-Window SSL reaches 0.801 utility, versus 0.723 without a Skill, 0.761 for adapted Trace2Skill (trajectory distillation), and 0.751 for adapted SKILL-KD (contrastive distillation). Across three model pairs on shared families, interleaved guidance outperforms expert authoring and post-hoc teaching. Process analysis links guidance to Student writing and package revision; a case study shows the Student testing and refining a proposed procedure for later reuse. As demand for new lessons falls, the cost of fixed-window review grows. Issue-Guided SSL tracks unresolved questions across Skill revisions to select and focus review, reaching 0.805 utility with 61.7% fewer Educator tokens and 16.9% fewer total acquisition tokens than Fixed-Window SSL on the main cohort.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-10-14
The authors' main claim, as the classifier picked it out 0.87
“Across three model pairs on shared families, interleaved guidance outperforms expert authoring and post-hoc teaching.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.07 · states limitation 0.88 · human participants 0.07 · controlled comparison 0.96 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.98 · mentions code or data availability 0.07
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Agent Skills improve LLM agents by making task knowledge reusable across sessions. | background 1.00 | 0.09 | 0.04 | asserted 1.00 |
| Learning transferable Skills requires evidence about a procedure and its conditions of use, which a successful task attempt can leave untested. | gap 0.80 | 0.10 | 0.03 | asserted 0.97 |
| We introduce Socratic Skill Learning (SSL), which interleaves expert teaching with Student practice. | contribution 0.84 | 0.87 | 0.06 | asserted 0.99 |
| An Educator guides investigation while the Student remains the sole environment operator and Skill author, able to question, test, and adapt advice. | method 0.96 | 0.56 | 0.04 | asserted 0.99 |
| To test reuse beyond the learning task, we construct SkillTransferBench, with 35 task families and 408 executable cases, pairing public variation specifications with held-out evaluation in fresh solver sessions. | method 0.73 | 0.96 | 0.04 | asserted 0.99 |
| On the main 22-family, 155-case cohort, Fixed-Window SSL reaches 0.801 utility, versus 0.723 without a Skill, 0.761 for adapted Trace2Skill (trajectory distillation), and 0.751 for adapted SKILL-KD (contrastive distillation). | result 1.00 | 0.96 | 0.33 | asserted 1.00 |
| Across three model pairs on shared families, interleaved guidance outperforms expert authoring and post-hoc teaching. | result 1.00 | 0.94 | 0.27 | asserted 1.00 |
| Process analysis links guidance to Student writing and package revision; a case study shows the Student testing and refining a proposed procedure for later reuse. | contribution 0.32 | 0.92 | 0.06 | asserted 0.99 |
| As demand for new lessons falls, the cost of fixed-window review grows. | limitation 0.90 | 0.39 | 0.03 | asserted 0.99 |
| Issue-Guided SSL tracks unresolved questions across Skill revisions to select and focus review, reaching 0.805 utility with 61.7% fewer Educator tokens and 16.9% fewer total acquisition tokens than Fixed-Window SSL on the main cohort. | result 0.77 | 0.96 | 0.22 | asserted 1.00 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.89
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-14: 1 request, 7,679 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (7)
Figures are the authors', reproduced from the paper.
References (51)
- [1] Zhang, Genrui and Zhu, Erle and Zhou, Jinfeng and Jia, Caiyan and Wang, Hongning , title =. arXiv preprint arXiv:2605.10500 , year = on PithPith's reviewarXiv
- [3] He, Yu and Yang, Weikai , title =. arXiv preprint arXiv:2608.15165 , year = on PithPith's reviewarXiv
- [5] Agent Skills Overview , year = on Pith
- [6] Chi, Michelene T. H. and Siler, Stephanie A. and Jeong, Heisawn and Yamauchi, Takashi and Hausmann, Robert G. , title =. Cognitive Science , volume = on Pith
- [7] Introducing Composer 2.5 , year = on Pith
- [8] Alzubi, Salaheddin and Provenzano, Noah and Bingham, Jaydon and Chen, Weiyuan and Vu, Tu , title =. arXiv preprint arXiv:2603.02766 , year = on PithPith's reviewarXiv
- [9] Hattie, John and Timperley, Helen , title =. Review of Educational Research , volume = on Pith
- [10] Madaan, Aman and Tandon, Niket and Gupta, Prakhar and Hallinan, Skyler and Gao, Luyu and Wiegreffe, Sarah and Alon, Uri and Dziri, Nouha and Prabhumoye, Shrimai and Yang, Yiming and others , title =. Advances in Neural Information Processing Systems , year = on Pith
- [11] Lightman, Hunter and Kosaraju, Vineet and Burda, Yura and Edwards, Harrison and Baker, Bowen and Lee, Teddy and Leike, Jan and Schulman, John and Sutskever, Ilya and Cobbe, Karl , title =. International Conference on Learning Representations , year = on Pith
- [14] Shinn, Noah and Cassano, Federico and Gopinath, Ashwin and Narasimhan, Karthik and Yao, Shunyu , title =. Advances in Neural Information Processing Systems , year = on Pith
- [15] Yu, Zhuoyun and Xie, Xin and Yao, Wuguannan and Wang, Chenxi and Liang, Lei and Qi, Xiang and Deng, Shumin , title =. arXiv preprint arXiv:2606.01311 , year = on PithPith's reviewarXiv
- [20] VanLehn, Kurt , title =. Educational Psychologist , volume = on Pith
- [22] Yao, Shunyu and Zhao, Jeffrey and Yu, Dian and Du, Nan and Shafran, Izhak and Narasimhan, Karthik and Cao, Yuan , title =. International Conference on Learning Representations , year = on Pith
- [23] Zhao, Andrew and Huang, Daniel and Xu, Quentin and Lin, Matthieu and Liu, Yong-Jin and Huang, Gao , title =. Proceedings of the AAAI Conference on Artificial Intelligence , volume =. 2024 , doi = on Pith
- [24] Merrill, Mike A. and Shaw, Alexander G. and Carlini, Nicholas and Li, Boxuan and Raj, Harsh and others , title =. International Conference on Learning Representations , year = on Pith
- [26] Ma, Zeyao and Zhang, Bohan and Zhang, Jing and Yu, Jifan and Zhang, Xiaokang and Zhang, Xiaohan and Luo, Sijia and Wang, Xi and Tang, Jie , title =. Advances in Neural Information Processing Systems , volume =. 2024 , doi = on Pith
- [27] Li, Yujia and Choi, David and Chung, Junyoung and Kushman, Nate and Schrittwieser, Julian and others , title =. Science , volume = on Pith
- [30] Wei, Tianxin and Shi, Zhan and Lin, Minhua and He, Bing and Liu, Zewen and Sang, Yisi and Bei, Yuanchen and Ning, Xuying and Zou, Jiaru and Li, Ting-Wei and Lin, Xiao and Zhao, Yanjun and Wang, Chi and Dumoulin, Benoit and Wang, Dakuo and He, Jingrui and Lu, Hanqing , title =.... on PithPith's reviewarXiv
- [31] Zweiger, Adam and Pari, Jyothish and Guo, Han and Akyürek, Ekin and Kim, Yoon and Agrawal, Pulkit , title =. arXiv preprint arXiv:2506.10943 , year = on PitharXiv
- [33] Jiang, Yangqin and Huang, Chao , journal = on Pith
show 31 more
- [34] Liu, Hongjun and Ming, Yifei and Joty, Shafiq and Zhao, Chen , journal =. Harnessing on Pith
- [35] Liu, Yuxuan and Su, Zhaochen and Xie, Lingyun and Zhang, Yuhao and Zong, Qing and Guo, Jiahe and Xie, Zhongwei and Ji, Yiyan and Yim, Yauwai and Luo, Hongyu and Ren, Xiyu and Chenyu, Ruan and Li, Haoran and Song, Yangqiu , journal = on Pith
- [36] Liu, Jiayu and Huang, Zhenya and Xiao, Tong and Sha, Jing and Wu, Jinze and Liu, Qi and Wang, Shijin and Chen, Enhong , booktitle =. 2024 , url = on Pith
- [37] Quan, Yinzhu and Liu, Zefang , journal = on Pith
- [38] Anthropic . Agent skills overview. https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview, 2025. Accessed 2026-08-02 on Pith
- [39] Kaiyuan Chen, Qimin Wu, Taiyu Hou, Tianhao Tang, Xueyu Hu, et al. AgentIF-OneDay : A task-level instruction-following benchmark for general AI agents in daily scenarios. arXiv preprint arXiv:2601.20613, 2026 on PitharXiv
- [40] Michelene T. H. Chi, Stephanie A. Siler, Heisawn Jeong, Takashi Yamauchi, and Robert G. Hausmann. Learning from human tutoring. Cognitive Science, 25 0 (4): 0 471--533, 2001 on Pith
- [41] Tianyi Guan, Yiding Wang, Haotong Yang, Siyuan Cao, Shirui Liu, Yi Hu, Jiaqi Li, and Muhan Zhang. ContinualSkillBench : Can LLM agents truly evolve their capabilities? arXiv preprint arXiv:2608.03874, 2026 on PithPith's reviewarXiv
- [42] John Hattie and Helen Timperley. The power of feedback. Review of Educational Research, 77 0 (1): 0 81--112, 2007 on Pith
- [43] Yangqin Jiang and Chao Huang. AgentBrew : Lifelong knowledge brewing from strong teachers to weak LLM agents. arXiv preprint arXiv:2607.16851, 2026 on PithPith's reviewarXiv
- [44] Han Li, Yifan Yao, Letian Zhu, Rili Feng, Hongyi Ye, Jiaming Wang, Yancheng He, Pengyu Zou, Lehan Zhang, Xinping Lei, Haoyang Huang, Ken Deng, Ming Sun, Zhaoxiang Zhang, He Ye, and Jiaheng Liu. CodeTracer : Towards traceable agent states. arXiv preprint arXiv:2604.11641, 2026 a on PithPith's reviewarXiv
- [45] Xiangyi Li, Yimin Liu, Wenbo Chen, Bingran You, Zonglin Di, et al. SkillsBench : Benchmarking how well agent skills work across diverse tasks. arXiv preprint arXiv:2602.12670, 2026 b on PithPith's reviewarXiv
- [46] Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, et al. Competition-level code generation with AlphaCode . Science, 378 0 (6624): 0 1092--1097, 2022 on Pith
- [47] Zhiwei Li and Yong Hu. SkillHone : A harness for continual agent skill evolution through persistent decision history. arXiv preprint arXiv:2606.08671, 2026 on PithPith's reviewarXiv
- [48] Hongjun Liu, Yifei Ming, Shafiq Joty, and Chen Zhao. Harnessing LLM agents with skill programs. arXiv preprint arXiv:2605.17734, 2026 a on PithPith's reviewarXiv
- [49] Jiayu Liu, Zhenya Huang, Tong Xiao, Jing Sha, Jinze Wu, Qi Liu, Shijin Wang, and Enhong Chen. SocraticLM : Exploring socratic personalized teaching with large language models. In Advances in Neural Information Processing Systems, volume 37, 2024 a . https://papers.neurips.cc/p... on Pith
- [50] Siyao Liu, He Zhu, Jerry Liu, Shulin Xin, Aoyan Li, Rui Long, Li Chen, Jack Yang, Jinxiang Xia, Z. Y. Peng, Shukai Liu, Zhaoxiang Zhang, Ge Zhang, Wenhao Huang, Kai Shen, and Liang Xiang. FullStack Bench : Evaluating LLM s as full stack coders. arXiv preprint arXiv:2412.00535, 2024 b on PithPith's reviewarXiv
- [51] Yuxuan Liu, Zhaochen Su, Lingyun Xie, Yuhao Zhang, Qing Zong, Jiahe Guo, Zhongwei Xie, Yiyan Ji, Yauwai Yim, Hongyu Luo, Xiyu Ren, Ruan Chenyu, Haoran Li, and Yangqiu Song. SkillRevise : Improving LLM -authored agent skills via trace-conditioned skill revision. arXiv preprint ... on PithPith's reviewarXiv
- [52] Yuchen Ma, Yue Huang, Han Bao, Haomin Zhuang, Swadheen Shukla, Michel Galley, Xiangliang Zhang, and Stefan Feuerriegel. SkillGen : Verified inference-time agent skill synthesis. arXiv preprint arXiv:2605.10999, 2026 on PithPith's reviewarXiv
- [53] Zeyao Ma, Bohan Zhang, Jing Zhang, Jifan Yu, Xiaokang Zhang, Xiaohan Zhang, Sijia Luo, Xi Wang, and Jie Tang. SpreadsheetBench : Towards challenging real world spreadsheet manipulation. In Advances in Neural Information Processing Systems, volume 37, pages 94871--94908, 2024. ... on Pithdoi:10.52202/079017-3007
- [54] Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with self-feedback. In Advances in Neural Information Processing Systems, 2023 on Pith
- [55] Mike A. Merrill, Alexander G. Shaw, Nicholas Carlini, Boxuan Li, Harsh Raj, et al. Terminal-Bench : Benchmarking agents on hard, realistic tasks in command line interfaces. In International Conference on Learning Representations, 2026 on Pith
- [56] Jingwei Ni, Yihao Liu, Xinpeng Liu, Yutao Sun, Mengyu Zhou, Pengyu Cheng, Dexin Wang, Erchao Zhao, Xiaoxi Jiang, and Guanjun Jiang. Trace2Skill : Distill trajectory-local lessons into transferable agent skills. arXiv preprint arXiv:2603.25158, 2026 on PithPith's reviewarXiv
- [57] Yinzhu Quan and Zefang Liu. EconSkills : Studying skill transfer and retrieval for web agents on live economic data. arXiv preprint arXiv:2609.19523, 2026 on PithPith's reviewarXiv
- [58] Qiming Shi, Yibo Dou, Jiawen Zhu, Yulong Tao, Linbo Jin, Zhaolu Kang, Yunfan Zhou, and Di Weng. SKILL-KD : Contrastive skill distillation for LLM agents. arXiv preprint arXiv:2607.28048, 2026 on PithPith's reviewarXiv
- [59] Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems, 2023 on Pith
- [60] Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291, 2023 on PithPith's reviewarXiv
- [61] Zilong Wang, Yuedong Cui, Li Zhong, Zimin Zhang, Da Yin, Bill Yuchen Lin, and Jingbo Shang. OfficeBench : Benchmarking language agents across multiple applications for office automation. arXiv preprint arXiv:2407.19056, 2024 on PithPith's reviewarXiv
- [62] Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. ExpeL : LLM agents are experiential learners. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 19632--19642, 2024. doi:10.1609/aaai.v38i17.29936 on Pithdoi:10.1609/aaai.v38i17.29936
- [63] Xiangxin Zhao, Han Li, Shuaiting Li, Tianyi Zhao, Earl T. Barr, Federica Sarro, and He Ye. Failure as a process: An anatomy of CLI coding agent trajectories. arXiv preprint arXiv:2607.09510, 2026 on PithPith's reviewarXiv
- [64] Shanshan Zhong, Yi Lu, Jingjie Ning, Yibing Wan, Lihan Feng, Yuyi Ao, Leonardo F. R. Ribeiro, Markus Dreyer, Sean Ammirati, and Chenyan Xiong. SkillLearnBench : Benchmarking continual learning methods for agent skill generation on real-world tasks. arXiv preprint arXiv:2604.20... on PithPith's reviewarXiv
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment