Pith. sign in

REVIEW 2 major objections 4 minor 45 references

Self-evolving agent skills are sparse, validation-filtered search, not steady improvement, this paper argues.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-05 04:21 UTC pith:6TLY3PYT

load-bearing objection A well-executed controlled study of agent skill evolution whose qualitative claims hold, but whose headline counts need uncertainty quantification before they can be taken at face value. the 2 major comments →

arxiv 2608.02636 v1 pith:6TLY3PYT submitted 2026-07-31 cs.SE cs.AI

Rethinking Self-Evolving Agent Skills: Feedback Dynamics over Multiple Rounds

Yuxuan Liu , Zhaochen Su , Yuhao Zhang , Jiahe Guo , Zhongwei Xie , Huihao Jing , Lingyun Xie , Qing Zong
show 4 more authors
Yauwai Yim Zhixiong Zhang Haoran Li Yangqiu Song
This is my paper
classification cs.SE cs.AI
keywords self-evolving agentsskill evolutionfeedback compositionvalidation-filtered searchtest-time scalingagent skillsLLM agents
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to settle when and how self-evolving agent skills improve. Using a controlled protocol that changes only which execution feedback an optimizer sees, it finds evidence that persistent skill evolution behaves like sparse, validation-filtered search: only 55 of 388 candidate skills establish strict validation new bests, and validation selects an evolved skill in 11 of 14 model-benchmark settings. All 11 selected evolved skills came from feedback that included failed trajectories; Success-only feedback was never selected in the primary study. Additional rounds mostly stop yielding new bests after the early stage, and extra per-instance inference recovers the evolved skill's gain on SearchQA but not on SpreadsheetBench. The result matters because it tells when self-evolving skills are worth their compute and when endpoint scores alone mislead.

Core claim

The central claim is that self-evolving agent skills should be understood as validation-filtered search over persistent skill artifacts, not as monotonic improvement from additional evolution rounds. Across 42 feedback runs spanning three models and five benchmarks, only 55 of 388 candidates establish byte-distinct validation bests; accepted revisions are rare, and runs show late discovery, early saturation, regression, and stagnation. Validation-based selection picks an evolved skill in 11 of 14 settings, nine of which improve released-test performance, and all 11 selections are produced by feedback views containing failed trajectories: Normal is selected nine times, Fail-only twice, and Su

What carries the argument

The central object is a feedback-conditioned, validation-gated evolution loop. Three matched feedback views—Normal (successes and failures), Fail-only, and Success-only—start from the same parent skill, and at each round a revision operator proposes a candidate that is accepted only if validation does not decrease; a separate best checkpoint updates only on strict validation improvement by a byte-distinct artifact. This machinery isolates feedback composition as the sole variable and defines 'evolution' as the set of candidates that pass the validation gate, making sparse search visible.

Load-bearing premise

The load-bearing premise is that a validation score that 'does not decrease' on splits of 35 to 53 items reflects a real change in skill quality; if those small differences are mostly run-to-run noise, the counts of new bests and the feedback rankings would not survive.

What would settle it

Re-run the same 42 evolution runs with the validation split expanded, or bootstrap-resample the existing validation verdicts, and check whether 55/388 byte-distinct new bests and the all-failure-containing selection pattern persist; if chance-level variation on 35-53 item splits can produce that pattern, the central claim is a validator artifact.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • A four-round budget would capture most validation new bests (38 of 55) but would miss six of the 11 final evolved selections, so round budgets should be judged by timing of new bests, not number of revisions.
  • Failure-containing feedback is necessary for selection in the primary study, but the ranking of Normal vs Fail-only varies by model and benchmark; no single feedback view has a fixed advantage.
  • Validation improvement does not guarantee test improvement: GPT-5.5 LiveMath's only validation gain reduces released-test performance by 6.6 points.
  • Persistent skill gains and test-time scaling measure different outcomes: parallel sampling can recover answer-form guidance on SearchQA but not the multi-step SpreadsheetBench workflow.
  • Evaluation reports should include search trajectories, skill identity, downstream generalization, and test-time-compute controls rather than endpoint scores alone.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the validation splits' small sizes (49, 39, 35, and 53 items) have a noise floor comparable to the 0.5-2.0 point deltas the paper counts as new bests, then the headline sparsity counts could be driven more by the validator than by genuine skill change; a bootstrap or repeated-run analysis of the same candidates would separate these.
  • The Success-only result may be partly a data-availability artifact: success pools can be small and can exhaust before ten rounds, so the comparison is not fully symmetric in the amount of evidence each view receives.
  • Because validation selection rewards skills that satisfy the verifier, Fail-only's success on SpreadsheetBench may indicate that evolution is optimizing verifier-visible properties, which would make selected skills fragile to verifier changes—a channel the paper's single-alternative verifier check only begins to probe.
  • The paper's own cost accounting (1 + 2,750/n calls per deployment for the selected SearchQA run) implies that at large deployment scales evolution becomes cheaper per task than parallel sampling at high budgets, so the evolution-versus-test-time-scaling tradeoff flips with deployment size.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper presents a controlled evaluation of multi-round self-evolving agent skills across five benchmarks and three models (14 model–benchmark settings, 42 feedback runs). Within each setting, Normal, Fail-only, and Success-only feedback views are evolved from the same parent skill under a fixed optimizer, revision procedure, validation rule, and round budget. The main empirical claims are that evolution is sparse (only 55 of 388 candidates establish byte-distinct validation bests), that validation selects an evolved skill in 11 of 14 settings, that all 11 selected skills come from feedback conditions containing failed trajectories while Success-only is never selected in the primary study, and that test-time-scaling controls cannot consistently recover the gains of persistent skill evolution. The paper also reports robustness, transfer, repeated-execution, and verifier-sensitivity analyses, plus a broader eight-model SearchQA study.

Significance. If the aggregate results are reliable, this is a valuable empirical contribution: it moves the discussion of self-evolving skills from endpoint scores to search trajectories, artifact identity, feedback composition, and test-time-compute baselines. The controlled protocol is a clear strength: same parent skill, same optimizer procedure, byte-distinct artifact identity, validation-based selection before test evaluation, and evaluation by external benchmark verifiers. The public implementation, the detailed per-run records, and the explicit test-time-scaling controls are also commendable. The main risk is that the headline sparse-search and feedback-dependence counts depend on single-run validation on very small splits, and the paper itself provides evidence that this validation signal is noisy.

major comments (2)
  1. [§Experiments/Experimental Setup; Tables A1 and A13; identity-aware validation results] The headline counts—55/388 byte-distinct validation bests and 11/14 validation-selected evolved skills, including 'Success-only is never selected'—rest entirely on single-run validation on small splits: SearchQA 200, OfficeQA 49, SpreadsheetBench 39, LiveMath 35, DocVQA 53 (Table A1). Many reported new-best deltas are 0.5–3.0 points, i.e., one to a few items. The paper's own repeated-execution data show this gate is noisy: the same Gemini–OfficeQA Success-only artifact scored 71.43–83.67 across eight validation executions (SD 3.92 on a 49-item split), and Table A13 shows frozen-artifact repeated-panel differences of +15.8 to +19.3 pp for OfficeQA and +5.0 to +7.0 pp for LiveMath. With no confidence intervals, significance tests, or repeated validation runs for accepted or rejected candidates, the sparse-search percentages and the feedback-dependence selections may be properties of valida
  2. [§Framework Overview (Eq. 2); §Effects of Additional Evolution Rounds] The acceptance rule 'a candidate becomes the next-round skill when validation does not decrease' (Eq. 2) makes the evolution trajectory path-dependent on noisy validation outcomes. Because only the incumbent skill carries forward, a candidate accepted on a positive noise fluctuation changes all subsequent feedback and revision, which can generate late-round improvements or stagnation that are artifacts of the gate rather than of skill content. This is especially relevant to the claimed late-round dynamics (e.g., OfficeQA Normal improving at rounds 1, 8, and 9, and SearchQA reaching its best only at round 9). The authors should test sensitivity by re-running at least a subset of settings with a stricter acceptance threshold (e.g., improvement beyond one standard error, or repeated validation), or by reporting how many accepted candidates would be rejected under such a threshold.
minor comments (4)
  1. [Appendix A1] The OfficeQA train/validation/test split is listed as 49/49/148, which is exceptionally small for both training and validation. Please state explicitly why this split was chosen and discuss its implications for the reliability of the validation gate.
  2. [Table 2 and Figure 1] The color shading used to mark validation-selected views and positive/negative comparisons may be inaccessible in monochrome print. Consider adding textual markers or symbols, as already done in Figure 2.
  3. [Appendix A12] The broader SearchQA analysis reports that 191 of 210 candidates change the incoming skill but only 29 establish byte-distinct validation bests. This is interesting, but the same validation-noise concern applies; a sentence acknowledging this and pointing to the uncertainty analysis would help.
  4. [General] The terms 'validation-selected skill' and 'selected evolved skill' are used interchangeably in places. Defining them once near the first use would reduce ambiguity.

Circularity Check

0 steps flagged

No significant circularity: the empirical counts are read off external verifiers, not derived from the protocol's definitions.

full rationale

The paper is an empirical, controlled evaluation. Its equations (1)-(4) only describe the evolution and evaluation pipeline; they do not map outputs back to inputs. The headline counts (55/388 byte-distinct bests, 11/14 validation-selected evolved skills) are tallies of externally verified validation outcomes under a fixed gate, and the test/robustness/transfer scores come from frozen skill artifacts evaluated by benchmark verifiers. No parameter is fitted to the reported test endpoints, and no prediction is derived from a model whose inputs include the target result. The validation gate ('a candidate becomes the next-round skill when validation does not decrease; only strict improvement updates the best checkpoint') is a protocol rule, but the sparsity and feedback-dependence findings are not forced by that rule: the runs could have produced many improvements or selected Success-only, and the fact that they did not is an empirical observation. The paper's self-citations (e.g., SkillRevise, Liu et al. 2026e) are contextual descriptions of prior feedback policies, not load-bearing evidence for the present conclusions. The small validation-split sizes and observed run-to-run variability (Table A13) raise legitimate concerns about whether the validation gate is reliable, but that is a statistical/reliability issue, not circular reasoning. No circular step is exhibited.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

No fitted model parameters or invented entities. The central claims are empirical and rest on benchmark verifiers, the matched-feedback protocol, and the validation gate. The protocol constants (round budget, edit cap, batch sizes) are fixed choices that condition the reported counts.

free parameters (3)
  • round_budget = 10 rounds
    Fixed protocol constant; sparsity and late-round selection counts are conditional on the 10-round search horizon.
  • edit_cap = 4 edits per proposal
    Optimizer may make at most four minimal edits; the number and size of accepted candidate changes depend on this cap.
  • per_round_trajectory_batch = 40 (36 for LiveMath)
    Each round samples 40 training trajectories (36 for LiveMath) to build feedback views; batch size determines how many success/failure records are visible.
axioms (4)
  • domain assumption Validation hard scores on fixed small splits are a reliable ordinal signal of skill quality, and byte-distinct artifacts isolate genuine updates.
    The acceptance gate and all new-best counts rest on this assumption; validation n is 35-53 for four benchmarks and no uncertainty/significance analysis is supplied.
  • domain assumption Benchmark verifier success/failure labels are accurate enough to serve as feedback and validation.
    Output-locked verifier sensitivity (Appendix A28) shows 4.8% disagreement on 100-item panels, so labels are not perfectly stable.
  • domain assumption Normal, Fail-only, and Success-only are matched: same executor/optimizer config, revision procedure, validation rule, and round budget, differing only in visible feedback.
    The entire feedback-composition comparison assumes this matching; API nondeterminism is acknowledged but not eliminated.
  • domain assumption Test-time scaling controls share executor, tool interface, test pool, verifier, and call budget with the evolution runs.
    The contrast between persistent skill gains and per-instance inference depends on these controls being equated.

pith-pipeline@v1.3.0-alltime-deepseek · 32039 in / 16137 out tokens · 175427 ms · 2026-08-05T04:21:31.933255+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Rethinking Self-Evolving Agent Skills: Feedback Dynamics over Multiple Rounds." pith.science (2026). https://pith.science/paper/6TLY3PYT

@misc{pith2026260802636,
  author       = {Pith},
  title        = {Pith review of: Rethinking Self-Evolving Agent Skills: Feedback Dynamics over Multiple Rounds},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6TLY3PYT}},
  note         = {Machine review of arXiv:2608.02636}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Self-evolving skill systems promise to improve agents by turning execution feedback into persistent skill updates without changing the underlying model. Yet it remains unclear when further evolution helps, how successful and failed trajectories shape revision, and whether extra test-time computation can recover the same gains. To address these questions, we present a controlled evaluation framework across five benchmarks and three models. Our primary study contains 42 feedback runs across 14 supported model-benchmark settings. Within each setting, we hold the executor and optimizer configuration, revision procedure, validation rule, and round budget fixed, while varying only the feedback shown to the optimizer: successes and failures (Normal), failures only, or successes only. Evolution is sparse: only 55 of 388 candidates establish byte-distinct validation bests. Validation-based selection chooses an evolved skill in 11 of 14 settings, nine of which improve released-test performance. All 11 selections come from feedback conditions that include failed trajectories, although the relative ranking of Normal and Fail-only varies across settings. Validation and downstream evaluations on test, robustness, and transfer sometimes favor different feedback views. A broader SearchQA analysis covering eight models shows similarly sparse, feedback-dependent dynamics. In the GPT-5.5 test-time-scaling controls, oracle Parallel Sampling comes within 0.43 points of the evolved SearchQA skill but remains 30.96 points behind on SpreadsheetBench; Sequential Refinement recovers neither gain. Overall, persistent skill self-evolution is better understood as sparse, validation-filtered search with model- and benchmark-dependent returns, rather than steady improvement from additional rounds. The implementation is available at https://github.com/HKUST-KnowComp/rethinkskill.

Figures

Figures reproduced from arXiv: 2608.02636 by Haoran Li, Huihao Jing, Jiahe Guo, Lingyun Xie, Qing Zong, Yangqiu Song, Yauwai Yim, Yuhao Zhang, Yuxuan Liu, Zhaochen Su, Zhixiong Zhang, Zhongwei Xie.

Figure 1
Figure 1. Figure 1: Feedback-conditioned skill evolution and controlled evaluation. Each round executes the current skill, constructs a [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Evolution events across 14 model–benchmark settings. Gray dots mark other evaluated candidates, blue dots mark [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: GPT-5.5 test-time scaling on benchmarks with byte-distinct evolved skills. (a) One-call parent and evolved-skill [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

45 extracted references · 14 canonical work pages · 9 internal anchors

  1. [1]

    Alzubi, S.; Provenzano, N.; Bingham, J.; Chen, W.; and Vu, T. 2026. EvoSkill: Automated Skill Discovery for Multi-Agent Systems . arXiv:2603.02766

    Pith/arXiv arXiv 2026

  2. [2]

    Chen, Z.; Guo, Z.; Huang, B.; Lu, B.; Lin, J.; Zhou, Y.; and Zhang, W. 2026. SkillJuror: Measuring How Agent Skill Organization Changes Runtime Behavior . arXiv:2606.11543

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  3. [3]

    U.; Cirik, V.; and Cho, K

    Dunn, M.; Sagun, L.; Higgins, M.; Guney, V. U.; Cirik, V.; and Cho, K. 2017. SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine . arXiv:1704.05179

    Pith/arXiv arXiv 2017

  4. [4]

    Han, T.; Zhang, Y.; Song, W.; Fang, C.; Chen, Z.; Sun, Y.; and Hu, L. 2026. SWE-Skills-Bench: Do Agent Skills Actually Help in Real-World Software Engineering? arXiv:2603.15401

    arXiv 2026

  5. [5]

    He, L.; Yu, Q.; Dong, H.; Liao, B.; Xu, X.; Goldblum, M.; Bian, J.; and Mesgarani, N. 2026. LiveMathematicianBench: A Live Benchmark for Mathematician-Level Reasoning with Proof Sketches . arXiv:2604.01754

    arXiv 2026

  6. [6]

    Huang, Z.; Xu, J.; Yang, Y.; Gong, Z.; Yang, Q.; Tian, M.; Wang, X.; Lv, C.; Gao, X.; Dai, Q.; Liu, B.; Qiu, K.; Yang, X.; Chen, D.; Zheng, X.; and Luo, C. 2026. From Raw Experience to Skill Consumption: A Systematic Study of Model-Generated Agent Skills . arXiv:2605.23899

    Pith/arXiv arXiv 2026

  7. [7]

    Jiang, G.; Su, Z.; Qu, X.; and Fung, Y. R. 2026 a . XSkill: Continual Learning from Experience and Skills in Multimodal Agents . arXiv:2603.12056

    Pith/arXiv arXiv 2026

  8. [8]

    Jiang, S.; Ma, L.; Hong, Z.; Wang, K.; Lu, Z.; Wang, T.; Chen, S.; Zhang, J.; Pan, T.; Li, W.; Liang, J.; and Xiao, Y. 2026 b . SEA-Eval: A Benchmark for Evaluating Self-Evolving Agents Beyond Episodic Assessment . arXiv:2604.08988

    Pith/arXiv arXiv 2026

  9. [9]

    Kim, J.; Yang, W.; Niu, K.; Zhang, H.; Zhu, Y.; Helenowski, E.; Silva, R.; Chen, Z.; Iyer, S.; Zaheer, M.; Fried, D.; Hajishirzi, H.; Arora, S.; Synnaeve, G.; Salakhutdinov, R.; and Goyal, A. 2026. Scaling Test-Time Compute for Agentic Coding . arXiv:2604.16529

    Pith/arXiv arXiv 2026

  10. [10]

    W.; Sun, J.; Wang, S.; Tao, C.; Li, B.; Zhao, X.; Geng, H.; Wu, X.; Zhou, J.; Chen, X.; Xing, H.; Li, Y.; Zeng, Q.; Wang, D.; Wang, Y.; Chaim, R

    Li, X.; Liu, Y.; Chen, W.; You, B.; Di, Z.; He, Y.; Zheng, S.; Choe, K. W.; Sun, J.; Wang, S.; Tao, C.; Li, B.; Zhao, X.; Geng, H.; Wu, X.; Zhou, J.; Chen, X.; Xing, H.; Li, Y.; Zeng, Q.; Wang, D.; Wang, Y.; Chaim, R. B.; Jiang, P.; Shen, H.; Kong, L.; Liu, X.; Wang, R.; Liu, X.; Li, J.; Lan, X.; Lin, Y.; Ye, W.; He, J.; Li, S.; Zhang, Y.; Gao, Y.; Li, Y....

    Pith/arXiv arXiv 2026

  11. [11]

    Li, X.; Ming, R.; Setlur, P.; Paladugu, A.; Tang, A.; Kang, H.; Shao, S.; Jin, R.; and Xiong, C. 2026 b . Benchmark Test-Time Scaling of General LLM Agents . arXiv:2602.18998

    arXiv 2026

  12. [12]

    Li, Z.; and Hu, Y. 2026. SkillHone: A Harness for Continual Agent Skill Evolution Through Persistent Decision History . arXiv:2606.08671

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  13. [13]

    Liu, H.; Yang, H.; Jiang, T.; Tang, B.; Xiong, F.; Luo, Y.; and Li, Z. 2026 a . SkillsVote: Lifecycle Governance of Agent Skills from Collection, Recommendation to Evolution . arXiv:2605.18401

    Pith/arXiv arXiv 2026

  14. [14]

    Liu, X.; Luo, X.; Li, L.; Huang, G.; Liu, J.; and Qiao, H. 2026 b . SkillForge: Forging Domain-Specific, Self-Evolving Agent Skills in Cloud Technical Support . arXiv:2604.08618

    Pith/arXiv arXiv 2026

  15. [15]

    Liu, Y. 2025. Exploring ConceptNet Knowledge for Enhancing LLMs Performance in Argument Impact Classification Tasks . In Proceedings of 17th International Conference on Machine Learning and Computing, volume 1475 of Lecture Notes in Networks and Systems, 379--395. Springer

    work page 2025

  16. [16]

    Liu, Y.; Ji, J.; An, L.; Jaakkola, T.; Zhang, Y.; and Chang, S. 2026 c . How Well Do Agentic Skills Work in the Wild: Benchmarking LLM Skill Usage in Realistic Settings . arXiv:2604.04323

    Pith/arXiv arXiv 2026

  17. [17]

    Liu, Y.; Liu, Y.; Yin, X.; Wang, B.; Zhang, C.; Yin, H.; and Han, Z. 2026 d . OpenClawBench: Benchmarking Process-side Anomalies in Real-world Agent Execution Trajectories . arXiv:2605.29253

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  18. [18]

    Liu, Y.; Su, Z.; Xie, L.; Zhang, Y.; Zong, Q.; Guo, J.; Xie, Z.; Ji, Y.; Yim, Y.; Luo, H.; Ren, X.; Ruan, C.; Li, H.; and Song, Y. 2026 e . SkillRevise: Improving LLM-Authored Agent Skills via Trace-Conditioned Skill Revision . arXiv:2606.01139

    Pith/arXiv arXiv 2026

  19. [19]

    Ma, Y.; Huang, Y.; Bao, H.; Zhuang, H.; Shukla, S.; Galley, M.; Zhang, X.; and Feuerriegel, S. 2026. SkillGen: Verified Inference-Time Agent Skill Synthesis . arXiv:2605.10999

    Pith/arXiv arXiv 2026

  20. [20]

    Ma, Z.; Zhang, B.; Zhang, J.; Yu, J.; Zhang, X.; Zhang, X.; Luo, S.; Wang, X.; and Tang, J. 2024. SpreadsheetBench: Towards Challenging Real World Spreadsheet Manipulation . arXiv:2406.14991

    Pith/arXiv arXiv 2024

  21. [21]

    Mathew, M.; Karatzas, D.; and Jawahar, C. V. 2021. DocVQA: A Dataset for VQA on Document Images . arXiv:2007.00398

    Pith/arXiv arXiv 2021

  22. [22]

    Ni, J.; Liu, Y.; Liu, X.; Sun, Y.; Zhou, M.; Cheng, P.; Wang, D.; Zhao, E.; Jiang, X.; and Jiang, G. 2026. Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skills . arXiv:2603.25158

    Pith/arXiv arXiv 2026

  23. [23]

    Shen, Y.; Li, B.; and Zhang, X. 2026. SkillOpt-Lite: Better and Faster Agent Self-evolution via One Line of Vibe . arXiv:2607.03451

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  24. [24]

    Shridhar, M.; Yuan, X.; C \^o t \'e , M.-A.; Bisk, Y.; Trischler, A.; and Hausknecht, M. 2021. ALFWorld: Aligning Text and Embodied Environments for Interactive Learning . arXiv:2010.03768

    Pith/arXiv arXiv 2021

  25. [25]

    R.; and He, J

    Su, Z.; Gao, J.; Guo, H.; Liu, Z.; Zhang, L.; Geng, X.; Huang, S.; Xia, P.; Jiang, G.; Wang, C.; Zhang, Y.; Fung, Y. R.; and He, J. 2026. AgentVista: Evaluating Multimodal Agents in Ultra-Challenging Realistic Visual Scenarios . arXiv:2602.23166

    arXiv 2026

  26. [26]

    Tian, Y.; Chen, J.; Zheng, L.; Tao, M.; Zeng, X.; Yin, Z.; Su, H.; and Sun, X. 2026. Skills-Coach: A Self-Evolving Skill Optimizer via Training-Free GRPO . arXiv:2604.27488

    Pith/arXiv arXiv 2026

  27. [27]

    Wang, Y.; Zhu, H.; Hu, Z.; Yuan, Y.; Chen, Z.; Senthil, S.; Hajishirzi, H.; Tsvetkov, Y.; Dasigi, P.; and Xiao, T. 2026. Rethinking the Evaluation of Harness Evolution for Agents . arXiv:2607.12227

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  28. [28]

    Xia, P.; Chen, J.; Wang, H.; Liu, J.; Zeng, K.; Wang, Y.; Han, S.; Zhou, Y.; Zhao, X.; Chen, H.; Zheng, Z.; Xie, C.; and Yao, H. 2026. SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning . arXiv:2602.08234

    Pith/arXiv arXiv 2026

  29. [29]

    Xie, Z.; Liao, R.; Wang, Z.; Chen, C.; Hua, X.-S.; and Luo, X. 2026 a . GALA: Geometric Data Selection with Strategic Prospecting for Large Language Model Self-training . In Findings of the Association for Computational Linguistics: ACL 2026, 10281--10293. Association for Computational Linguistics

    work page 2026

  30. [30]

    Xie, Z.; Ren, X.; Zheng, T.; Bai, J.; Fan, W.; Xu, B.; Li, H.; Jing, H.; and Song, Y. 2026 b . A Survey on AI Agent Harness . ResearchGate preprint

    work page 2026

  31. [31]

    Yang, C.; Jiang, Y.; Deng, Y.; and Wu, C. 2026 a . Skill-Conditioned Visual Geolocation for Vision-Language Models . arXiv:2604.09025

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  32. [32]

    Yang, H.; Zhao, K.; Ma, M.; Lu, X.; Wang, X.; and Qian, H. 2026 b . OptSkills: Learning Generalizable Optimization Skills from Problem Archetypes via Cluster-Based Distillation . arXiv:2605.29829

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  33. [33]

    Yang, Y.; Gong, Z.; Huang, W.; Yang, Q.; Zhou, Z.; Huang, Z.; Li, Y.; Gao, X.; Dai, Q.; Liu, B.; Qiu, K.; Yang, Y.; Chen, D.; Yang, X.; and Luo, C. 2026 c . SkillOpt: Executive Strategy for Self-Evolving Agent Skills . arXiv:2605.23904

    Pith/arXiv arXiv 2026

  34. [34]

    Yu, Z.; Xie, X.; Yao, W.; Wang, C.; Liang, L.; Qi, X.; and Deng, S. 2026. SkillAdaptor: Self-Adapting Skills for LLM Agents from Trajectories . arXiv:2606.01311

    Pith/arXiv arXiv 2026

  35. [35]

    P.; Chen, Y.; Wang, Z.; Zhou, J.; Li, C.; Huang, W.-C.; Yao, Y.; Zheng, K.; Liu, X.; Li, X.; and Yu, P

    Zhang, H.; Fan, S.; Zou, H. P.; Chen, Y.; Wang, Z.; Zhou, J.; Li, C.; Huang, W.-C.; Yao, Y.; Zheng, K.; Liu, X.; Li, X.; and Yu, P. S. 2026 a . CoEvoSkills: Self-Evolving Agent Skills via Co-Evolutionary Verification . arXiv:2604.01687

    Pith/arXiv arXiv 2026

  36. [36]

    Zhang, H.; Long, Q.; Bao, J.; Feng, T.; Zhang, W.; Yue, H.; and Wang, W. 2026 b . MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents . arXiv:2602.02474

    Pith/arXiv arXiv 2026

  37. [37]

    Zhang, X.; Wang, G.; Cui, Y.; Li, Z.; Qiu, W.; Zhu, B.; and He, P. 2026 c . Who Grades the Grader? Co-Evolving Evaluation Metrics and Skills for Self-Improving LLM Agents . arXiv:2607.12790

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  38. [38]

    Zhang, Y.; Duan, S.; Su, J.; Liu, C.; and Han, P. 2025 a . SPFT-SQL: Enhancing Large Language Model for Text-to-SQL Parsing by Self-Play Fine-Tuning . In Findings of the Association for Computational Linguistics: EMNLP 2025, 1092--1110. Association for Computational Linguistics

    work page 2025

  39. [39]

    Zhang, Y.; Yan, L.; Duan, S.; Zha, X.; Su, J.; Han, P.; and Liu, C. 2026 d . AFT-Tab: Adversarial Fine-Tuning for Tabular Data Synthesis with Long Text Columns . In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 4581--4594. Association for Computational Linguistics

    work page 2026

  40. [40]

    Zhang, Z.; Ding, S.; Dong, X.; He, S.; Lin, J.; Tang, J.; Zang, Y.; Cao, Y.; Lin, D.; and Wang, J. 2025 b . SeC: Advancing Complex Video Object Segmentation via Progressive Concept Construction . arXiv:2507.15852

    arXiv 2025

  41. [41]

    Zhang, Z.; Li, Y.; Ding, S.; Zang, Y.; Ding, S.; Xing, L.; Wang, Y.; Zhang, Q.; and Wang, J. 2026 e . SetCon: Towards Open-Ended Referring Segmentation via Set-Level Concept Prediction . arXiv:2605.20110

    Pith/arXiv arXiv 2026

  42. [42]

    Zhong, S.; Lu, Y.; Ning, J.; Wan, Y.; Feng, L.; Ao, Y.; Ribeiro, L. F. R.; Dreyer, M.; Ammirati, S.; and Xiong, C. 2026. SkillLearnBench: Benchmarking Continual Learning Methods for Agent Skill Generation on Real-World Tasks . arXiv:2604.20087

    Pith/arXiv arXiv 2026

  43. [43]

    A.; and Hu, X

    Zhou, X.; Liu, J.; Li, L.; Rossi, R. A.; and Hu, X. 2026. Counterfactual Trace Auditing of LLM Agent Skills . arXiv:2605.11946

    Pith/arXiv arXiv 2026

  44. [44]

    Zhu, J.; Mao, K.; Guo, Y.; He, D.; Xu, S.; Gu, S.; and Yue, Y. 2026. SkillCoach: Self-Evolving Rubrics for Evaluating and Enhancing Agentic Skill-Use . arXiv:2607.01874

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  45. [45]

    E.; Zhang, C.; Lin, C.; Wang, J.; Zhang, G.; and Zhou, W

    Zhu, K.; Li, H.; Wu, S.; Xing, T.; Ma, D.; Tang, X.; Liu, M.; Yang, J.; Liu, J.; Jiang, Y. E.; Zhang, C.; Lin, C.; Wang, J.; Zhang, G.; and Zhou, W. 2025. Scaling Test-time Compute for LLM Agents . arXiv:2506.12928

    Pith/arXiv arXiv 2025