Pith. sign in

REVIEW 4 major objections 5 minor 45 references

A reusable agent skill can be optimized from unlabeled task instances alone, using an LLM judge's pairwise comparisons of executions, and on closed-ended benchmarks this approaches or exceeds ground-truth-based skill optimization.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 00:23 UTC pith:QRM53F7X

load-bearing objection SSO is a genuinely new behavior-level GT-free skill optimizer with strong closed-ended results; the open-ended win rates and missing judge calibration are the main questions. the 4 major comments →

arxiv 2607.28777 v1 pith:QRM53F7X submitted 2026-07-30 cs.CL

Self-Supervised Skill Optimization

Siran Peng , Cuiyu Yang , Tianyu Fu , Tianshuo Zhang , Haoyuan Zhang , Weisong Zhao , Anyang Su , Minghui Wu
show 4 more authors
Huiying Li Xiangyu Zhu Chenxu Zhao Zhen Lei
This is my paper
classification cs.CL
keywords self-supervised skill optimizationground-truth-free optimizationreusable agent skillsskill documentspairwise LLM judgebehavior-level evidence aggregationprompt optimizationfrozen LLM agents
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper's method, Self-Supervised Skill Optimization (SSO), asks whether a reusable skill document for a frozen LLM agent can be improved using only unlabeled task instances, with no ground-truth labels, scores, rewards, or task-specific evaluators. The paper argues yes, provided optimization is driven by pairwise judge comparisons of executions: complete skill probes run alongside the current skill, an LLM judge says which execution is better, and a behavior extractor blind to the judge identifies what changed. Equivalent behaviors are clustered across tasks, evidence for and against each behavior is aggregated and normalized, and a new skill is rendered from the top-ranked behaviors, accepted only if it wins a validation comparison. Across six closed-ended benchmarks and three open-ended dialogue tasks, SSO beats existing ground-truth-free prompt optimizers and approaches, sometimes exceeding, the strongest ground-truth-based skill optimizer. If correct, this makes skill adaptation practical wherever unlabeled instances and a comparative judge exist but labeled feedback does not.

Core claim

The paper's central claim is that a single natural-language skill document can be optimized for a frozen agent using only unlabeled task instances and an LLM judge's pairwise preferences. SSO runs the current skill and several complete skill probes on the same tasks, gets an anchored win/loss/tie decision for each probe-versus-current comparison, then uses a behavior extractor that never sees the judge's decisions to record observable behavioral differences and their activation states. Equivalent behavior descriptions are clustered, and each decisive comparison contributes one normalized unit of evidence split among the behaviors that changed, with sign set by whether the preferred execution

What carries the argument

The central unit is the behavior record: a natural-language description of an observable behavioral difference paired with an activation vector marking that behavior as present, absent, or unclear in each execution. The argument is carried by the evidence-aggregation formula e = y * d / |A|, where y is the judge's preference, d is +1 or -1 depending on whether the behavior appears or disappears in the preferred execution, and |A| is the number of changed behavior clusters in that comparison. This splits one normalized unit of evidence per decisive comparison, preventing any one execution from dominating because it differs in many behaviors; clustering lets the same behavior accumulate eviden

Load-bearing premise

The entire optimization signal rests on the untested premise that an LLM judge's pairwise preferences over unlabeled executions are a reliable proxy for task quality, and the paper does not report calibration of the judge's agreement with ground truth.

What would settle it

Hold out ground-truth labels from the optimizer but keep them available to the experimenter, run SSO on a closed-ended benchmark, and measure the agreement between the judge's pairwise decisions and the true outcomes; if agreement is near chance yet the same judge reports large pairwise win-rate gains for the accepted skill, the reported improvement is likely judge self-consistency rather than genuine task quality. A direct variant would flip the judge's preferences and check whether accepted updates still appear to improve.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Reusable skills can be improved in settings where ground-truth labels, task scores, rewards, or task-specific evaluators are unavailable, as long as unlabeled task instances and a comparative judge exist.
  • Behavior-level evidence aggregation beats candidate-level selection: the rendered skill outperforms the best individual probe, so optimization should not be reduced to choosing among generated candidates.
  • A judge-based validation gate prevents regressions, allowing iterative refinement to continue across optimization steps without external supervision.
  • The approach transfers across several frozen target models and both closed-ended and open-ended multi-turn dialogue tasks, so it is not tied to a single model or answer format.
  • On closed-ended benchmarks, ground-truth-free skill optimization can approach and in one tested configuration exceed the best ground-truth-based skill optimizer, narrowing the practical gap between supervised and unsupervised skill tuning.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because the paper's open-ended evaluation also relies on an LLM judge of the same general kind as the optimization judge, some gains may reflect the agent conforming to the judge's stylistic preferences rather than improving on an external notion of quality; a small human or ground-truth calibration set would settle how much is externally valid.
  • The behavior-extraction, clustering, and normalized-evidence steps are not specific to a single skill document; they could plausibly update a whole skill library or combine evidence across heterogeneous execution formats such as answers, trajectories, artifacts, and terminal states.
  • A biased but self-consistent judge could encode its bias into the validation gate; a testable variant would gate updates with a different judge or occasional low-cost human checks.
  • SSO could be hybridized with sparse supervision: a few labeled examples used only to calibrate the judge might make the loop robust without requiring dense ground truth, though the paper does not explore this.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. SSO is a framework for optimizing a reusable natural-language skill document for a frozen LLM agent using only unlabeled task instances. At each step it generates K complete skill probes from a subset of the current executions, runs them on the same training batch, and uses an LLM judge to compare each probe execution against the anchor (Eq. 5). A behavior extractor that does not see judge decisions identifies behavioral differences (Eq. 6), which are clustered (Eq. 8); evidence is aggregated per changed cluster with normalization (Eqs. 9–13), and the top-ranked behaviors are rendered into a new skill (Eq. 15). The candidate is accepted only if it wins a judge-based validation comparison (Eqs. 17–18). Experiments across three target models, six closed-ended benchmarks, and three MT-Bench-101 open-ended tasks report that SSO outperforms GT-free prompt optimizers SPO and PDO, and approaches or exceeds the GT-based SkillOpt on some model/benchmark combinations.

Significance. SSO introduces a novel optimization unit—behavior-level evidence aggregated across probes and tasks—rather than candidate-level selection, and addresses a practically important setting where GT feedback is unavailable. The design is principled: the behavior extractor is blinded to judge decisions, and evidence normalization controls for variable numbers of changed behaviors. The paper provides component ablations (Table 3), hyperparameter sensitivity sweeps (Figure 3), and an analysis distinguishing rendered candidates from the best probes (Figure 4), all of which strengthen the internal validity of the closed-ended results. However, the central validity question—whether the LLM judge's pairwise preferences are a reliable proxy for task quality—is not empirically addressed. This is especially problematic for the open-ended results (Table 2), where the evaluation judge is from the same family as the optimization judge. The paper is potentially significant, but the missing calibration and variance reporting are needed before the contribution is fully established.

major comments (4)
  1. [§Experimental Setup (Metrics), Table 2] The open-ended win rates are computed by an LLM judge (GPT-5.5) that is also the optimizer-side judge. The optimization signal, acceptance gate, and evaluation metric are therefore drawn from the same judge family, creating a self-consistency confound. No human evaluation or independent judge is reported. To support the claim that SSO is preferred on open-ended tasks, please report agreement with human judgments on the MT-Bench-101 tasks, or at least evaluate with a different-model judge and show that the two judges correlate.
  2. [§Methodology, Eqs. (5), (17), Algorithm 1 lines 9, 39–42] The method's core assumption is that pairwise LLM preferences are a reliable proxy for task quality. No calibration of the judge against ground truth is given anywhere in the paper. On the closed-ended benchmarks, where GT is available, compute judge–GT agreement (e.g., accuracy of judge choices versus correct answers, or rank correlation with task scores) on a held-out set. On open-ended tasks, report human agreement. Without this calibration, the closed-ended gains could be an incidental effect of judge bias that happens to align with GT on these tasks, and the open-ended results are uninterpretable as evidence of task quality.
  3. [Table 1, Experimental Setup] All reported numbers are point estimates with no error bars or repeated seeds. Given the stochastic components (batch sampling, probe generation, clustering, rendering), single-run results cannot establish that SSO reliably outperforms SPO/PDO, particularly for small differences such as 78.8 vs 76.3 on SearchQA with GPT-5.4-mini or 89.6 vs 88.0 on DocVQA with Qwen3.5-4B. Please report means and standard deviations across at least three seeds, or include significance tests.
  4. [§Experimental Setup (Baselines)] The adaptation of SPO and PDO to optimize complete skill documents is not described; the paper states only "We adapt both prompt optimizers to optimize complete skills." Because these baselines are the primary GT-free competitors and their numbers are claimed to be reproduced, the fairness of the comparison is not verifiable. Provide the exact adaptation details (prompt templates, number of candidates, validation procedure, and compute budget) or release the code.
minor comments (5)
  1. [General] The paper refers repeatedly to the supplementary material for schedules, prompts, and additional results, but the arXiv version does not include the supplementary. Ensure the supplementary file is attached in the submission.
  2. [Table 1 footnote] The note says SPO and PDO are reproduced, while other baselines are reported from (Yang et al. 2026a). This cross-paper comparison is acceptable for context but should be stated earlier in the text so readers do not infer all baselines were run under identical harnesses.
  3. [Table 2] The win-rate table reports no sample size or confidence interval. Please state the number of comparisons per cell (e.g., number of tasks × number of repeats) so the reader can assess the precision of the 50–85% win rates.
  4. [Figure 3] The text claims the default settings achieve the best performance in all sweeps, but the figure is small and not all points are annotated with values. Add numeric labels or a table of the swept values and corresponding scores.
  5. [Introduction] Minor typographical issue: "Skil-lOpt" appears split across lines in the Introduction; it should be "SkillOpt" throughout.

Circularity Check

1 steps flagged

Partially self-referential: open-ended win rates are judged by the same LLM-judge preference signal used to train and select SSO; hard-accuracy closed-ended results remain independent.

specific steps
  1. other [Algorithm 1 lines 39-42; Experiments: Metrics and Open-ended Tasks (Table 2); Implementation Details]
    "v_m ← f_judge(x_m, r^n_m, r^o_m) ... S_{t+1} ← eSt+1 if W_t > Lt; otherwise St (Algorithm 1). For MT-Bench-101, an LLM judge compares SSO with each baseline. We report the percentage of comparisons in which SSO is preferred as the pairwise win rate ... All optimizer-side LLM modules (f_gen, f_judge, f_beh, f_cluster, and f_render) use GPT-5.5."

    The Table 2 open-ended metric is the same construct SSO maximizes: pairwise LLM-judge preference. Eq. 5 (training) and Eq. 17/line 42 (validation acceptance) both use f_judge; the implementation specifies GPT-5.5 for f_judge and no other judge is named for MT-Bench-101, which has no GT labels/scores. Thus the reported open-ended 'win rates' are not independent evidence of task quality; they measure how well the skill satisfies the very preference function used to select updates. Closed-ended Table 1 uses hard accuracy, so the core empirical claim is not wholly circular, but the open-ended portion lacks independent grounding.

full rationale

Walking the derivation, no equation reduces to a fitted parameter or to a load-bearing self-citation. The SSO update rule (Eqs. 5-18) is an algorithmic pipeline, not a derivation of one quantity from another. The one self-referential part is empirical: the open-ended evaluation (Table 2) uses an LLM judge and no external/human check is reported, while the paper's only specified judge family (GPT-5.5) is also f_judge used at lines 9 and 39 of Algorithm 1. This makes the open-ended win rates a consistency measure with the optimization objective. The closed-ended evaluations, by contrast, use hard accuracy, official task accuracy, and success rate, so the main result that SSO approaches or exceeds SkillOpt on some model/benchmark combinations is independently grounded. The only author-overlap citation (UPA, Peng et al. 2026) is in related work and is not load-bearing. Hence score 4, not higher: there is partial self-referential evaluation for open-ended tasks, but no definitional or fitted-parameter circularity in the core closed-ended claim.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 2 invented entities

The method depends on several LLM-based modules and hand-set hyperparameters whose reliability is not independently verified. The main free choices are K, |H_t|, U, and per-dataset schedules; the judge/extractor/clusterer/renderer are all assumed to work as intended. No external falsifiable entity is introduced.

free parameters (5)
  • K (probe count) = 3
    Number of complete skill probes generated per step; hand-set and confirmed by sensitivity sweep in Figure 3; all main results use K=3.
  • |H_t| (history subset size) = 8
    Task-execution pairs used to condition probe generation; hand-set; Figure 3 sweep selects 8 as default.
  • U (behavior update budget) = 4
    Maximum number of behavior clusters retained for each model update; hand-set; Figure 3 sweep.
  • Optimization schedule (T, |X_t|, |V_t|) = T=8-40, |X_t|=20-40, |V_t|=16-40 per dataset
    Per-dataset values chosen by the authors to keep scale comparable to SkillOpt; exact mapping is only in supplementary, so main results depend on unpublished tuning.
  • Optimizer-side model = GPT-5.5 for all LLM modules
    All f_gen/f_judge/f_beh/f_cluster/f_render use GPT-5.5; when the target model is also GPT-5.5, the judge and executor are the same model, which especially affects open-ended self-evaluation.
axioms (4)
  • domain assumption LLM judge pairwise preferences are a reliable optimization signal without ground truth.
    Used in Eq. (5) and Algorithm 1 lines 9, 39-42 to generate and accept updates; never calibrated against GT or human preference.
  • domain assumption The behavior extractor and clusterer faithfully identify present/absent/unclear behaviors and group semantically equivalent descriptions.
    Eqs. (6)-(8); if extraction or clustering is noisy, the evidence aggregation in Eq. (13) is built on unreliable units.
  • domain assumption The renderer translates ranked behavior intents into a new complete skill without semantic drift or unsupported additions.
    Eq. (15); the output skill is never verified to actually contain the intended behavioral changes beyond the judge's win/loss count.
  • domain assumption An unlabeled validation batch win-count gate selects skills that generalize to unseen tasks.
    Eq. (18); assumes the validation batch is representative and that judge preferences on it correlate with target performance.
invented entities (2)
  • Behavior record (β_ij, α_ij) no independent evidence
    purpose: Atomic evidence unit linking a textual behavior description to activation states in each execution; allows evidence to be aggregated across probes and tasks.
    Internal abstraction; its correctness depends entirely on the LLM extractor/clusterer and is not falsifiable outside the paper.
  • Evidence cluster with intent (encourage/suppress) and scores Q_c/E_c no independent evidence
    purpose: Groups semantically equivalent behaviors and ranks them for rendering the next skill.
    Internal bookkeeping construct; no external handle; validity is established only through final benchmark performance.

pith-pipeline@v1.3.0-alltime-deepseek · 13671 in / 13956 out tokens · 137053 ms · 2026-08-03T00:23:16.894960+00:00 · methodology

0 comments
read the original abstract

Agent skills provide frozen large language model (LLM) agents with reusable procedural guidance, and recent work shows that such skills can be optimized with ground-truth (GT) feedback. Many applications, however, lack GT labels, task scores, rewards, or reliable task-specific evaluators. We therefore introduce Self-Supervised Skill Optimization (SSO), a comparative framework that learns a reusable skill from unlabeled task instances alone. At each step, SSO runs the current skill on an unlabeled batch, uses a subset of the resulting executions to generate complete skill probes, and runs the probes on the same batch. An LLM judge compares the resulting answers, trajectories, artifacts, or terminal states. A separate behavior extractor identifies behavioral differences without seeing the judge's decisions. SSO uses these decisions to aggregate evidence for and against the observed behaviors across instances. It then ranks the behaviors by the resulting evidence and renders a new complete skill from the highest-ranked behaviors. The update is accepted only if the new skill outperforms the current one on an unlabeled validation set. SSO outperforms existing GT-free prompt optimizers on both closed-ended and open-ended tasks. On closed-ended benchmarks, it approaches and sometimes exceeds the strongest GT-based skill optimizer without using any GT feedback.

Figures

Figures reproduced from arXiv: 2607.28777 by Anyang Su, Chenxu Zhao, Cuiyu Yang, Haoyuan Zhang, Huiying Li, Minghui Wu, Siran Peng, Tianshuo Zhang, Tianyu Fu, Weisong Zhao, Xiangyu Zhu, Zhen Lei.

Figure 1
Figure 1. Figure 1: GT-based skill optimization and SSO. Existing methods use GT feedback to guide skill updates and selec￾tion. SSO instead derives optimization feedback from pair￾wise comparisons of observable executions on unlabeled task instances and uses no GT feedback during optimization. improves performance on a separate validation set. Its results show that reusable skills can be optimized across models, benchmarks, … view at source ↗
Figure 2
Figure 2. Figure 2: The SSO optimization loop. SSO executes the current skill and K complete probes on the same unlabeled training batch. An LLM judge then produces anchored pairwise decisions. A behavior extractor identifies observable differences without seeing the decisions, and a separate clusterer groups equivalent descriptions. SSO combines the decisions with behavior activations to rank behaviors and render a new skill… view at source ↗
Figure 3
Figure 3. Figure 3: Average test performance with GPT-5.4-mini across [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Test accuracy of rendered candidates and post-hoc [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

45 extracted references · 19 linked inside Pith

  1. [1]

    Chain-of-Thought Prompting Elicits Reasoning in Large Language Models , volume =

    Wei, Jason and Wang, Xuezhi and Schuurmans, Dale and Bosma, Maarten and ichter, brian and Xia, Fei and Chi, Ed and Le, Quoc V and Zhou, Denny , booktitle =. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models , volume =

  2. [2]

    Proceedings of the International Conference on Learning Representations (ICLR) , year=

    Large language models are human-level prompt engineers , author=. Proceedings of the International Conference on Learning Representations (ICLR) , year=

  3. [3]

    gradient descent

    Automatic prompt optimization with “gradient descent” and beam search , author=. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages=

    2023

  4. [4]

    Proceedings of the International Conference on Learning Representations (ICLR) , volume=

    Large language models as optimizers , author=. Proceedings of the International Conference on Learning Representations (ICLR) , volume=

  5. [5]

    Proceedings of the International Conference on Learning Representations (ICLR) , volume=

    Promptagent: Strategic planning with language models enables expert-level prompt optimization , author=. Proceedings of the International Conference on Learning Representations (ICLR) , volume=

  6. [6]

    Proceedings of the International Conference on Machine Learning (ICML) , pages=

    Promptbreeder: Self-Referential Self-Improvement via Prompt Evolution , author=. Proceedings of the International Conference on Machine Learning (ICML) , pages=. 2024 , organization=

    2024

  7. [7]

    differentiation

    Textgrad: Automatic "differentiation" via text , author=. arXiv preprint arXiv:2406.07496 , year=

    Pith/arXiv arXiv

  8. [8]

    First Workshop on Foundations of Reasoning in Language Models , year=

    GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning , author=. First Workshop on Foundations of Reasoning in Language Models , year=

  9. [9]

    arXiv preprint arXiv:2502.06855 , year=

    Self-supervised prompt optimization , author=. arXiv preprint arXiv:2502.06855 , year=

    Pith/arXiv arXiv

  10. [10]

    Findings of the Association for Computational Linguistics: ACL 2026 , pages=

    Llm prompt duel optimizer: Efficient label-free prompt optimization , author=. Findings of the Association for Computational Linguistics: ACL 2026 , pages=

    2026

  11. [11]

    arXiv preprint arXiv:2601.23273 , year=

    UPA: Unsupervised Prompt Agent via Tree-Based Search and Selection , author=. arXiv preprint arXiv:2601.23273 , year=

    Pith/arXiv arXiv

  12. [12]

    Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena , volume =

    Zheng, Lianmin and Chiang, Wei-Lin and Sheng, Ying and Zhuang, Siyuan and Wu, Zhanghao and Zhuang, Yonghao and Lin, Zi and Li, Zhuohan and Li, Dacheng and Xing, Eric and Zhang, Hao and Gonzalez, Joseph and Stoica, Ion , booktitle =. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena , volume =

  13. [13]

    Proceedings of the International Conference on Learning Representations (ICLR) , year=

    ReAct: Synergizing Reasoning and Acting in Language Models , author=. Proceedings of the International Conference on Learning Representations (ICLR) , year=

  14. [14]

    Toolformer: Language Models Can Teach Themselves to Use Tools , volume =

    Schick, Timo and Dwivedi-Yu, Jane and Dessi, Roberto and Raileanu, Roberta and Lomeli, Maria and Hambro, Eric and Zettlemoyer, Luke and Cancedda, Nicola and Scialom, Thomas , booktitle =. Toolformer: Language Models Can Teach Themselves to Use Tools , volume =

  15. [15]

    Transactions on Machine Learning Research , year =

    Voyager: An Open-Ended Embodied Agent with Large Language Models , author=. Transactions on Machine Learning Research , year =

  16. [16]

    arXiv preprint arXiv:2602.12670 , year=

    SkillsBench: Benchmarking how well agent skills work across diverse tasks , author=. arXiv preprint arXiv:2602.12670 , year=

    Pith/arXiv arXiv

  17. [17]

    arXiv preprint arXiv:2602.20867 , year=

    SoK: Agentic Skills--Beyond Tool Use in LLM Agents , author=. arXiv preprint arXiv:2602.20867 , year=

    Pith/arXiv arXiv

  18. [18]

    arXiv preprint arXiv:2603.25158 , year=

    Trace2skill: Distill trajectory-local lessons into transferable agent skills , author=. arXiv preprint arXiv:2603.25158 , year=

    Pith/arXiv arXiv

  19. [19]

    arXiv preprint arXiv:2603.01145 , year=

    Autoskill: Experience-driven lifelong learning via skill self-evolution , author=. arXiv preprint arXiv:2603.01145 , year=

    arXiv

  20. [20]

    arXiv preprint arXiv:2601.22758 , year=

    AutoRefine: From Trajectories to Reusable Expertise for Continual LLM Agent Refinement , author=. arXiv preprint arXiv:2601.22758 , year=

    arXiv

  21. [21]

    arXiv preprint arXiv:2604.03964 , year=

    Skillfoundry: Building self-evolving agent skill libraries from heterogeneous scientific resources , author=. arXiv preprint arXiv:2604.03964 , year=

    Pith/arXiv arXiv

  22. [22]

    arXiv preprint arXiv:2603.02766 , year=

    Evoskill: Automated skill discovery for multi-agent systems , author=. arXiv preprint arXiv:2603.02766 , year=

    Pith/arXiv arXiv

  23. [23]

    arXiv preprint arXiv:2604.08618 , year=

    Skillforge: Forging domain-specific, self-evolving agent skills in cloud technical support , author=. arXiv preprint arXiv:2604.08618 , year=

    Pith/arXiv arXiv

  24. [24]

    arXiv preprint arXiv:2604.01687 , year=

    Coevoskills: Self-evolving agent skills via co-evolutionary verification , author=. arXiv preprint arXiv:2604.01687 , year=

    Pith/arXiv arXiv

  25. [25]

    arXiv preprint arXiv:2605.10999 , year=

    Skillgen: Verified inference-time agent skill synthesis , author=. arXiv preprint arXiv:2605.10999 , year=

    Pith/arXiv arXiv

  26. [26]

    arXiv preprint arXiv:2605.23904 , year=

    Skillopt: Executive strategy for self-evolving agent skills , author=. arXiv preprint arXiv:2605.23904 , year=

    Pith/arXiv arXiv

  27. [27]

    arXiv preprint arXiv:2606.22330 , year=

    Hypothesis-Driven Skill Optimization for LLM Agents , author=. arXiv preprint arXiv:2606.22330 , year=

    Pith/arXiv arXiv

  28. [28]

    arXiv preprint arXiv:2606.14239 , year=

    SkillAudit: Ground-Truth-Free Skill Evolution via Paired Trajectory Auditing , author=. arXiv preprint arXiv:2606.14239 , year=

    arXiv

  29. [29]

    arXiv preprint arXiv:2604.27660 , year=

    From Context to Skills: Can Language Models Learn from Context Skillfully? , author=. arXiv preprint arXiv:2604.27660 , year=

    Pith/arXiv arXiv

  30. [30]

    arXiv preprint arXiv:2604.27488 , year=

    Skills-coach: A self-evolving skill optimizer via training-free grpo , author=. arXiv preprint arXiv:2604.27488 , year=

    Pith/arXiv arXiv

  31. [31]

    arXiv preprint arXiv:2606.06416 , year=

    Unsupervised Skill Discovery for Agentic Data Analysis , author=. arXiv preprint arXiv:2606.06416 , year=

    Pith/arXiv arXiv

  32. [32]

    arXiv preprint arXiv:2606.10546 , year=

    SkillAxe: Sharpening LLM-Authored Agent Skills Through Evaluation-Guided Self-Refinement , author=. arXiv preprint arXiv:2606.10546 , year=

    Pith/arXiv arXiv

  33. [33]

    arXiv preprint arXiv:2606.06741 , year=

    OpenSkill: Open-World Self-Evolution for LLM Agents , author=. arXiv preprint arXiv:2606.06741 , year=

    Pith/arXiv arXiv

  34. [34]

    arXiv e-prints , pages=

    Evolving Agents in the Dark: Retrospective Harness Optimization via Self-Preference , author=. arXiv e-prints , pages=

  35. [35]

    arXiv preprint arXiv:1704.05179 , year=

    Searchqa: A new q&a dataset augmented with context from a search engine , author=. arXiv preprint arXiv:1704.05179 , year=

    Pith/arXiv arXiv

  36. [36]

    SpreadsheetBench: Towards Challenging Real World Spreadsheet Manipulation , volume =

    Ma, Zeyao and Zhang, Bohan and Zhang, Jing and Yu, Jifan and Zhang, Xiaokang and Zhang, Xiaohan and Luo, Sijia and Wang, Xi and Tang, Jie , booktitle =. SpreadsheetBench: Towards Challenging Real World Spreadsheet Manipulation , volume =

  37. [37]

    arXiv preprint arXiv:2603.08655 , year=

    Officeqa pro: An enterprise benchmark for end-to-end grounded reasoning , author=. arXiv preprint arXiv:2603.08655 , year=

    arXiv

  38. [38]

    , title =

    Mathew, Minesh and Karatzas, Dimosthenis and Jawahar, C.V. , title =. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) , month =. 2021 , pages =

    2021

  39. [39]

    arXiv preprint arXiv:2604.01754 , year=

    Livemathematicianbench: A live benchmark for mathematician-level reasoning with proof sketches , author=. arXiv preprint arXiv:2604.01754 , year=

    arXiv

  40. [40]

    Proceedings of the International Conference on Learning Representations (ICLR) , year =

    ALFWorld: Aligning Text and Embodied Environments for Interactive Learning , author=. Proceedings of the International Conference on Learning Representations (ICLR) , year =

  41. [41]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL) , pages=

    Mt-bench-101: A fine-grained benchmark for evaluating large language models in multi-turn dialogues , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL) , pages=

  42. [42]

    First International KDD Workshop on Prompt Optimization , year=

    Tournament of Prompts: Evolving LLM Instructions Through Structured Debates and Elo Ratings , author=. First International KDD Workshop on Prompt Optimization , year=

  43. [43]

    Large Language Models are Zero-Shot Reasoners , volume =

    Kojima, Takeshi and Gu, Shixiang (Shane) and Reid, Machel and Matsuo, Yutaka and Iwasawa, Yusuke , booktitle =. Large Language Models are Zero-Shot Reasoners , volume =

  44. [44]

    Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective , volume =

    Feng, Guhao and Zhang, Bohang and Gu, Yuntian and Ye, Haotian and He, Di and Wang, Liwei , booktitle =. Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective , volume =

  45. [45]

    2026 , url =

    Introducing. 2026 , url =

    2026