Pith. sign in

REVIEW 4 major objections 5 minor 73 references

Teaching agents from verified recoveries beats scoring their failed actions: outcome-checked teacher continuations and first-divergence comparison internalize skills without retrieval at test time.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-31 22:49 UTC pith:JQ7CIAGH

load-bearing objection Solid empirical agent-training recipe with real gains and low overhead; the adaptive outcome-verification story is oversold relative to the paper’s own ablations. the 4 major comments →

arxiv 2607.27937 v1 pith:JQ7CIAGH submitted 2026-07-30 cs.AI

From Scoring to Acting: Outcome-Verified Comparative Self-Distillation for LLM Agents

Xu Xia , Jinghua Piao , Min Yang , Xiaochong Lan , Jiaju Chen , Yong Li This is my paper
classification cs.AI
keywords LLM agentson-policy self-distillationcapability internalizationoutcome verificationcomparative learningALFWorldWebShopskill-free deployment
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Language-model agents often get temporary boosts from external skills or memories, then lose them when that help is removed. This paper argues that the common fix—having a privileged teacher score the student’s own actions—is weak in long tasks, because a high local score need not lead to success and a low score does not show what to do instead. The authors propose Outcome-Verified Comparative Self-Distillation (OVCSD): when a skill-free student fails a whole batch of rollouts, those failures are arranged as a prefix tree; a skill-conditioned teacher is launched from deep shared student states and only continuations that actually finish the task are kept; learning then pushes up the teacher’s first differing action and pushes down the student’s failed alternatives at that shared state, then copies the teacher’s remaining completion steps into the skill-free policy. On household and shopping benchmarks across three model sizes, the resulting skill-free agents beat plain reinforcement learning and prior self-distillation methods by large margins while using under three percent extra privileged environment steps in training. A sympathetic reader cares because this is a concrete path from “borrow a skill once” to “keep the skill in the weights.”

Core claim

OVCSD shows that skill-free LLM agents learn more from environment-verified teacher recoveries, compared against failed student branches at the first state-aligned action divergence and followed by suffix distillation, than from teacher scores on student trajectories. Across ALFWorld and WebShop and three model scales, this yields the best skill-free success rates, with gains of up to 29.7 points on ALFWorld and 5.4 on WebShop over the strongest baselines, at under 3% privileged interaction overhead.

What carries the argument

Outcome-Verified Comparative Self-Distillation (OVCSD): organize all-failure student rollouts into a prefix tree; adaptively backtrack from deep shared states to launch a skill-conditioned teacher; accept only continuations with successful environment reward; at the first aligned divergence assign balanced local advantages (up for the teacher action, down for student alternatives) and KL-distill the post-divergence teacher suffix into the skill-free policy.

Load-bearing premise

A skill-equipped teacher started from the student’s own failed prefixes will often enough produce real task successes, and teaching only the first differing choice plus the remaining completion steps is enough to put that recovery into the skill-free agent.

What would settle it

On the same ALFWorld/WebShop skill-free eval, if ablating outcome verification or the first-divergence comparative term erased the reported gains over SDAR/GRPO, or if privileged steps rose well above a few percent without matching success, the central claim would fail; likewise if the method did not beat score-only self-distillation when teacher recoveries are rare.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Skill-free deployment can match or beat skill-at-inference agents if training uses verified recoveries rather than teacher scores alone.
  • Failed multi-turn rollouts should be treated as structured prefix trees, not discarded or reduced to a single terminal reward.
  • Credit for correction belongs at the first shared-state action split; completion behavior can be transferred separately via suffix distillation.
  • Privilege during training can stay a small fraction of the interaction budget if interventions are adaptive and outcome-filtered.
  • Largest gains should appear on compositional multi-step categories where one early wrong choice dooms the trajectory.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same divergence-plus-suffix pattern may transfer to other long-horizon tool-use or web agents where teachers can be conditioned on memories or demos rather than skills.
  • If shared prefixes are rare (as on WebShop relative to ALFWorld), backtracking cost and root launches will dominate—suggesting hybrid designs that also seed teachers from partial successes.
  • Outcome rejection of nearly half of skill-conditioned continuations implies score-based distillation silently trains on many false targets; filtering may matter as much as the comparative loss.
  • A natural next test is whether repeated OVCSD rounds shrink the need for any privileged teacher as the skill-free policy internalizes more recovery patterns.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Outcome-Verified Comparative Self-Distillation (OVCSD) for internalizing privileged skill-conditioned behavior into a skill-free LLM agent policy. Failed student rollouts are organized into a prefix tree; a skill-conditioned teacher is launched from adaptively chosen student-reached states; only environment-successful continuations are kept. Learning then combines (i) localized comparative advantages at the first state-aligned action divergence (Eqs. 6–7) with (ii) KL distillation of the post-divergence teacher suffix (Eq. 8). On ALFWorld and WebShop across Qwen 1.7B/3B/7B, OVCSD reports the best skill-free success, with up to +29.7 (ALFWorld) and +5.4 (WebShop) absolute points over the strongest self-distillation baselines, while claiming <3% privileged environment interaction. Supporting analyses include budget-matched curves, objective and intervention ablations, and policy-shift diagnostics.

Significance. If the gains are real and transferable, this is a useful contribution to capability internalization for multi-turn LLM agents: it moves beyond teacher scoring of student actions toward executed, outcome-checked teacher continuations, and it pairs a divergence-local corrective signal with completion-suffix distillation. Strengths include multi-scale evaluation, explicit interaction-budget accounting (Table 2, Figure 3), complementary objective ablations (Table 3), and forward diagnostics of teacher-action log-prob and KL concentration. A public code link is provided. The work is of clear interest to the agent RL / self-distillation community even if the relative importance of individual design choices needs sharper attribution.

major comments (4)
  1. [Table 4; Method (Adaptive Outcome-Verified Intervention); Abstract] Table 4 undercuts the paper’s central attribution of gains to adaptive, outcome-verified intervention from student prefixes. Over 50 updates, Random anchor 79.69, No verification 80.47, and full OVCSD 81.25 differ by ~1 point on ALFWorld validation, while No verification still trains on 43.9% task-failing branches. Deepest-first mainly lowers cost per verified branch (25.60 vs 33.00), not final success. By contrast, Table 3 shows Local-PG and Suffix-KL each add >20 points over GRPO and combine to +39.84. The abstract, intro, and method framing present prefix-tree adaptive intervention plus outcome filtering as load-bearing; the ablations suggest most of the lift may come from any successful privileged continuations plus the dual objective. Please either (a) reframe claims to match Table 4, or (b) provide stronger evidence (matched full-run ablations vs SDAR/Skill-SD, multiple seeds, WebS
  2. [Observations (Fig. 1); Table 1; Table 5] Observation 1 and the gain asymmetry raise a generalizability concern that is not adequately stress-tested. Shared prefixes cover 85.8% of trajectories in ALFWorld all-failure groups but only 4.9% on WebShop (Fig. 1b); correspondingly, the largest claimed win is +29.7 on ALFWorld vs +5.4 on WebShop over the strongest baseline. Table 5 further shows WebShop is often root-only (42.1% available depth 0). If the distinctive machinery (prefix tree, deepest-first backtrack, state-aligned divergence) fires rarely on WebShop, the method’s necessity is weakest where the second benchmark is used to support breadth. Please quantify intervention fire rate, verified-branch yield, and dual-objective contribution separately on WebShop, and discuss when the framework reduces to “privileged success distillation + GRPO.”
  3. [Eqs. (6)–(8); Alignment-Aware Comparative Learning; Table 3] The alignment-aware credit rule (Eqs. 6–7) is motivated by Observation 3 but only lightly validated as the right inductive bias. Advantages are set to ±1/√m at the first aligned divergent action, with all other student tokens retaining original (often zeroed) group advantages, then a separate top-k KL on the suffix (Eq. 8). Table 3 shows both terms help, but there is no comparison to natural alternatives that would falsify the “comparability breaks after first divergence” axiom—e.g., full-trajectory teacher advantages, divergence-plus-k-step local advantages, or reverse-KL / sequence-level likelihood on the whole teacher continuation. Without that, it remains unclear whether first-divergence localization is required or whether any outcome-verified teacher imitation would suffice. A compact comparison on the same 50-update protocol as Tables 3–4 would make the dual-objective design claim
  4. [Table 1; Experiments (Q1–Q3)] Main results (Table 1) and several diagnostics appear to be single-run point estimates without seed variance, confidence intervals, or statistical tests, while several category-level wins are mixed (e.g., OVCSD does not lead Pick/Clean at 3B or Look at 7B). Given that intervention fires on only 49/100 updates and yields 52 verified branches in the budget study, run-to-run variability could be material—especially for the 29.7-point claim over SDAR at 1.7B. Please report mean±std over multiple seeds for the micro-average (and for the Table 3–4 ablations), or justify single-run reporting and temper peak-margin language accordingly.
minor comments (5)
  1. [Abstract; Figure 2; Experiments] Typos and wording: “scoringactionsalongstudent-generatedtrajectories”, “interactio?” (Q2), “Te ac h e r” artifacts in Figure 2, and several missing spaces in the early pages. A full proofread pass is needed.
  2. [Figure 2] Figure 2 is information-dense but hard to read in grayscale; the local advantage schematic and suffix KL formula overlap visually. Consider splitting into intervention vs. learning panels.
  3. [Overall Objective; Experimental Setup] Hyperparameters λ_suf, N_T, top-k for suffix KL, and r_succ are listed as free choices but not given definitive values or sensitivity in the main text; point to a specific appendix table from the Method/Experiments sections.
  4. [Related Work] Related Work cites Skill-SD, SDAR, SIRI, and OPSD appropriately; a short explicit difference table (teacher acts? outcome filter? divergence-local credit?) in the main text would help readers, even if details are in the supplement.
  5. [Eq. (3)] Eq. (3): the indicator notation is somewhat compressed (z_fail definition); clarifying that intervention requires both all-fail and near-zero reward spread would help reproduction.

Circularity Check

0 steps flagged

No significant circularity: empirical agent-learning method with externally measured success rates, not identity-by-construction claims.

full rationale

OVCSD is an on-policy training procedure whose headline claims are skill-free success rates on ALFWorld and WebShop versus GRPO, OPSD, Skill-SD, and SDAR. Supervision is built from environment terminal rewards (acceptance R(τ^T_v) ≥ r_succ), first state-aligned divergences (Eq. 6–7), and KL to executed teacher suffix tokens (Eq. 8). These quantities are not fitted constants renamed as predictions, nor are they defined in terms of the reported success rates. Advantages are design normalizations balancing local positive/negative mass at divergence sites; they do not force the benchmark numbers by construction. Related-work citations (Skill-SD, SDAR, OPSD, GRPO) are baselines and framing, not load-bearing uniqueness theorems that make the result tautological. Ablation gaps (e.g., Table 4 intervention variants) raise attribution questions about which design piece drives gains, but that is an empirical-correctness issue, not circular derivation. The paper is self-contained against external interactive benchmarks; score 0.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 2 invented entities

Load-bearing content is methodological and empirical, not axiomatic physics. The claim rests on standard RL/distillation machinery (GRPO, KL distillation, shared student/teacher weights), domain assumptions about discrete multi-turn text MDPs and skill context, and several hand-chosen training knobs (group size, intervention depth policy, λ_suf, top-k, success threshold). No new physical entities; the ‘prefix tree’ and dual objective are engineering constructs.

free parameters (5)
  • λ_suf (suffix KL weight)
    Balances local comparative GRPO against post-divergence distillation in L_OVCSD; chosen as a training hyperparameter, not derived.
  • Student group size K and teacher continuation budget N_T = K=8 (reported budget tables)
    Control how often all-failure groups appear and how many privileged attempts are spent; nominal K=8 in budget analysis.
  • Top-k support for suffix KL
    Practical approximation of D_KL(q||p) over teacher top-k mass; implementation detail affecting the suffix objective.
  • Success threshold r_succ and reward standardization epsilons
    Gate which teacher continuations are accepted and when group advantages are zeroed (z_fail); environment/task-dependent choices.
  • Deepest-first backtracking / eligibility rules for intervention nodes = deepest-first (default OVCSD)
    Policy for choosing v in the prefix tree and when to fall back to root or individual trajectories; ablated vs random anchor but still a design choice.
axioms (5)
  • domain assumption Group-relative policy optimization (GRPO) with clipped likelihood ratios is a valid on-policy update for multi-turn LLM agents.
    Used as the RL backbone for both base student updates and divergence-local advantages (Method, Overall Objective).
  • domain assumption A skill-conditioned copy of the same frozen weights is a strictly stronger teacher whose successful environment trajectories are valid supervision for the skill-free student.
    Core privileged-teacher setup shared with Skill-SD/SDAR; OVCSD adds outcome filtering but still assumes skill context yields recoverable continuations (Eq. 4–5).
  • domain assumption Failed rollouts often share restorable intermediate environment states that can be snapshotted or replayed consistently.
    Observation 1 and prefix-tree construction; much weaker on WebShop (15.2% shared-prefix groups) than ALFWorld.
  • ad hoc to paper After the first state-aligned action divergence, step-wise action comparison is invalid and suffix behavior should be transferred by distribution matching rather than local advantages.
    Observation 3 motivates splitting Local-Â and L_suf; plausible but design-level, not a theorem.
  • standard math Standard math of KL divergence and advantage normalization (sqrt(m) balancing) as used in Eqs. 7–8.
    Ordinary probability/RL identities; not controversial.
invented entities (2)
  • OVCSD dual-objective alignment-aware distillation (divergence-local comparative advantages + post-divergence suffix KL) no independent evidence
    purpose: Convert outcome-verified teacher branches and failed student branches into skill-free policy updates without scoring-only supervision.
    Central methodological construct; not an external physical entity. Independent evidence is only the paper’s own ablations and benchmarks.
  • Failure prefix tree with adaptive deepest-first outcome-verified intervention no independent evidence
    purpose: Locate student-reached states for teacher takeover and accept only environment-successful continuations.
    Engineering structure built from rollouts; falsifiable only via the reported intervention statistics and task success, not outside this training loop.

pith-pipeline@v1.2.0-daily-grok45 · 18459 in / 3723 out tokens · 79086 ms · 2026-07-31T22:49:13.114740+00:00 · methodology

0 comments
read the original abstract

Recent work on LLM agents is shifting from external capability elicitation to capability internalization, enabling agents to retain useful skills without retrieval at inference time. On-policy self-distillation (OPSD) offers a promising direction, but many existing methods typically supervise students by scoring actions along student-generated trajectories. Such supervision has two limitations: teacher preferences are not validated by environment outcomes, and action-level scores underuse information from student rollouts, teacher rollouts, and their behavioral relationship. We therefore advocate outcome-verified teacher supervision and comparative learning over teacher-student trajectories. Based on this view, we propose Outcome-Verified Comparative Self-Distillation (OVCSD). OVCSD organizes failed student rollouts into a prefix tree, adaptively invokes a skill-conditioned teacher from student-reached states, and retains only outcome-verified successful continuations. It then applies localized comparative learning at the first state-aligned divergence and distills the post-divergence teacher suffix to transfer completion behavior. Experiments on ALFWorld and WebShop across three model scales show that OVCSD consistently outperforms skill-free RL and existing self-distillation baselines, achieving up to 29.7 and 5.4 absolute success-rate gains over the strongest baselines on ALFWorld and WebShop, respectively, while adding less than 3% privileged interaction during training.

Figures

Figures reproduced from arXiv: 2607.27937 by Jiaju Chen, Jinghua Piao, Min Yang, Xiaochong Lan, Xu Xia, Yong Li.

Figure 1
Figure 1. Figure 1: Preliminary observations on ALFWorld and WebShop. (a) Frequency of all-failure groups. (b) Shared-prefix coverage [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of OVCSD. Skill-free student rollouts provide the base group-relative update. For eligible all-failure groups, [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Skill-free validation success against cumulative en [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: shows a locally verified correction becoming reusable behavior. In the shirt task, student trajectories reach a product page whose size inventory stops at 3X although the Scored action GRPO OVCSD ∆ 95% CI Verified teacher +0.060 +0.342 +0.282 [0.17, 0.41] Failed student −0.054 −0.081 −0.027 [−0.24, 0.17] Prefix control −0.208 −0.162 +0.046 [−0.13, 0.23] States increased (%) 67.9 91.0 +23.1 [12.8, 34.6] [P… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

73 extracted references · 37 linked inside Pith

  1. [1]

    arXiv preprint arXiv:2602.08234 , year=

    Skillrl: Evolving agents via recursive skill-augmented reinforcement learning , author=. arXiv preprint arXiv:2602.08234 , year=

    Pith/arXiv arXiv

  2. [2]

    arXiv preprint arXiv:2604.02268 , year=

    Skill0: In-context agentic reinforcement learning for skill internalization , author=. arXiv preprint arXiv:2604.02268 , year=

    Pith/arXiv arXiv

  3. [3]

    arXiv preprint arXiv:1707.06347 , year=

    Proximal policy optimization algorithms , author=. arXiv preprint arXiv:1707.06347 , year=

    Pith/arXiv arXiv

  4. [4]

    arXiv preprint arXiv:2402.03300 , year=

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models , author=. arXiv preprint arXiv:2402.03300 , year=

    Pith/arXiv arXiv

  5. [5]

    arXiv preprint arXiv:2210.03629 , year=

    React: Synergizing reasoning and acting in language models , author=. arXiv preprint arXiv:2210.03629 , year=

    Pith/arXiv arXiv

  6. [6]

    arXiv preprint arXiv:2010.03768 , year=

    Alfworld: Aligning text and embodied environments for interactive learning , author=. arXiv preprint arXiv:2010.03768 , year=

    Pith/arXiv arXiv 2010

  7. [7]

    Advances in Neural Information Processing Systems , volume=

    Webshop: Towards scalable real-world web interaction with grounded language agents , author=. Advances in Neural Information Processing Systems , volume=

  8. [8]

    Proceedings of the 36th annual acm symposium on user interface software and technology , pages=

    Generative agents: Interactive simulacra of human behavior , author=. Proceedings of the 36th annual acm symposium on user interface software and technology , pages=

  9. [9]

    URL https://arxiv

    Reflexion: Language agents with verbal reinforcement learning, 2023 , author=. URL https://arxiv. org/abs/2303.11366 , volume=

    Pith/arXiv arXiv 2023

  10. [10]

    URL https://arxiv

    Voyager: An open-ended embodied agent with large language models, 2023 , author=. URL https://arxiv. org/abs/2305.16291 , volume=

    Pith/arXiv arXiv 2023

  11. [11]

    arXiv preprint arXiv:2508.03680 , year=

    Agent lightning: Train any ai agents with reinforcement learning , author=. arXiv preprint arXiv:2508.03680 , year=

    Pith/arXiv arXiv

  12. [12]

    arXiv preprint arXiv:2601.18734 , year=

    Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models , author=. arXiv preprint arXiv:2601.18734 , year=

    Pith/arXiv arXiv

  13. [13]

    arXiv preprint arXiv:2604.10674 , year=

    Skill-sd: Skill-conditioned self-distillation for multi-turn llm agents , author=. arXiv preprint arXiv:2604.10674 , year=

    Pith/arXiv arXiv

  14. [14]

    arXiv preprint arXiv:2605.15155 , year=

    Self-distilled agentic reinforcement learning , author=. arXiv preprint arXiv:2605.15155 , year=

    Pith/arXiv arXiv

  15. [15]

    International Conference on Learning Representations , volume=

    On-policy distillation of language models: Learning from self-generated mistakes , author=. International Conference on Learning Representations , volume=

  16. [16]

    Advances in neural information processing systems , volume=

    Toolformer: Language models can teach themselves to use tools , author=. Advances in neural information processing systems , volume=

  17. [17]

    2025 , month = oct, howpublished =

    Zhang, Barry and Lazuka, Keith and Murag, Mahesh , title =. 2025 , month = oct, howpublished =

    2025

  18. [18]

    Advances in neural information processing systems , volume=

    Retrieval-augmented generation for knowledge-intensive nlp tasks , author=. Advances in neural information processing systems , volume=

  19. [19]

    Transactions of the association for computational linguistics , volume=

    Lost in the middle: How language models use long contexts , author=. Transactions of the association for computational linguistics , volume=

  20. [20]

    International Conference on Machine Learning , pages=

    Large language models can be easily distracted by irrelevant context , author=. International Conference on Machine Learning , pages=. 2023 , organization=

    2023

  21. [21]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Same task, more tokens: the impact of input length on the reasoning performance of large language models , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  22. [22]

    International Conference on Learning Representations , volume=

    Making retrieval-augmented language models robust to irrelevant context , author=. International Conference on Learning Representations , volume=

  23. [23]

    arXiv preprint arXiv:2311.07911 , year=

    Instruction-following evaluation for large language models , author=. arXiv preprint arXiv:2311.07911 , year=

    Pith/arXiv arXiv

  24. [24]

    , author=

    MemGPT: towards LLMs as operating systems. , author=. 2023 , publisher=

    2023

  25. [25]

    Advances in Neural Information Processing Systems , volume=

    Dapo: An open-source llm reinforcement learning system at scale , author=. Advances in Neural Information Processing Systems , volume=

  26. [26]

    arXiv preprint arXiv:2503.09516 , year=

    Search-r1: Training llms to reason and leverage search engines with reinforcement learning , author=. arXiv preprint arXiv:2503.09516 , year=

    Pith/arXiv arXiv

  27. [27]

    arXiv preprint arXiv:2501.12948 , year=

    DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning , author=. arXiv preprint arXiv:2501.12948 , year=

    Pith/arXiv arXiv

  28. [28]

    arXiv preprint arXiv:2504.20073 , year=

    Ragen: Understanding self-evolution in llm agents via multi-turn reinforcement learning , author=. arXiv preprint arXiv:2504.20073 , year=

    Pith/arXiv arXiv

  29. [29]

    Advances in Neural Information Processing Systems , volume=

    Group-in-group policy optimization for llm agent training , author=. Advances in Neural Information Processing Systems , volume=

  30. [30]

    arXiv preprint arXiv:2606.02355 , year=

    SIRI: Self-Internalizing Reinforcement Learning with Intrinsic Skills for LLM Agent Training , author=. arXiv preprint arXiv:2606.02355 , year=

    Pith/arXiv arXiv

  31. [31]

    Advances in Neural Information Processing Systems , volume=

    Star: Bootstrapping reasoning with reasoning , author=. Advances in Neural Information Processing Systems , volume=

  32. [32]

    arXiv preprint arXiv:2308.08998 , year=

    Reinforced self-training (rest) for language modeling , author=. arXiv preprint arXiv:2308.08998 , year=

    Pith/arXiv arXiv

  33. [33]

    arXiv preprint arXiv:2312.06585 , year=

    Beyond human data: Scaling self-training for problem-solving with language models , author=. arXiv preprint arXiv:2312.06585 , year=

    Pith/arXiv arXiv

  34. [34]

    The Journal of Machine Learning Research , volume=

    Learning using privileged information: similarity control and knowledge transfer , author=. The Journal of Machine Learning Research , volume=. 2015 , publisher=

    2015

  35. [35]

    arXiv preprint arXiv:1511.03643 , year=

    Unifying distillation and privileged information , author=. arXiv preprint arXiv:1511.03643 , year=

    Pith/arXiv arXiv

  36. [36]

    arXiv preprint arXiv:1710.06542 , year=

    Asymmetric actor critic for image-based robot learning , author=. arXiv preprint arXiv:1710.06542 , year=

    Pith/arXiv arXiv

  37. [37]

    Science robotics , volume=

    Learning quadrupedal locomotion over challenging terrain , author=. Science robotics , volume=. 2020 , publisher=

    2020

  38. [38]

    2019 , eprint=

    Learning by Cheating , author=. 2019 , eprint=

    2019

  39. [39]

    arXiv preprint arXiv:2112.09332 , year=

    Webgpt: Browser-assisted question-answering with human feedback , author=. arXiv preprint arXiv:2112.09332 , year=

    Pith/arXiv arXiv

  40. [40]

    Proceedings of the 2018 conference on empirical methods in natural language processing , pages=

    HotpotQA: A dataset for diverse, explainable multi-hop question answering , author=. Proceedings of the 2018 conference on empirical methods in natural language processing , pages=

    2018

  41. [41]

    Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension , author=. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  42. [42]

    Transactions of the Association for Computational Linguistics , volume=

    Natural questions: a benchmark for question answering research , author=. Transactions of the Association for Computational Linguistics , volume=. 2019 , publisher=

    2019

  43. [43]

    Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: Long papers) , pages=

    When not to trust language models: Investigating effectiveness of parametric and non-parametric memories , author=. Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: Long papers) , pages=

  44. [44]

    Proceedings of the 28th International Conference on Computational Linguistics , pages=

    Constructing a multi-hop qa dataset for comprehensive evaluation of reasoning steps , author=. Proceedings of the 28th International Conference on Computational Linguistics , pages=

  45. [45]

    Transactions of the Association for Computational Linguistics , volume=

    MuSiQue: Multihop Questions via Single-hop Question Composition , author=. Transactions of the Association for Computational Linguistics , volume=. 2022 , publisher=

    2022

  46. [46]

    Findings of the Association for Computational Linguistics: EMNLP 2023 , pages=

    Measuring and narrowing the compositionality gap in language models , author=. Findings of the Association for Computational Linguistics: EMNLP 2023 , pages=

    2023

  47. [47]

    arXiv preprint arXiv:2604.17503 , year=

    SkillGraph: Self-Evolving Multi-Agent Collaboration with Multimodal Graph Topology , author=. arXiv preprint arXiv:2604.17503 , year=

    Pith/arXiv arXiv

  48. [48]

    arXiv preprint arXiv:2605.28791 , year=

    Skill-Conditioned Gated Self-Distillation for LLM Reasoning , author=. arXiv preprint arXiv:2605.28791 , year=

    Pith/arXiv arXiv

  49. [49]

    2024 , eprint=

    DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models , author=. 2024 , eprint=

    2024

  50. [50]

    NeurIPS 2022 Foundation Models for Decision Making Workshop , year=

    React: Synergizing reasoning and acting in language models , author=. NeurIPS 2022 Foundation Models for Decision Making Workshop , year=

    2022

  51. [51]

    arXiv preprint arXiv:2601.12538 , year=

    Agentic reasoning for large language models , author=. arXiv preprint arXiv:2601.12538 , year=

    Pith/arXiv arXiv

  52. [52]

    arXiv preprint arXiv:2512.13564 , year=

    Memory in the age of ai agents , author=. arXiv preprint arXiv:2512.13564 , year=

    Pith/arXiv arXiv

  53. [53]

    arXiv preprint arXiv:2509.25140 , year=

    Reasoningbank: Scaling agent self-evolving with reasoning memory , author=. arXiv preprint arXiv:2509.25140 , year=

    Pith/arXiv arXiv

  54. [54]

    arXiv preprint arXiv:2602.02488 , year=

    RLAnything: Forge Environment, Policy, and Reward Model in Completely Dynamic RL System , author=. arXiv preprint arXiv:2602.02488 , year=

    arXiv

  55. [55]

    arXiv preprint arXiv:2605.06130 , year=

    Skill1: Unified evolution of skill-augmented agents via reinforcement learning , author=. arXiv preprint arXiv:2605.06130 , year=

    Pith/arXiv arXiv

  56. [56]

    arXiv preprint arXiv:2606.29502 , year=

    UCOB: Learning to Utilize and Evolve Agentic Skills via Credit-Aware On-Policy Bidirectional Self-Distillation , author=. arXiv preprint arXiv:2606.29502 , year=

    Pith/arXiv arXiv

  57. [57]

    Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Memory-r1: Enhancing large language model agents to manage and utilize memories via reinforcement learning , author=. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  58. [58]

    arXiv preprint arXiv:2506.15841 , year=

    Mem1: Learning to synergize memory and reasoning for efficient long-horizon agents , author=. arXiv preprint arXiv:2506.15841 , year=

    Pith/arXiv arXiv

  59. [59]

    arXiv preprint arXiv:2203.02155 , year=

    Training language models to follow instructions with human feedback , author=. arXiv preprint arXiv:2203.02155 , year=

    Pith/arXiv arXiv

  60. [60]

    Advances in Neural Information Processing Systems , volume=

    Lima: Less is more for alignment , author=. Advances in Neural Information Processing Systems , volume=

  61. [61]

    arXiv preprint arXiv:2606.15912 , year=

    On-Policy Distillation with Curriculum Turn-level Guidance for Multi-turn Agents , author=. arXiv preprint arXiv:2606.15912 , year=

    arXiv

  62. [62]

    arXiv preprint arXiv:2001.08361 , year=

    Scaling laws for neural language models , author=. arXiv preprint arXiv:2001.08361 , year=

    Pith/arXiv arXiv 2001

  63. [63]

    arXiv preprint arXiv:2203.15556 , year=

    Training compute-optimal large language models , author=. arXiv preprint arXiv:2203.15556 , year=

    Pith/arXiv arXiv

  64. [64]

    Findings of the Association for Computational Linguistics: ACL 2024 , pages=

    Agenttuning: Enabling generalized agent abilities for llms , author=. Findings of the Association for Computational Linguistics: ACL 2024 , pages=

    2024

  65. [65]

    arXiv preprint arXiv:2509.17567 , year=

    Limi: Less is more for agency , author=. arXiv preprint arXiv:2509.17567 , year=

    arXiv

  66. [66]

    Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    Atlas: Agent tuning via learning critical steps , author=. Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    2025

  67. [67]

    Forty-third International Conference on Machine Learning , year=

    Student-Centered Distillation Narrows the Agentic Gap Between Small and Large LLMs , author=. Forty-third International Conference on Machine Learning , year=

  68. [68]

    Proceedings of the fourteenth international conference on artificial intelligence and statistics , pages=

    A reduction of imitation learning and structured prediction to no-regret online learning , author=. Proceedings of the fourteenth international conference on artificial intelligence and statistics , pages=. 2011 , organization=

    2011

  69. [69]

    arXiv preprint arXiv:2512.14895 , year=

    Imitation Learning for Multi-turn LM Agents via On-policy Expert Corrections , author=. arXiv preprint arXiv:2512.14895 , year=

    arXiv

  70. [70]

    arXiv preprint arXiv:2501.17161 , year=

    Sft memorizes, rl generalizes: A comparative study of foundation model post-training , author=. arXiv preprint arXiv:2501.17161 , year=

    Pith/arXiv arXiv

  71. [71]

    arXiv preprint arXiv:2603.25562 , year=

    Revisiting on-policy distillation: Empirical failure modes and simple fixes , author=. arXiv preprint arXiv:2603.25562 , year=

    Pith/arXiv arXiv

  72. [72]

    arXiv preprint arXiv:2605.20061 , year=

    Rewarding Beliefs, Not Actions: Consistency-Guided Credit Assignment for Long-Horizon Agents , author=. arXiv preprint arXiv:2605.20061 , year=

    Pith/arXiv arXiv

  73. [73]

    arXiv preprint arXiv:2605.26844 , year=

    Not all disagreement is learnable: Token teachability in on-policy distillation , author=. arXiv preprint arXiv:2605.26844 , year=

    Pith/arXiv arXiv