Pith. sign in

REVIEW 2 major objections 2 minor 2 cited by

OPID extracts hierarchical skills from on-policy trajectories to supply dense token-level supervision that supplements sparse outcome rewards in agent RL.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-06-26 05:05 UTC pith:EUSI362Q

load-bearing objection OPID adds hierarchical skill distillation from on-policy rollouts with critical-first routing to densify advantages in agentic RL, but the distribution match after injection is not obviously preserved. the 2 major comments →

arxiv 2606.26790 v1 pith:EUSI362Q submitted 2026-06-25 cs.CL

OPID: On-Policy Skill Distillation for Agentic Reinforcement Learning

Shuo Yang , Jinyang Wu , Zhengxi Lu , Yuhao Shen , Fan Zhang , Lang Feng , Shuai Zhang , Haoran Luo
show 3 more authors
Zheng Lian Zhengqi Wen Jianhua Tao
This is my paper
classification cs.CL
keywords on-policy skill distillationagentic reinforcement learninghindsight supervisionhierarchical skillscritical-first routinglanguage agentstoken-level advantage
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces OPID to address the lack of intermediate guidance in outcome-based RL for language agents. It extracts episode-level skills for global workflows and step-level skills for critical decisions directly from completed on-policy rollouts, then routes the appropriate skill into the history for re-scoring. The resulting log-probability shift creates a self-distillation advantage that combines with the outcome advantage during optimization. This keeps RL as the main objective while adding hindsight supervision that stays matched to the current policy's distribution. Experiments on ALFWorld, WebShop, and Search-based QA show gains in performance, sample efficiency, and robustness over baselines.

Core claim

OPID represents trajectory hindsight as hierarchical skills extracted from on-policy trajectories: episode-level skills capture global workflows or failure-avoidance rules while step-level skills capture local decision knowledge at critical timesteps. A critical-first routing mechanism applies step-level skills when critical decisions are identified and defaults to episode-level skills otherwise. The selected skill is injected into the interaction history so the old policy can re-score the sampled response under both original and skill-augmented contexts; the log-probability shift yields a token-level self-distillation advantage that is added to the outcome advantage for policy optimization.

What carries the argument

The critical-first routing mechanism that selects between step-level and episode-level skills extracted from completed on-policy trajectories, together with the token-level self-distillation advantage computed from the log-probability shift under skill-augmented context.

Load-bearing premise

Skills extracted from completed trajectories remain distribution-matched to the current policy's state distribution during multi-turn interaction and the critical-first routing correctly identifies when to apply step-level versus episode-level skills.

What would settle it

An experiment in which multi-turn state distributions diverge enough that the injected skills produce a negative distillation advantage and final performance falls below the outcome-only RL baseline.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Agent performance, sample efficiency, and robustness improve on ALFWorld, WebShop, and Search-based QA compared with outcome-only RL and prior skill-distillation methods.
  • RL remains the primary training objective while dense, distribution-matched supervision is added at the token level.
  • No external skill memories or retrieved privileged context are required because skills come directly from the agent's own on-policy trajectories.
  • The combination of hierarchical skill representation and critical-first routing supplies guidance at both global and local decision scales.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same on-policy extraction pattern could be tested in non-language sequential decision tasks where sparse rewards also limit intermediate credit assignment.
  • If the routing heuristic generalizes, similar hierarchical distillation might reduce reliance on large external retrieval stores in other agent frameworks.
  • The log-probability shift technique offers a concrete way to turn hindsight analysis into an advantage signal without changing the underlying RL algorithm.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The manuscript proposes OPID, an on-policy skill distillation method for agentic RL. Skills (episode-level for global workflows and step-level for local decisions) are extracted from completed on-policy trajectories; a critical-first router selects the skill type; the chosen skill is injected into the history; and the log-probability shift between the original and skill-augmented contexts on the identical response supplies a token-level distillation advantage that is added to the outcome advantage for policy optimization. Experiments on ALFWorld, WebShop and Search-based QA report gains in performance, sample efficiency and robustness versus outcome-only RL and prior skill-distillation baselines. Code is released.

Significance. If the distribution-matching property of the token-level advantage holds, OPID supplies a practical route to dense, on-policy hindsight supervision while retaining outcome RL as the primary objective. This could improve training stability and efficiency for multi-turn language agents without external skill stores. The public code release supports reproducibility.

major comments (2)
  1. [Abstract and method description of advantage construction] Abstract / method description of advantage construction: the token-level advantage is defined directly as the log-probability shift obtained by re-scoring the same sampled response under the original versus skill-injected context drawn from the identical trajectory. This construction is internal to the policy’s own outputs and lacks an external validation or parameter-free derivation showing independence from the outcome signal; the resulting quantity may therefore reduce to a fitted difference rather than independent supervision.
  2. [Abstract and description of critical-first routing and skill injection] Abstract / description of critical-first routing and skill injection: the claim that the injected skill preserves the original state occupancy measure (and thus on-policy status) is load-bearing for the distribution-matched advantage. Skill injection necessarily alters the context for subsequent turns, and the additional selection step performed by the router is not shown to leave the state distribution unchanged; no derivation or diagnostic is supplied that the post-injection occupancy matches the extraction distribution.
minor comments (2)
  1. [Abstract] The abstract states that skills are represented as 'hierarchical skills' but provides no concrete description of their format, extraction procedure, or storage; this detail is needed for readers to assess implementation cost and reproducibility.
  2. [Experimental results] No error bars, run counts, or statistical tests are mentioned in the abstract for the reported improvements; these should be added to the experimental section.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We appreciate the referee's constructive feedback on our work. Below we provide point-by-point responses to the major comments. We have revised the manuscript to improve clarity on the advantage construction and the on-policy properties.

read point-by-point responses

  1. Referee: [Abstract and method description of advantage construction] Abstract / method description of advantage construction: the token-level advantage is defined directly as the log-probability shift obtained by re-scoring the same sampled response under the original versus skill-injected context drawn from the identical trajectory. This construction is internal to the policy’s own outputs and lacks an external validation or parameter-free derivation showing independence from the outcome signal; the resulting quantity may therefore reduce to a fitted difference rather than independent supervision.

    Authors: The token-level advantage is constructed as the difference in log-probabilities for the exact same token sequence under two different conditioning contexts: the original history versus the history augmented with the extracted skill. This difference quantifies the policy's sensitivity to the skill information for that particular response. Because the outcome advantage depends only on the scalar terminal reward while this quantity depends on the full token-level likelihood shift induced by the skill, the two are mathematically distinct. We will revise the method description to explicitly state this separation and include a short derivation showing that the distillation advantage corresponds to an on-policy estimate of the advantage under the skill-conditioned policy, independent of the reward function. Additionally, we will report the correlation between the two advantage signals in the experiments to empirically support their complementarity. revision: partial

  2. Referee: [Abstract and description of critical-first routing and skill injection] Abstract / description of critical-first routing and skill injection: the claim that the injected skill preserves the original state occupancy measure (and thus on-policy status) is load-bearing for the distribution-matched advantage. Skill injection necessarily alters the context for subsequent turns, and the additional selection step performed by the router is not shown to leave the state distribution unchanged; no derivation or diagnostic is supplied that the post-injection occupancy matches the extraction distribution.

    Authors: We clarify that skill injection and routing are performed exclusively during the offline advantage computation phase on already-collected trajectories; they do not modify the online rollout distribution, which remains strictly on-policy with respect to the current policy parameters. The critical-first router operates on the completed trajectory to decide which skill to extract and inject for re-scoring purposes only. Consequently, the state occupancy measure relevant to policy sampling is unaffected. We acknowledge that the manuscript does not include an explicit derivation or diagnostic plot for the post-injection context distribution, and we will add a dedicated paragraph in Section 3.3 explaining the separation between sampling and advantage computation, along with a small-scale diagnostic verifying that the re-scored responses correspond to states visited under the original policy. revision: yes

Circularity Check

0 steps flagged

No significant circularity detected

full rationale

The paper defines a method in which skills extracted from on-policy trajectories are injected to produce a log-probability shift that is then used as a token-level advantage. This construction is presented explicitly as the source of the dense supervision signal rather than as a derived theorem or prediction that reduces to its own inputs by construction. No equations appear in the provided text that equate a claimed result to a fitted parameter or self-referential definition, and no self-citations are invoked as load-bearing uniqueness theorems. The central claim is supported by experiments on external environments (ALFWorld, WebShop, Search-based QA), rendering the derivation self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only; no explicit free parameters, axioms, or invented entities are stated. The method implicitly assumes that trajectory hindsight can be reliably decomposed into skills without additional fitted components.

pith-pipeline@v0.9.1-grok · 5851 in / 1077 out tokens · 19400 ms · 2026-06-26T05:05:01.406246+00:00 · methodology

0 comments
read the original abstract

Outcome-based reinforcement learning provides a stable optimization backbone for language agents, but its sparse trajectory-level rewards provide little guidance on which intermediate decisions should be reinforced or suppressed. On-policy self-distillation offers dense token-level supervision, yet existing skill-conditioned variants often rely on external skill memories or retrieved privileged context, which are costly to maintain and can be mismatched with the state distribution induced by the current policy in multi-turn interaction. We propose \textbf{OPID} (\textbf{O}n-\textbf{P}olicy Sk\textbf{i}ll \textbf{D}istillation), a framework that extracts skill supervision directly from completed on-policy trajectories. OPID represents trajectory hindsight as hierarchical skills: episode-level skills capture global workflows or failure-avoidance rules, while step-level skills capture local decision knowledge at critical timesteps. A critical-first routing mechanism uses step-level skills when critical decisions are identified and falls back to episode-level skills as default guidance otherwise. The selected skill is injected into the interaction history, allowing the old policy to re-score the same sampled response under both original and skill-augmented contexts. The resulting log-probability shift yields a token-level self-distillation advantage, which is combined with the outcome advantage for policy optimization. OPID thus preserves RL as the primary training objective while introducing dense, distribution-matched hindsight supervision. Experiments on ALFWorld, WebShop and Search-based QA demonstrate that OPID generally improves agent performance, sample efficiency, and robustness over outcome-only RL and existing skill-distillation baselines. Our code is available at https://github.com/jinyangwu/OPID/tree/main.

Figures

Figures reproduced from arXiv: 2606.26790 by Fan Zhang, Haoran Luo, Jianhua Tao, Jinyang Wu, Lang Feng, Shuai Zhang, Shuo Yang, Yuhao Shen, Zheng Lian, Zhengqi Wen, Zhengxi Lu.

Figure 1
Figure 1. Figure 1: Overall performance comparison. We compare OPID with training-free prompting methods, outcome-only RL, and skill-distillation baselines on ALFWorld, Search-based QA, and WebShop. OPID achieves the strongest average performance on ALFWorld and WebShop while remaining competitive on Search-based QA. indicate whether a trajectory succeeds, but not which intermediate decisions caused the outcome. This limitati… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of OPID. Starting from completed on-policy trajectories, OPID extracts hier￾archical hindsight skills and routes the most relevant skill to each decision, prioritizing step-level skills at critical states. The policy then re-scores the same sampled response with and without the routed skill, turning the token-wise log-probability difference into a dense skill advantage that com￾plements the episod… view at source ↗
Figure 3
Figure 3. Figure 3: Training dynamics of OPID and GRPO. We report Qwen2.5-3B-Instruct training on ALFWorld. Translucent curves denote raw measurements and solid curves denote smoothed trends. 2026a) are self-distillation or skill-distillation baselines that introduce auxiliary token-level or skill￾conditioned supervision during training. Rows marked with ∗ indicate validation with skills, follow￾ing the setting described in t… view at source ↗
Figure 5
Figure 5. Figure 5: Cross-domain generalization on ALFWorld Unseen. OPID improves the aver￾age success rate over GRPO and shows particu￾larly large gains on Look and Heat. OPID internalizes skills instead of depending on them at inference. The results further show that OPID gains from internalizing hindsight skills into the policy, rather than relying on skill prompts at inference time. Training directly with retrieved skills… view at source ↗
Figure 6
Figure 6. Figure 6: Qualitative comparison on ALFWorld. For the task “clean some spatula and put it in diningtable,” the GRPO-trained agent hallucinates a nonexistent target object, substitutes a spoon for the spatula, and fails to complete the final placement within the step limit. In contrast, OPID follows a coherent locate-clean-place workflow, grounding each action in the current observation and completing the task in six… view at source ↗
Figure 7
Figure 7. Figure 7: Average critical steps per sequence on ALFWorld. The curve reports how many timesteps are selected by the analyzer for step-level hindsight skills in each trajectory. The rela￾tively small number of critical steps indicates that OPID applies local skill supervision selectively, while relying on episode-level skills as default guidance for non-critical decisions. C.3 TRAINING DIAGNOSTICS AND SKILL EXTRACTIO… view at source ↗
Figure 8
Figure 8. Figure 8: Magnitudes of episode-level and skill-guided advantage signals during OPID train￾ing. Episode abs advantage measures the mean absolute advantage from group-relative outcome rewards, while skill abs advantage measures the mean absolute advantage induced by skill-guided log-probability shifts. The comparison shows how OPID combines sparse trajectory-level feedback with dense skill-conditioned supervision thr… view at source ↗
Figure 9
Figure 9. Figure 9: Prompt of analyzer. 31 [PITH_FULL_IMAGE:figures/full_fig_p031_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: A full trajectory of OPID on ALFWorld Example 1. [PITH_FULL_IMAGE:figures/full_fig_p032_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: A full trajectory of OPID on ALFWorld Example 2. [PITH_FULL_IMAGE:figures/full_fig_p033_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: A full trajectory of OPID on Search-QA Example 1. [PITH_FULL_IMAGE:figures/full_fig_p034_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: A full trajectory of OPID on Search-QA Example 2. [PITH_FULL_IMAGE:figures/full_fig_p035_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: A full trajectory of OPID on Webshop Example 1. [PITH_FULL_IMAGE:figures/full_fig_p036_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: A full trajectory of OPID on Webshop Example 2. [PITH_FULL_IMAGE:figures/full_fig_p037_15.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Group-Reflective Self-Distillation for Agentic Reinforcement Learning

    cs.AI 2026-07 conditional novelty 6.0

    Contrasting a policy’s own success and failure reflections yields turn-level credit that beats GRPO and skill-based self-distillation on agentic tasks.

  2. LLM-as-a-Coach: Experiential Learning for Non-Verifiable Tasks

    cs.LG 2026-07 conditional novelty 6.0

    Training a language model by distilling a coach's written experiential knowledge beats training on a scalar rubric score for open-ended tasks, with better out-of-distribution transfer.

Reference graph

Works this paper leans on

58 extracted references · 33 canonical work pages · cited by 2 Pith papers · 32 internal anchors

  1. [1]

    DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

    DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models , author=. arXiv preprint arXiv:2402.03300 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  2. [2]

    International Conference on Learning Representations , year=

    On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes , author=. International Conference on Learning Representations , year=

  3. [3]

    Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models

    Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models , author=. arXiv preprint arXiv:2601.18734 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  4. [4]

    Self-Distillation Zero: Self-Revision Turns Binary Rewards into Dense Supervision

    Self-Distillation Zero: Self-Revision Turns Binary Rewards into Dense Supervision , author=. arXiv preprint arXiv:2604.12002 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  5. [5]

    Beyond Examples: High-level Automated Reasoning Paradigm in In-Context Learning via MCTS

    Beyond examples: High-level automated reasoning paradigm in in-context learning via mcts , author=. arXiv preprint arXiv:2411.18478 , year=

    work page Pith review arXiv

  6. [6]

    Maestro: Reinforcement Learning to Orchestrate Hierarchical Model-Skill Ensembles

    Maestro: Reinforcement Learning to Orchestrate Hierarchical Model-Skill Ensembles , author=. arXiv preprint arXiv:2605.22177 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  7. [7]

    The Fourteenth International Conference on Learning Representations , year=

    IterResearch: Rethinking Long-Horizon Agents with Interaction Scaling , author=. The Fourteenth International Conference on Learning Representations , year=

  8. [8]

    OdysseyArena: Benchmarking Large Language Models For Long-Horizon, Active and Inductive Interactions

    OdysseyArena: Benchmarking Large Language Models For Long-Horizon, Active and Inductive Interactions , author=. arXiv preprint arXiv:2602.05843 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  9. [9]

    The Landscape of Agentic Reinforcement Learning for LLMs: A Survey

    The landscape of agentic reinforcement learning for llms: A survey , author=. arXiv preprint arXiv:2509.02547 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  10. [10]

    Carlos E Jimenez and John Yang and Alexander Wettig and Shunyu Yao and Kexin Pei and Ofir Press and Karthik R Narasimhan , booktitle=

  11. [11]

    Self-Distilled Agentic Reinforcement Learning

    Self-Distilled Agentic Reinforcement Learning , author=. arXiv preprint arXiv:2605.15155 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  12. [12]

    Spark: Strategic Policy-Aware Exploration via Dynamic Branching for Long-Horizon Agentic Learning

    Spark: Strategic Policy-Aware Exploration via Dynamic Branching for Long-Horizon Agentic Learning , author=. arXiv preprint arXiv:2601.20209 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  13. [13]

    Large Language Model Agent: A Survey on Methodology, Applications and Challenges

    Large language model agent: A survey on methodology, applications and challenges , author=. arXiv preprint arXiv:2503.21460 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  14. [14]

    Atlas: Orchestrating Heterogeneous Models and Tools for Multi-Domain Complex Reasoning

    Atlas: Orchestrating Heterogeneous Models and Tools for Multi-Domain Complex Reasoning , author=. arXiv preprint arXiv:2601.03872 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  15. [15]

    SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization

    SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization , author=. arXiv preprint arXiv:2604.02268 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  16. [16]

    Skill-SD: Skill-Conditioned Self-Distillation for Multi-turn LLM Agents

    Skill-SD: Skill-Conditioned Self-Distillation for Multi-turn LLM Agents , author=. arXiv preprint arXiv:2604.10674 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  17. [17]

    Self-Distilled RLVR

    Self-Distilled RLVR , author=. arXiv preprint arXiv:2604.03128 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  18. [18]

    Reinforcement Learning via Self-Distillation

    Reinforcement Learning via Self-Distillation , author=. arXiv preprint arXiv:2601.20802 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  19. [19]

    ALFWorld: Aligning Text and Embodied Environments for Interactive Learning

    ALFWorld: Aligning Text and Embodied Environments for Interactive Learning , author=. arXiv preprint arXiv:2010.03768 , year=

    work page internal anchor Pith review Pith/arXiv arXiv 2010

  20. [20]

    Advances in Neural Information Processing Systems , year=

    WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents , author=. Advances in Neural Information Processing Systems , year=

  21. [21]

    Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning

    Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning , author=. arXiv preprint arXiv:2503.09516 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  22. [22]

    SWE-bench: Can Language Models Resolve Real-World GitHub Issues?

    SWE-bench: Can Language Models Resolve Real-World GitHub Issues? , author=. arXiv preprint arXiv:2310.06770 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  23. [23]

    International Conference on Learning Representations , year=

    Let's Verify Step by Step , author=. International Conference on Learning Representations , year=

  24. [24]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics , year=

    Math-Shepherd: Verify and Reinforce LLMs Step-by-Step without Human Annotations , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics , year=

  25. [25]

    Process Reinforcement through Implicit Rewards

    Process Reinforcement through Implicit Rewards , author=. arXiv preprint arXiv:2502.01456 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  26. [26]

    International Conference on Learning Representations , year=

    MiniLLM: Knowledge Distillation of Large Language Models , author=. International Conference on Learning Representations , year=

  27. [27]

    Advances in Neural Information Processing Systems , year=

    Reflexion: Language Agents with Verbal Reinforcement Learning , author=. Advances in Neural Information Processing Systems , year=

  28. [28]

    Transactions on Machine Learning Research , year=

    Voyager: An Open-Ended Embodied Agent with Large Language Models , author=. Transactions on Machine Learning Research , year=

  29. [29]

    SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning

    SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning , author=. arXiv preprint arXiv:2602.08234 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  30. [30]

    Qwen2.5 Technical Report

    Qwen2.5 Technical Report , author=. arXiv preprint arXiv:2412.15115 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  31. [31]

    Qwen3 Technical Report

    Qwen3 Technical Report , author=. arXiv preprint arXiv:2505.09388 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  32. [32]

    Transactions of the Association for Computational Linguistics , volume=

    Natural Questions: A Benchmark for Question Answering Research , author=. Transactions of the Association for Computational Linguistics , volume=

  33. [33]

    Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics , pages=

    TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension , author=. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics , pages=

  34. [34]

    Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics , pages=

    When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories , author=. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics , pages=

  35. [35]

    Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing , pages=

    HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering , author=. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing , pages=

    2018

  36. [36]

    Proceedings of the 28th International Conference on Computational Linguistics , pages=

    Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps , author=. Proceedings of the 28th International Conference on Computational Linguistics , pages=

  37. [37]

    Transactions of the Association for Computational Linguistics , volume=

    MuSiQue: Multihop Questions via Single-hop Question Composition , author=. Transactions of the Association for Computational Linguistics , volume=

  38. [38]

    Findings of the Association for Computational Linguistics: EMNLP 2023 , pages=

    Measuring and Narrowing the Compositionality Gap in Language Models , author=. Findings of the Association for Computational Linguistics: EMNLP 2023 , pages=

    2023

  39. [39]

    Distilling the Knowledge in a Neural Network

    Distilling the Knowledge in a Neural Network , author =. arXiv preprint arXiv:1503.02531 , year =. 1503.02531 , archivePrefix =

    work page internal anchor Pith review Pith/arXiv arXiv

  40. [40]

    IEEE Transactions on Information Theory , volume =

    Divergence Measures Based on the Shannon Entropy , author =. IEEE Transactions on Information Theory , volume =. 1991 , doi =

    1991

  41. [41]

    Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing , pages =

    Sequence-Level Knowledge Distillation , author =. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing , pages =. 2016 , publisher =

    2016

  42. [42]

    Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics , series =

    A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning , author =. Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics , series =. 2011 , publisher =

    2011

  43. [43]

    Gu, Yuxian and Dong, Li and Wei, Furu and Huang, Minlie , booktitle =

  44. [44]

    Proximal Policy Optimization Algorithms

    Proximal Policy Optimization Algorithms , author =. arXiv preprint arXiv:1707.06347 , year =. 1707.06347 , archivePrefix =

    work page internal anchor Pith review Pith/arXiv arXiv

  45. [45]

    The Annals of Mathematical Statistics , volume =

    On Information and Sufficiency , author =. The Annals of Mathematical Statistics , volume =. 1951 , doi =

    1951

  46. [46]

    2026 , eprint =

    Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes , author =. 2026 , eprint =

    2026

  47. [47]

    Thinking Machines Lab: Connectionism , year =

    On-Policy Distillation , author =. Thinking Machines Lab: Connectionism , year =

  48. [48]

    Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe

    Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe , author =. arXiv preprint arXiv:2604.13016 , year =

    work page internal anchor Pith review Pith/arXiv arXiv

  49. [49]

    Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes

    Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes , author =. arXiv preprint arXiv:2603.25562 , year =

    work page internal anchor Pith review Pith/arXiv arXiv

  50. [50]

    On-Policy Context Distillation for Language Models

    On-Policy Context Distillation for Language Models , author =. arXiv preprint arXiv:2602.12275 , year =

    work page internal anchor Pith review Pith/arXiv arXiv

  51. [51]

    Oh, Minjae and Song, Sangjun and Choi, Gyubin and Choi, Yunho and Jo, Yohan , journal =

  52. [52]

    2026 , month = jun, day =

    2026

  53. [53]

    Double: Breaking the Acceleration Limit via Double Retrieval Speculative Parallelism

    Double: Breaking the Acceleration Limit via Double Retrieval Speculative Parallelism , author=. arXiv preprint arXiv:2601.05524 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  54. [54]

    RobotEQ: Transitioning from Passive Intelligence to Active Intelligence in Embodied AI

    RobotEQ: Transitioning from Passive Intelligence to Active Intelligence in Embodied AI , author=. arXiv preprint arXiv:2605.06234 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  55. [55]

    AgentBench: Evaluating LLMs as Agents

    AgentBench: Evaluating LLMs as Agents , author=. arXiv preprint arXiv:2308.03688 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  56. [56]

    WebArena: A Realistic Web Environment for Building Autonomous Agents

    WebArena: A Realistic Web Environment for Building Autonomous Agents , author=. arXiv preprint arXiv:2307.13854 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  57. [57]

    Mind2Web: Towards a Generalist Agent for the Web

    Mind2Web: Towards a Generalist Agent for the Web , author=. arXiv preprint arXiv:2306.06070 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  58. [58]

    VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks

    VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks , author=. arXiv preprint arXiv:2401.13649 , year=

    work page internal anchor Pith review Pith/arXiv arXiv