Pith. sign in

REVIEW 3 major objections 5 minor 42 references

An agent can teach itself better turn-by-turn credit by contrasting its own successes and failures on the same task.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-31 18:33 UTC pith:LCTZHSOZ

load-bearing objection Solid agentic-RL recipe with real aggregate wins; the credit story is weaker than the tables, but the paper is honest enough about that to still be worth engaging. the 3 major comments →

arxiv 2607.28076 v1 pith:LCTZHSOZ submitted 2026-07-30 cs.AI cs.LG

Group-Reflective Self-Distillation for Agentic Reinforcement Learning

Binbin Zheng , Zijun Xie , Guanqun Zhao , Enlei Gong , Xing Ma , Xiaoliang Fu , Zeyu Chen This is my paper
classification cs.AI cs.LG
keywords agentic reinforcement learningself-distillationverifiable rewardsturn-level credit assignmentgroup-reflective guidanceGRPOLLM agents
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Sparse end-of-episode rewards tell an interactive language agent only whether a whole trajectory worked, not which decisions mattered. This paper argues that the agent can supply that missing detail from its own experience: after sampling a group of attempts at one prompt, it reflects on each verified rollout, then a frozen copy of itself contrasts the successful and failed reflections into a compact DO/AVOID guide. That guide is used only in training as privileged context so a self-teacher can reweight turn-level advantages without flipping the verifier’s sign. Across embodied household tasks, search-based QA, and web shopping, and across three model sizes, the method beats outcome-only RL and prior skill-based self-distillation, and transfers better to unseen ALFWorld tasks. The practical claim is that capability-aligned, outcome-discriminative guidance does not need an external skill library or a stronger analyzer—only the policy’s own mixed group of verified rollouts.

Core claim

Group-Reflective Self-Distillation (GRSD) shows that policy-native reflections on an on-policy group of verified successes and failures can be synthesized into group-level privileged guidance that refines trajectory-level advantages into turn-specific credit while preserving the verifier-determined learning direction, yielding stronger aggregate agent performance and better unseen-task generalization than GRPO and prior skill-based self-distillation baselines.

What carries the argument

Group-reflective guidance construction plus bounded turn-level advantage modulation: the policy reflects on each verified rollout (Stage A); a stop-gradient snapshot contrasts successful vs failed reflections into DO/AVOID guidance (Stage B); teacher–student log-likelihood gaps under that guidance modulate GRPO advantages with a mask and clip so the update sign cannot reverse.

Load-bearing premise

The gap in how much the frozen policy likes a turn with versus without the hindsight group guide is assumed to mark that turn’s real contribution to success or failure, not just stylistic agreement with the guide.

What would settle it

Train the same backbones with mixed-outcome groups ablated or with modulation driven by random/shuffled guidance: if validation gains on ALFWorld, Search QA, and WebShop, and the Unseen split, disappear while reflection quality stays high, the credit-refinement claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Agentic RL with only terminal verifiers can densify credit from the policy’s own success/failure contrast without external skill memories at inference.
  • Guidance that stays inside the current policy’s reflection ability should transfer better to unseen tasks than skills written by a stronger external model.
  • When a rollout group lacks both successes and failures, training safely falls back to plain trajectory-level GRPO.
  • Jointly training a small reflection objective can improve later guidance quality and late-stage task success without changing the inference prompt.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same success–failure contrast loop may help tool-use and multi-hop agents wherever binary verifiers exist but intermediate labels do not.
  • As policies get stronger and groups become mostly successful, the mixed-outcome mask may fire less often, so the method’s benefit could shrink unless exploration or group size is adjusted.
  • Bounded positive reweighting of GRPO suggests other hindsight natural-language critiques could plug into the same modulation slot if they stay detached and sign-preserving.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Group-Reflective Self-Distillation (GRSD) for multi-turn LLM agents trained with verifiable outcome rewards. For each prompt, the policy generates reflections on a group of on-policy rollouts (Stage A); a stop-gradient snapshot then contrasts successful vs. failed reflections into compact DO/AVOID guidance zx (Stage B). A self-teacher uses zx as privileged context to compute a turn-level log-likelihood gap Δi,t, which modulates the GRPO trajectory advantage into turn-specific advantages bAi,t while preserving verifier sign (λbG≤1). The joint objective also trains reflection quality via an external rubric judge. Experiments on ALFWorld, Search-based QA, and WebShop with Qwen3-1.7B / Qwen2.5-3B / 7B report consistent aggregate gains over GRPO, OPSD, RLSD, Skill-SD, and SDAR, plus stronger ALFWorld Unseen generalization. Ablations, training dynamics, hyperparameter sweeps, and a theory appendix (bounded positive reweighting of the GRPO surrogate) support the design.

Significance. If the results hold under tighter controls, GRSD is a useful contribution to agentic RLVR: it addresses sparse trajectory-level rewards with policy-native, contrastive natural-language guidance and requires no privileged context at inference. Strengths include multi-environment, multi-scale evaluation; component ablations (Table 2); dynamics and hyperparameter analyses; an explicit sign-preservation constraint with supporting Propositions 2–3; and released code. The group success–failure contrast and capability-aligned reflection loop are clearer than retrieval-based or single-trajectory external-skill pipelines. Even if Δ is best read as a hindsight compatibility signal rather than causal credit, a reproducible recipe that improves late-stage optimization and unseen-task transfer is of practical interest to the agent post-training community.

major comments (3)
  1. [§3.2–3.3, Eqs. 5–11; Appendix A.5] Central mechanism vs. construction of zx (§3.2–3.3, Eqs. 5–6, 9–11; Appendix A.5): Stage B builds zx from the full mixed set S+x ∪ S−x, so when turn t of trajectory i is re-scored, si can already appear in the DO/AVOID text. Appendix A.5 correctly states that Δi,t is a hindsight-conditioned compatibility signal, not causal credit, and that zx may depend on the scored trajectory. Under self-inclusion, a large positive qi,t can mean agreement with a summary that partly describes the same rollout rather than contribution to the verified outcome. The abstract and §3.3 still frame the method as refining turn-level credit assignment. This is load-bearing for the explanatory claim that GRSD beats Skill-SD/SDAR via outcome-discriminative credit refinement. Please either (i) report a leave-one-out / held-out-trajectory guidance variant (build zx without si when scoring τi) as a primary or ablatio
  2. [Table 2; Eqs. 9–11] Table 2 ablations show that removing group-reflective guidance, group aggregation, failed reflections, or turn-level modulation hurts aggregate scores, but they do not isolate genuine decision-level credit from (a) self-inclusion leakage or (b) generic training-time reweighting that amplifies turns stylistically consistent with any plausible DO/AVOID text. A minimal additional control—e.g., scrambled/shuffled guidance, guidance from a different prompt, or random sign-preserving turn weights with the same |bqi,t| budget—would test whether the teacher–student gap tracks task-relevant structure. Without such a control, the weakest assumption in the pipeline (that sign(Ai)Δi,t is a reliable proxy for turn usefulness) remains under-validated relative to the strength of the credit-assignment narrative.
  3. [§3.2 Eq. 4; §1 contributions; Figure 4] Policy-native framing vs. external judge (§3.2 Eq. 4; §4.1; Appendix C): Reflections and zx are generated by the policy snapshot, which is a real distinction from external skill extractors, but Stage A learning depends on a fixed DeepSeek-V4-Flash judge scoring reflections on {0,1,2,3}. Figure 4 compares policy-native vs. external reflection generation, not judge-free reflection learning. Please quantify sensitivity to the judge (weaker/open judge, or α=0 with frozen reflection behavior beyond the existing α sweep) and state clearly in the main text that reflection supervision is not fully self-contained. This does not invalidate the method but currently overstates the “without external models for skill generation” contrast relative to the full training loop.
minor comments (5)
  1. [Table 1] Table 1: several subset wins are non-uniform (e.g., Search QA Bam for Qwen2.5-3B GRSD 16.2 vs GRPO 64.1). A short discussion of when group contrast helps or hurts would strengthen interpretation of “strongest aggregate” claims.
  2. [Figure 1] Figure 1 caption and panel labels are dense; ensure consistent metric names (Score vs Succ) match Table 1 and that error bars or multi-seed variance are reported if available.
  3. [Appendix A.1; §3.2] Proposition 1’s idealized uncorrelated noise model is carefully scoped in the appendix; a one-sentence pointer in §3.2 that Stage B is not claimed to recover μ1−μ0 would help main-text readers.
  4. [Abstract; §1] Typographical/spacing artifacts in the abstract and early pages (“Reinforcementlearningwithverifiablerewards”, “andmodelscalesdemonstratethat”) should be cleaned in the camera-ready source.
  5. [§4.1; Figure 7] Implementation: G=8, λ=0.5, bG=0.2, α=0.01 are given; briefly note whether mixed-outcome mask mx rate (Figure 7b) differs across environments, since modulation is off when mx=0.

Circularity Check

0 steps flagged

No derivation-chain circularity: GRSD is an empirical self-distillation recipe evaluated against external verifiers, not a first-principles prediction that collapses to its inputs.

full rationale

The paper’s load-bearing claims are empirical (Table 1, Figs. 3–6): GRSD improves aggregate success/accuracy versus GRPO, OPSD, RLSD, Skill-SD, and SDAR across ALFWorld, Search-based QA, and WebShop. Task rewards Ri and binary labels bi come from environment verifiers independent of the reflection/guidance generators (Sec. 3.1–3.2). Group-reflective guidance zx is stop-gradient privileged context used only to modulate advantages (Eqs. 9–11) with the explicit constraint λbG≤1 so signs of Ai are preserved (Prop. 2); when mixed outcomes are absent, mx=0 and the method falls back to GRPO. Appendix A.5 states openly that Δi,t is a hindsight compatibility signal, not a causal credit estimator—so the paper does not claim a first-principles derivation that could be circular. Reflections and zx are produced by the same policy family by design (on-policy self-distillation), which is a methodological choice, not a self-definitional or fitted-input-as-prediction loop. No uniqueness theorem, ansatz, or renamed known law is imported as a forced derivation. Benchmarks and verifiers are external; the result does not reduce by construction to its inputs.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 2 invented entities

Load-bearing commitments are standard RLVR/GRPO machinery plus several method-specific modeling choices: mixed-outcome groups for guidance, external rubric judge for reflection rewards, likelihood-gap-as-credit, and hand-set modulation hyperparameters. No new physical entities; invented constructs are algorithmic.

free parameters (5)
  • self-distillation strength λ = 0.5
    Scales turn-level modulation of trajectory advantages; chosen by sweep, default 0.5.
  • modulation clip bound bG = 0.2
    Caps |bqi,t|; with λ enforces λbG≤1 so signs cannot flip.
  • reflection-loss weight α = 0.01
    Balances Lref against Ltask; sweep picks 0.01 over {0,0.001,0.01,0.1}.
  • GRPO group size G = 8
    Number of on-policy rollouts per prompt; sets contrast sample size.
  • KL coefficient / learning rate / clip ε = 0.01 / 1e-6 / 0.2
    Standard GRPO training knobs shared with baselines (0.01, 1e-6, 0.2).
axioms (5)
  • domain assumption Verifier rewards Ri and binary outcomes bi are reliable labels of trajectory success/failure.
    Entire advantage and S+/S− partition rest on environment/task verifiers (§3.1).
  • ad hoc to paper Teacher–student mean log-likelihood gap under guidance tracks turn usefulness aligned with sign(Ai).
    Core of §3.3 credit refinement; appendix A.5 admits hindsight compatibility, not causal credit.
  • ad hoc to paper A fixed external LLM judge’s {0,1,2,3} rubric is a valid reward for reflection quality.
    Stage A uses DeepSeek-V4-Flash scores for Aref; judge never edits content but defines Lref (§3.2, App. C).
  • domain assumption Clipped GRPO surrogate with group-normalized advantages is a sound base objective for agentic RLVR.
    Inherited from Shao et al. 2024 and used as Ltask backbone.
  • ad hoc to paper Stop-gradient snapshot π_θ̄ produces stable guidance and teacher scores without needing gradients through zx.
    Stage B detachment and teacher evaluation design (§3.2–3.3).
invented entities (2)
  • Group-reflective guidance zx (DO/AVOID synthesis) no independent evidence
    purpose: Privileged training-only context contrasting success vs failure reflections for one prompt.
    Central constructed object of Stage B; not observed in the environment.
  • Turn-level modulated advantage bAi,t no independent evidence
    purpose: Reweight trajectory Ai per turn via normalized clipped likelihood gaps.
    Defined in Eq. (11); the optimization target that implements self-distillation.

pith-pipeline@v1.2.0-daily-grok45 · 30364 in / 3545 out tokens · 73091 ms · 2026-07-31T18:33:43.842504+00:00 · methodology

0 comments
read the original abstract

Reinforcement learning with verifiable rewards (RLVR) is effective for training large language model agents. However, terminal rewards provide only coarse trajectory-level supervision, leaving successful behaviors, recurring mistakes, and incidental choices entangled in the same outcome signal. Existing agentic self-distillation methods enrich sparse supervision with natural-language skills, but skills retrieved externally or extracted from a single trajectory by stronger models may mismatch current experience, exceed the policy's capability, or remain path-specific. We propose Group-Reflective Self-Distillation (GRSD), which derives capability-aligned and outcome-discriminative guidance from the policy's own verified rollouts. For each prompt, the policy reflects on each verified trajectory in an on-policy group, and a stop-gradient snapshot contrasts the resulting reflections from successful and failed rollouts to construct group-level privileged guidance. Conditioned on this guidance, a self-teacher refines turn-level credit assignment by modulating outcome-based advantages while preserving the verifier-determined learning direction. Experiments across multiple agentic environments and model scales demonstrate that GRSD consistently outperforms competitive baselines and generalizes more effectively to unseen tasks.

Figures

Figures reproduced from arXiv: 2607.28076 by Binbin Zheng, Enlei Gong, Guanqun Zhao, Xiaoliang Fu, Xing Ma, Zeyu Chen, Zijun Xie.

Figure 1
Figure 1. Figure 1: Performance overview of GRSD across multiple [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of GRSD. Stage A learns policy-native reflection skills from verified trajectories. Stage B contrasts skills [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Training dynamics on ALFWorld using Qwen3-1.7B. (a) Validation success rate over training steps. (b) Mean teacher– [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Comparison of the complete policy-native reflec [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Hyperparameter sensitivity analysis on ALFWorld [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: Training dynamics on ALFWorld. (a) Mean judge [PITH_FULL_IMAGE:figures/full_fig_p015_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Validation success over 240 training steps with Qwen2.5-3B-Instruct across three environments. [PITH_FULL_IMAGE:figures/full_fig_p016_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Validation success over 240 training steps with Qwen2.5-7B-Instruct across three environments. [PITH_FULL_IMAGE:figures/full_fig_p016_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

42 extracted references · 23 linked inside Pith

  1. [1]

    arXiv preprint arXiv:2601.18734 , year=

    Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models , author=. arXiv preprint arXiv:2601.18734 , year=

    Pith/arXiv arXiv

  2. [2]

    International Conference on Learning Representations , volume=

    On-policy distillation of language models: Learning from self-generated mistakes , author=. International Conference on Learning Representations , volume=

  3. [3]

    International Conference on Learning Representations , volume=

    Minillm: Knowledge distillation of large language models , author=. International Conference on Learning Representations , volume=

  4. [4]

    arXiv preprint arXiv:2604.10688 , year=

    Scope: Signal-calibrated on-policy distillation enhancement with dual-path adaptive weighting , author=. arXiv preprint arXiv:2604.10688 , year=

    Pith/arXiv arXiv

  5. [5]

    arXiv preprint arXiv:2511.10643 , year=

    Black-Box On-Policy Distillation of Large Language Models , author=. arXiv preprint arXiv:2511.10643 , year=

    arXiv

  6. [6]

    arXiv preprint arXiv:2605.07711 , year=

    Simct: Recovering lost supervision for cross-tokenizer on-policy distillation , author=. arXiv preprint arXiv:2605.07711 , year=

    Pith/arXiv arXiv

  7. [7]

    arXiv preprint arXiv:2603.25562 , year=

    Revisiting on-policy distillation: Empirical failure modes and simple fixes , author=. arXiv preprint arXiv:2603.25562 , year=

    Pith/arXiv arXiv

  8. [8]

    arXiv preprint arXiv:2604.14084 , year=

    Tip: Token importance in on-policy distillation , author=. arXiv preprint arXiv:2604.14084 , year=

    Pith/arXiv arXiv

  9. [9]

    arXiv preprint arXiv:2604.13016 , year=

    Rethinking on-policy distillation of large language models: Phenomenology, mechanism, and recipe , author=. arXiv preprint arXiv:2604.13016 , year=

    Pith/arXiv arXiv

  10. [10]

    arXiv preprint arXiv:2602.22495 , year=

    Reinforcement-aware knowledge distillation for llm reasoning , author=. arXiv preprint arXiv:2602.22495 , year=

    Pith/arXiv arXiv

  11. [11]

    arXiv preprint arXiv:2604.03128 , year=

    Self-distilled rlvr , author=. arXiv preprint arXiv:2604.03128 , year=

    Pith/arXiv arXiv

  12. [12]

    arXiv preprint arXiv:2605.15155 , year=

    Self-distilled agentic reinforcement learning , author=. arXiv preprint arXiv:2605.15155 , year=

    Pith/arXiv arXiv

  13. [13]

    arXiv preprint arXiv:2604.10674 , year=

    Skill-sd: Skill-conditioned self-distillation for multi-turn llm agents , author=. arXiv preprint arXiv:2604.10674 , year=

    Pith/arXiv arXiv

  14. [14]

    arXiv preprint arXiv:2606.26790 , year=

    OPID: On-Policy Skill Distillation for Agentic Reinforcement Learning , author=. arXiv preprint arXiv:2606.26790 , year=

    Pith/arXiv arXiv

  15. [15]

    arXiv preprint arXiv:2210.03629 , year=

    React: Synergizing reasoning and acting in language models , author=. arXiv preprint arXiv:2210.03629 , year=

    Pith/arXiv arXiv

  16. [16]

    Advances in neural information processing systems , volume=

    Toolformer: Language models can teach themselves to use tools , author=. Advances in neural information processing systems , volume=

  17. [17]

    arXiv preprint arXiv:2402.03300 , year=

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models , author=. arXiv preprint arXiv:2402.03300 , year=

    Pith/arXiv arXiv

  18. [18]

    arXiv preprint arXiv:2606.31650 , year=

    ECHO: Prune to act, trace to learn with selective turn memory in agentic RL , author=. arXiv preprint arXiv:2606.31650 , year=

    Pith/arXiv arXiv

  19. [19]

    Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Maspo: Unifying gradient utilization, probability mass, and signal reliability for robust and sample-efficient llm reasoning , author=. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  20. [20]

    Fu, Xiaoliang and Lin, Jiaye and Fang, Yangyi and Hu, Chaowen and Qin, Cong and Shao, Zekai and Zheng, Binbin and Pan, Lu and Zeng, Ke , booktitle=. From

  21. [21]

    Advances in Neural Information Processing Systems , volume=

    Dapo: An open-source llm reinforcement learning system at scale , author=. Advances in Neural Information Processing Systems , volume=

  22. [22]

    arXiv preprint arXiv:2506.13585 , year=

    Minimax-m1: Scaling test-time compute efficiently with lightning attention , author=. arXiv preprint arXiv:2506.13585 , year=

    Pith/arXiv arXiv

  23. [23]

    arXiv preprint arXiv:2510.14545 , year=

    Agentic entropy-balanced policy optimization , author=. arXiv preprint arXiv:2510.14545 , year=

    arXiv

  24. [24]

    Advances in Neural Information Processing Systems , volume=

    Group-in-group policy optimization for llm agent training , author=. Advances in Neural Information Processing Systems , volume=

  25. [25]

    arXiv preprint arXiv:2503.09516 , year=

    Search-r1: Training llms to reason and leverage search engines with reinforcement learning , author=. arXiv preprint arXiv:2503.09516 , year=

    Pith/arXiv arXiv

  26. [26]

    arXiv preprint arXiv:2603.08754 , year=

    Hindsight credit assignment for long-horizon llm agents , author=. arXiv preprint arXiv:2603.08754 , year=

    arXiv

  27. [27]

    arXiv preprint arXiv:2606.11709 , year=

    RLCSD: Reinforcement Learning with Contrastive On-Policy Self-Distillation , author=. arXiv preprint arXiv:2606.11709 , year=

    Pith/arXiv arXiv

  28. [28]

    arXiv preprint arXiv:2601.20802 , year=

    Reinforcement Learning via Self-Distillation , author=. arXiv preprint arXiv:2601.20802 , year=

    Pith/arXiv arXiv

  29. [29]

    arXiv preprint arXiv:2605.06597 , year=

    UniSD: Towards a Unified Self-Distillation Framework for Large Language Models , author=. arXiv preprint arXiv:2605.06597 , year=

    Pith/arXiv arXiv

  30. [30]

    arXiv preprint arXiv:2605.12400 , year=

    OGLS-SD: On-Policy Self-Distillation with Outcome-Guided Logit Steering for LLM Reasoning , author=. arXiv preprint arXiv:2605.12400 , year=

    Pith/arXiv arXiv

  31. [31]

    arXiv preprint arXiv:2010.03768 , year=

    Alfworld: Aligning text and embodied environments for interactive learning , author=. arXiv preprint arXiv:2010.03768 , year=

    Pith/arXiv arXiv 2010

  32. [32]

    Advances in Neural Information Processing Systems , volume=

    Webshop: Towards scalable real-world web interaction with grounded language agents , author=. Advances in Neural Information Processing Systems , volume=

  33. [33]

    Transactions of the Association for Computational Linguistics , volume=

    Natural questions: a benchmark for question answering research , author=. Transactions of the Association for Computational Linguistics , volume=. 2019 , publisher=

    2019

  34. [34]

    Proceedings of the 2018 conference on empirical methods in natural language processing , pages=

    HotpotQA: A dataset for diverse, explainable multi-hop question answering , author=. Proceedings of the 2018 conference on empirical methods in natural language processing , pages=

    2018

  35. [35]

    Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension , author=. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  36. [36]

    Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: Long papers) , pages=

    When not to trust language models: Investigating effectiveness of parametric and non-parametric memories , author=. Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: Long papers) , pages=

  37. [37]

    Proceedings of the 28th International Conference on Computational Linguistics , pages=

    Constructing a multi-hop qa dataset for comprehensive evaluation of reasoning steps , author=. Proceedings of the 28th International Conference on Computational Linguistics , pages=

  38. [38]

    Transactions of the Association for Computational Linguistics , volume=

    MuSiQue: Multihop Questions via Single-hop Question Composition , author=. Transactions of the Association for Computational Linguistics , volume=. 2022 , publisher=

    2022

  39. [39]

    Findings of the Association for Computational Linguistics: EMNLP 2023 , pages=

    Measuring and narrowing the compositionality gap in language models , author=. Findings of the Association for Computational Linguistics: EMNLP 2023 , pages=

    2023

  40. [40]

    arXiv preprint arXiv:2605.07725 , year=

    Sod: Step-wise on-policy distillation for small language model agents , author=. arXiv preprint arXiv:2605.07725 , year=

    Pith/arXiv arXiv

  41. [41]

    arXiv preprint arXiv:2212.03533 , year=

    Text embeddings by weakly-supervised contrastive pre-training , author=. arXiv preprint arXiv:2212.03533 , year=

    Pith/arXiv arXiv

  42. [42]

    2026 , eprint=

    Deconstructing Off-Policy Ratios: Entropy-Scaled Trust Regions for Asynchronous Reinforcement Learning , author=. 2026 , eprint=

    2026