Pith. sign in

REVIEW 4 major objections 5 minor 17 references

Self-declared hypotheses turn agent trajectories into real training phases

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 09:39 UTC pith:VHSDNDZK

load-bearing objection Novel collection-time self-segmentation with honest controls; the missing no-declaration arm leaves the epistemic-state claim unproven, but the paper is worth a serious referee. the 4 major comments →

arxiv 2608.02302 v1 pith:VHSDNDZK submitted 2026-08-03 cs.AI cs.LGcs.SE

Trajectories That Segment Themselves: Agent-Declared Boundaries as a Training Unit

Jingxi Wei This is my paper
classification cs.AI cs.LGcs.SE
keywords semantic self-segmentationagent-declared boundariescausal hypothesessemantic phasescredit assignmentpreference optimizationcoding-agent trajectoriesphase-boundary training
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Long-horizon coding-agent trajectories are hard to turn into training data because neither single actions, episode labels, nor fixed windows align with what actually matters—the stretch of work governed by one causal hypothesis. The paper proposes that the acting agent declare each such hypothesis as it works, so a trajectory segments itself into variable-length semantic phases at collection time. The declaration costs one prompt instruction and no gold patch, milestone vocabulary, replayable environment, teacher logits, or retrospective segmenter. Most of the paper tests whether these boundaries are real: with every declaration scrubbed, an independent observer still attributes action blocks to their governing hypothesis at over twice chance and better than equal-length windows, while a mechanical test-event rule fails. One collection yields four supervised targets, and a standard preference optimizer trained on the resulting pairs moves a small number of held-out decisions from wrong to right, which the paper reads as construction-bound fitting rather than proof of generalization.

Core claim

The paper's central claim: a falsifiable causal hypothesis declared by the acting agent during a tool-using rollout is a legitimate training-data boundary. Each declaration opens a semantic phase; actions and evidence bind to it until the next declaration or trajectory end. Because the agent names its conjecture, a reviewer can negate it by name, producing wrong-cause-then-correction transitions. The decisive test deletes every declaration: action blocks are still attributed to their governing hypothesis at 2.38 times chance and better than equal-length windows (paired sign test p = 0.0002), while a mechanical test-event rule fails. The boundary is thus legible but not derivable; the paper d

What carries the argument

The semantic phase is the central object: the variable-length interval governed by one declared, falsifiable causal hypothesis about the cause of the problem, the search region, or the mechanism a repair must change. Its phase record carries the trajectory prefix before the decision, the hypothesis, the bound actions and evidence, the closing boundary, and a retrospective audit verdict. The declaration protocol is the instrument that produces it at collection time; the name of the hypothesis is what makes the phase an addressable handle, letting a reviewer negate one conjecture, later training targets condition on which direction was eliminated, and a boundary mark a deployment-faithful stat

Load-bearing premise

The load-bearing premise is that an agent prompted to declare its current hypothesis emits declarations that faithfully mark genuine changes in its causal belief state without changing its search behavior; the paper notes the reviewer-without-declarations ablation was never run, and the 9.8% token-cost measurement does not measure behavioral invariance.

What would settle it

Collect matched task budgets under the declaration contract and under a no-declaration contract and compare the distributions of tool calls, actions, and outcomes; a significant divergence would mean the prompt itself changed search, so the boundary is compliance rather than epistemic state. As a second check, generate plausible hypotheses after the trajectory and rerun the attribution test—if post-hoc restatements attribute as well as live declarations, then the boundary position rather than the declaration carries the signal.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • With the declaration contract in place before collection, one rollout yields four supervised targets—audit judgments (2,721 rows), proposed next hypotheses (1,405), localized fixes (553), and phase-boundary preference pairs (2,551)—including audit supervision drawn from exactly the failed spans an episode label discards.
  • The declared boundary tracks behavior rather than format: with declarations scrubbed, attribution to the governing hypothesis reaches 2.38 times chance and beats equal-length blocks on the same trajectories (paired sign test p = 0.0002), so an outside observer recovers a substantial part of the structure.
  • The boundary is not cheaply reproducible: a mechanical test-event rule matches no more boundaries than random placement at either a strict or a permissive threshold, and the code-blind annotator recovers boundaries only by over-segmenting.
  • Preference optimization on phase-boundary pairs yields an orientation-sensitive signal that a label-inverted arm does not reproduce, and on matched-construction held-out items the adapter changes four of sixty decisions, all from wrong to right, where two controls change none.
  • The phase-level credit assigner, normalized within a trajectory, gives opposite-signed advantages to phases of differing verdict class in 68% of multi-phase resolved episodes—a sign structure an episode label cannot express.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: if declarations faithfully mark epistemic state, the same protocol could apply beyond coding to any tool-using agent whose work is organized by checkable causal conjectures, turning trajectory logging into an instrumented experiment.
  • Editorial extension: because the paper did not run the reviewer without declarations, a direct test is whether post-hoc-inferred hypotheses (or no hypotheses at all) give the same attribution and downstream properties; if they do, the declaration is a convenience rather than the source of the boundary's meaning.
  • Editorial extension: the matched-construction DPO result points to pair diversity as the next lever; one concrete test is to build preference pairs from several independent generators and see whether held-out transfer improves, which would separate construction effects from boundary effects.
  • Editorial extension: the compression experiment measures retention of artifacts, not policy performance; running a deployed policy with the compressed ledger as its context would test whether addressability translates into better search, a step the paper identifies but leaves open.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a collection-time protocol in which a coding agent is prompted to declare falsifiable causal hypotheses during a rollout, and the interval governed by each hypothesis is treated as a variable-length 'semantic phase' that serves as a training unit. The central claim is that these declared boundaries are real and non-trivial: after deleting the declaration text and scrubbing hypothesis strings, an external model can still attribute action blocks to their governing hypothesis at about twice chance, better than equal-length windows; a code-blind human annotator matches boundaries above random; a mechanical test-event rule does not. The paper also constructs phase-boundary preference pairs, trains a DPO adapter, and reports that on adversarial held-out items no decisions change while on matched-construction items 4/60 change, with controls changing none. The writing is unusually candid about limitations, including the absence of a no-declaration control and the post-hoc nature of some evaluation choices.

Significance. If the central claim holds, the protocol provides a low-cost, dependency-free segmentation unit for long-horizon agent trajectories, with potential applications to credit assignment, process supervision, segment-level preference learning, and context compression. The paper's strengths include a concrete data/code release, multiple control conditions for the attribution test (label permutation, lexical baseline, equal-length windows, offset decay), and a clear separation between what is claimed and what is not. However, the load-bearing evidence for the reality of the boundaries is weakened by the absence of a no-declaration control, by an acknowledged filter asymmetry in the main paired comparison, and by the use of an interested, post-hoc human annotation. These issues are fixable in principle and do not invalidate the overall idea, but they do require revision before the central claims can be accepted.

major comments (4)
  1. [Table 3, §5.1] Missing no-declaration control. The central claim (contribution 3, §1.4) is that declared boundaries track the policy's own epistemic commitments and 'are not an artefact of the logging.' The §5 tests scrub the declaration text and hypothesis strings, but they cannot remove the behavioral effect of having been told to maintain named conjectures. Every trajectory in the corpus was collected under the declaration contract; an agent prompted to keep a conjecture may organize subsequent edits and tests to be consistent with that conjecture, which would produce exactly the attribution and boundary-placement signal observed. The paper itself admits the critical ablation was not run: 'we did not run the reviewer without declarations' (§3.1), and §9.1 states 'the boundary exists because the agent was asked to declare it.' The 9.8% token-cost statistic (§1.3) measures verbosity, not behavioral in
  2. [Table 3, §5.1] The main paired comparison is biased by a filter that favors the declared arm. The caption states that blocks under 200 rendered characters are skipped in both arms, but it also states that this drops more declared than equal-length blocks and 'the filter runs in the declared arm's favour.' This asymmetry undermines the paired sign test (p=0.0002) and the disjoint intervals (2.38x vs 2.07x). To support the claim that declared blocks are better than equal-length blocks over the same trajectories, the comparison should be reported with the filter applied symmetrically (or with no filter), and the results should be shown to be insensitive to the filter threshold. As reported, the headline attribution advantage may be an artifact of which blocks were deleted.
  3. [§5.2, §9.2] The 'code-blind' annotation is an interested, post-hoc floor. The annotator is the first author, knew the segmentation criterion, and the mark budget and primary statistic were chosen after the metrics were computed; the paper acknowledges the bias runs toward over-recovery. The permutation result (24 vs 11.5 expected, p<0.0001) is suggestive, but it is not independent evidence. To carry the 'not cheaply reproducible' claim, the annotation should be repeated by an independent annotator who does not know the criterion, with a pre-registered mark budget and a pre-specified primary metric. The current result is a lower bound, as stated, but it cannot serve as a principal piece of evidence for the reality of the boundaries.
  4. [§6, Table 4, §9.4] The downstream preference result is not statistically distinguishable from noise. On the matched-construction set, 4/60 changes has exact McNemar p=0.125, instance-level p=0.125, and shift-direction p=0.093; the adversarial set shows 0/91 changes. The paper's conclusion that 'the fitted preference is real but bound to how the pairs were written' is stronger than the evidence: all interval estimates include the null. This does not undermine the segmentation claim, but it should be reported as a null/underpowered result, not as evidence that the boundary is the bottleneck. The sentence in the Conclusion ('the next step is a more diverse pair corpus, not a different boundary') goes beyond what the data show.
minor comments (5)
  1. [Table 3] The header 'Chance×chance' is confusing; it should read '×chance' or 'multiple of chance' to match the values in the table.
  2. [§4] The phrase 'A blinded annotator finds the spans usable' initially suggests a human annotator, but the text later reveals it is a model annotator. Please call it a 'blinded model annotator' on first mention to avoid ambiguity.
  3. [§7] The sentence 'the identical keep-categories with no phase structure at all—one global bucket, one global eight-hit cap—retain exactly as many edits and tests in 79% of the record's size' is hard to parse. Please rephrase to state explicitly what budget the ablation row is compared against.
  4. [Figure 3 caption] The caption 'as declared boundary slid by (messages)' should be reworded, e.g., 'attribution as the boundary is slid by N messages from the declared position.'
  5. [§5.1] The offset curve shows a maximum at -2 messages, but the paper reports no interval and does not test that difference. The text already notes this, but the figure and surrounding discussion should not imply that the declared position is optimal.

Circularity Check

0 steps flagged

No circular derivation: the declaration-defined boundaries are tested by external-recovery probes with the declaration scrubbed; the missing no-declaration control is a confound, not a circularity.

full rationale

The paper's load-bearing claims are empirical rather than derivational. It explicitly concedes that a declared boundary is trivially retrievable from the log ('A declared boundary is trivially retrievable from the log that declared it, and that fact is worth nothing: it shows only that we recorded what we recorded,' §1.4), and then asks whether an observer who never sees the declaration can recover the structure. That test is not a renaming or a fitted-input-called-prediction: it compares declared blocks to equal-length windows over the same trajectories, includes a model-free lexical control, collapses under label permutation, and uses a code-blind annotator and a mechanical test-event rule. No parameter is fitted to a subset and then predicted on a closely related quantity; no result is imported from a self-citation (the author has no prior works in the reference list); no uniqueness theorem or ansatz is imported from the authors' earlier work. The main weakness — the absence of a no-declaration control ('we did not run the reviewer without declarations,' §3.1; 'The boundary exists because the agent was asked to declare it,' §9.1) — is a causal confound about whether the declaration protocol induces the coherence that the recovery tests measure. That is a limitation on what the experiments establish, not a circular step in which an output is equivalent to an input by construction, and the paper itself flags it. No specific circular reduction can therefore be exhibited, and the circularity score is 0.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 2 invented entities

The protocol depends on the declaration contract being behaviorally faithful, which is plausible but not directly measured. The paper's own audits are model-assisted, and the 200-character filter tilts the headline attribution comparison in the declared arm's favour. The invented entities are data-structure abstractions rather than physical posits, and they lack independent evidence outside the pipeline that constructs them.

free parameters (3)
  • minimum block length (attribution filter) = 200 rendered characters
    Table 3 caption: blocks under 200 characters are skipped in both arms; this drops more declared than window blocks and 'runs in the declared arm's favour'. Post-hoc threshold that inflates the headline declared-vs-window comparison.
  • boundary-match tolerance = ±1 step
    Section 5.2 and Appendix D: human and mechanical matching use greedy one-to-one matching within ±1 step. Not pre-registered; affects absolute hit counts (24/40 vs 11.5) and mechanical-rule results.
  • human annotator mark budget = 62 marks
    Section 5.2: annotator placed 62 marks and knew the segmentation criterion, giving a 1.55-fold over-segmentation. Not a pre-specified budget, so precision statistics are conditional on the annotator's choices.
axioms (5)
  • domain assumption Declared hypothesis boundaries coincide with the agent's actual causal-hypothesis changes.
    Section 3.1: the declaration protocol assumes the agent's announced conjecture reflects its real epistemic state. No ground truth exists and no undeclared control was run; Section 5 tests recoverability, not truthfulness.
  • domain assumption Scrubbing declarations and hypothesis strings removes all directive lexical signal, so residual attribution reflects semantic coherence.
    Section 5 setup: deleting the declaring message and redacting hypothesis strings is claimed to make the task non-trivial. The TF-IDF and permutation controls indirectly support this, but a model matcher could exploit higher-order stylistic correlations.
  • domain assumption Behavior under the declaration protocol is representative of behavior without it.
    No behavioral ablation was run; the 9.8% token overhead (Section 1.3) measures cost, not invariance. Section 3.1 explicitly admits the reviewer-without-declarations arm was not run.
  • standard math Statistical baselines (sign test, permutation test, Wilson CI) are correctly applied to clustered trajectory data.
    Used in Sections 5.1-5.2 and Appendix C. The paper sometimes ignores instance clustering (e.g., matched-set tests) and states this weakens the reported p-values.
  • domain assumption Model-assisted audits and preference labels are reliable enough for the claims made.
    Section 4: most labels are model-produced rather than human-verified, and the blinded phase audit is itself model-assisted. The paper accepts this as a limitation but relies on it for the central data unit.
invented entities (2)
  • semantic phase (hypothesis-governed interval) no independent evidence
    purpose: Local experience and credit unit; source of phase-boundary preference pairs and audit supervision.
    Defined in Sections 3.2 and 3.5. Observable only through the declaration protocol; attribution experiments provide indirect support but no independent ground truth for the claimed epistemic boundaries.
  • phase-boundary preference pair no independent evidence
    purpose: DPO supervised target comparing an advancing opening against a stalling one at a masked boundary state.
    Constructed by the pipeline from declared boundaries and generator-specific pair types. Held-out and matched tests show transfer is construction-bound, so the pair is not an independently verified training signal.

pith-pipeline@v1.3.0-daily-deepseek · 19331 in / 12350 out tokens · 104577 ms · 2026-08-04T09:39:31.594899+00:00 · methodology

0 comments
read the original abstract

Long-horizon coding-agent trajectories are poorly matched to the credit units available to train on: a single action has no stable value, an episode label merges productive exploration with abandoned directions, and a fixed window cuts where the logging mechanics fall. We introduce collection-time semantic self-segmentation, in which a declarative contract has the acting agent expose its own boundaries while the trajectory is generated. Instantiated with falsifiable causal hypotheses, successive adoptions expose variable-length semantic phases, and no milestone vocabulary, gold patch, environment replay, teacher logits, or retrospective segmenter places a boundary. Because the agent names its conjecture, a reviewer can negate it by name, which lets our protocol manufacture wrong-cause-then-correction transitions that recorded work rarely contains; one collection then yields four supervised targets, including audit supervision from exactly the failed regions an episode label discards. We then ask what survives deleting the declaration. Given the cut points but not the hypothesis, a model attributes action blocks to their governing hypothesis at over twice chance, beating equal-length blocks over the same trajectories (paired sign test $p = 0.0002$), surviving a lexical control and collapsing under label permutation. Asked instead to place boundaries, a code-blind annotator matches 24 of 40 where random placement matches 11.5, while a mechanical test-event rule beats chance at neither end of a strict-to-permissive sweep. The segments are therefore coherent and not cheaply reproducible. Downstream, DPO on 2,551 phase-boundary pairs changes no decision on 91 adversarial held-out items, while four of 60 change on matched-construction items, all wrong to right, where two controls change none: with 1,825 pairs from one generator, the variable to vary next is corpus diversity, not the boundary.

Figures

Figures reproduced from arXiv: 2608.02302 by Jingxi Wei.

Figure 1
Figure 1. Figure 1: A code-blind reader finds boundaries; a mechanical rule with the same access does not. [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: What a positional rule costs in budget. A recency tail — the simplest positional policy, [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Attribution decays away from the declared position. Second control sample, 502 phases [PITH_FULL_IMAGE:figures/full_fig_p019_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Paired difference by trajectory. Declared minus equal-length attribution accuracy for each [PITH_FULL_IMAGE:figures/full_fig_p019_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Why no decision changes on the adversarial set. Each point is one held-out item, placed by [PITH_FULL_IMAGE:figures/full_fig_p020_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Hypothesis-defined phases as a data interface. A hypothesis lifecycle creates a variable [PITH_FULL_IMAGE:figures/full_fig_p021_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

17 extracted references · 3 canonical work pages · 3 internal anchors

  1. [1]

    Agentic-DPO: From imitation to agentic policy optimization on expert trajectories.arXiv preprint arXiv:2607.10601,

    Yixiong Chen and Alan Yuille. Agentic-DPO: From imitation to agentic policy optimization on expert trajectories.arXiv preprint arXiv:2607.10601,

    Pith/arXiv arXiv

  2. [3]

    Thomas Kipf, Yujia Li, Hanjun Dai, Vinicius Zambaldi, Alvaro Sanchez-Gonzalez, Edward Grefen- stette, Pushmeet Kohli, and Peter Battaglia

    arXiv:2604.16529. Thomas Kipf, Yujia Li, Hanjun Dai, Vinicius Zambaldi, Alvaro Sanchez-Gonzalez, Edward Grefen- stette, Pushmeet Kohli, and Peter Battaglia. CompILE: Compositional imitation learning and execution. InProceedings of the 36th International Conference on Machine Learning, vol- ume 97 ofProceedings of Machine Learning Research, pages 3418–3428. PMLR,

    Pith/arXiv arXiv

  3. [5]

    Jens Lehmann, Andrei Aioanei, and Sahar Vahdati

    arXiv:2605.27788. Jens Lehmann, Andrei Aioanei, and Sahar Vahdati. Tycho: Active abstraction with programmatic world models for ARC-AGI-3.arXiv preprint arXiv:2607.28287,

    Pith/arXiv arXiv

  4. [6]

    doi: 10.48550/arXiv.2607. 28287. URL https://arxiv.org/abs/2607.28287. Sijia Li, Yuchen Huang, Zifan Liu, Yanping Li, Jingjing Fu, Li Zhao, Jiang Bian, Ling Zhang, Jun Zhang, and Rui Wang. GEAR: Granularity-adaptive advantage reweighting for LLM agents via self-distillation.arXiv preprint arXiv:2605.11853,

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2607.28287

  5. [7]

    URL https://arxiv.org/abs/2605.11853

    doi: 10.48550/arXiv.2605.11853. URL https://arxiv.org/abs/2605.11853. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step.arXiv preprint arXiv:2305.20050,

    Pith/arXiv arXiv doi:10.48550/arxiv.2605.11853

  6. [9]

    URL https://arxiv.org/abs/2508.03680

    doi: 10.48550/arXiv.2508.03680. URL https://arxiv.org/abs/2508.03680. Murong Ma, Tianyu Chen, Yun Lin, Shuai Lu, Qinglin Zhu, Yeyun Gong, Zhiyong Huang, Peng Cheng, Yan Lu, and Jin Song Dong. From patches to trajectories: Privileged process supervision for software-engineering agents.arXiv preprint arXiv:2605.21996,

    Pith/arXiv arXiv doi:10.48550/arxiv.2508.03680

  7. [10]

    From Patches to Trajectories: Privileged Process Supervision for Software-Engineering Agents

    doi: 10.48550/arXiv.260 5.21996. URL https://arxiv.org/abs/2605.21996. Jiangweizhi Peng, Yuanxin Liu, Ruida Zhou, Charles Fleming, Zhaoran Wang, Alfredo Garcia, and Mingyi Hong. HiPER: Hierarchical reinforcement learning with explicit credit assignment for large language model agents.arXiv preprint arXiv:2602.16165,

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2605.21996

  8. [11]

    URL https://arxiv.org/abs/2602.16165

    doi: 10.48550/arXiv.2602.16165. URL https://arxiv.org/abs/2602.16165. Fangxin Shang and Yehui Yang. Hypothesis-driven skill optimization for LLM agents

    Pith/arXiv arXiv doi:10.48550/arxiv.2602.16165

  9. [12]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y

    arXiv:2606.22330. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y . K. Li, Y . Wu, and Daya Guo. DeepSeekMath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300,

    Pith/arXiv arXiv

  10. [13]

    Extra dependency

    The permissive rule fires on any command the pipeline classifies as a check and places 273 marks, match- ing 29 of 40 against a random expectation of 29.4. Matching is greedy one-to-one within a±1 step tolerance, identical to the human annotation scorer, and the random expectation is taken by permuting the same number of marks over the same trajectory len...

    1999

  11. [15]

    Chenchen Zhang

    arXiv:2508.19229. Chenchen Zhang. From reasoning to agentic: Credit assignment in reinforcement learning for large language models.arXiv preprint arXiv:2604.09459,

    Pith/arXiv arXiv

  12. [16]

    URL https://arxiv.org/abs/2604.09459

    doi: 10.48550/arXiv.2604.09459. URL https://arxiv.org/abs/2604.09459. A Release and reproducibility The project repository carries the materials this study was produced from: phase extraction and evidence binding, phase audit and preference construction, the SFT and DPO training configurations, the held-out preference item sets with their construction cod...

    Pith/arXiv arXiv doi:10.48550/arxiv.2604.09459 1917

  13. [1999]

    URL https://www.sciencedirect.com/science/article/pii/S0 004370299000521

    doi: 10.1016/S0004-3702(99)00052-1. URL https://www.sciencedirect.com/science/article/pii/S0 004370299000521. 16 Shengjie Wang, Guanghe Li, Zonghan Yang, and Yang Gao. Hindsight hint distillation: Scaffolded reasoning for SWE agents from cot-free answers. 2026a. arXiv:2605.11556. Zixuan Wang, Yuchen Yan, Hongxing Li, Teng Pan, Dingming Li, Ruiqing Zhang, ...

    Pith/arXiv arXiv

  14. [2023]

    Xufang Luo, Yuge Zhang, Zhiyuan He, Zilong Wang, Siyun Zhao, Dongsheng Li, Luna K

    URL https://arxiv.org/abs/2305.20050. Xufang Luo, Yuge Zhang, Zhiyuan He, Zilong Wang, Siyun Zhao, Dongsheng Li, Luna K. Qiu, and Yuqing Yang. Agent lightning: Train ANY AI agents with reinforcement learning.arXiv preprint arXiv:2508.03680,

    Pith/arXiv arXiv

  15. [2024]

    Richard S

    URL https://arxiv.org/abs/2402.03300. Richard S. Sutton, Doina Precup, and Satinder Singh. Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning.Artificial Intelligence, 112(1–2):181–211,

    Pith/arXiv arXiv

  16. [2025]

    URL https://aclanthology.org/2025.acl-long.607/

    doi: 10.18653/v1/2025.acl-long.607. URL https://aclanthology.org/2025.acl-long.607/. Abhijit Kumar, Zoey Wu, and Mohit Suley. Knowing when to ask: Segment-level credit assignment for LLM tool use

    doi:10.18653/v1/2025.acl-long.607 2025

  17. [2026]

    Agentic-DPO: From Imitation to Agentic Policy Optimization on Expert Trajectories

    doi: 10.48550/arXiv.2607.10601. URL https://arxiv.org/abs/2607.10601. Joongwon Kim, Wannan Yang, Kelvin Niu, Hongming Zhang, Yun Zhu, Eryk Helenowski, Ruan Silva, Zhengxing Chen, Srinivasan Iyer, Manzil Zaheer, Daniel Fried, Hannaneh Hajishirzi, Sanjeev Arora, Gabriel Synnaeve, Ruslan Salakhutdinov, and Anirudh Goyal. Scaling test-time compute for agentic coding

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2607.10601