Pith. sign in

REVIEW 3 major objections 6 minor 46 references

LLM agents can turn their own failed rollouts into successful training data by relabeling the goals they actually achieved.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 20:44 UTC pith:CWLYRXSC

load-bearing objection Solid, usable post-training recipe: multi-goal hindsight relabeling plus masking/reweighting turns agent rollouts into demos and beats full-data SFT/DPO on ALFWorld with ~1/4 of the expert data. the 3 major comments →

arxiv 2607.04235 v1 pith:CWLYRXSC submitted 2026-07-05 cs.CL

Spinning Straw into Gold: Relabeling LLM Agent Trajectories in Hindsight for Successful Demonstrations

Zichao Li , Gang Wu , Zichao Wang , Ruiyi Zhang , Wanrong Zhu , Ryan A. Rossi , Vlad I Morariu , Jihyung Kil This is my paper
classification cs.CL
keywords LLM agentshindsight relabelingsupervised fine-tuninggoal-conditioned learningsample efficiencypartial observabilityALFWorldpost-training
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Language-model agents often fail at the task they were given, yet still complete other valid goals along the way. This paper argues that those unintended successes are a usable source of supervision. An auxiliary model looks back over each finished trajectory, names every natural-language goal the agent did achieve, and turns the same rollout into a successful demonstration for those goals. The agent is then fine-tuned on the relabeled pairs, with two simple fixes that ignore irrelevant actions and reweight easy versus hard examples. The method plugs into ordinary supervised fine-tuning and preference optimization, needs far fewer expert demonstrations, and produces the largest gains on long-horizon tasks that contain many possible goals. In short, the paper claims that hindsight relabeling converts noisy agent experience into high-quality training signal without requiring the agent to model the environment’s dynamics.

Core claim

Hindsight Supervised Learning recovers successful demonstrations from ordinary agent trajectories by having an auxiliary LLM identify every natural-language goal that was actually achieved and then fine-tuning the agent on those relabeled pairs. With irrelevant-action masking and sample reweighting, the approach raises success rates for both SFT and DPO, is most helpful on long-horizon tasks with diverse goal spaces, and can outperform full-dataset baselines while using only a quarter of the ground-truth demonstrations.

What carries the argument

Hindsight Supervised Learning (HSL): an auxiliary LLM that, after a trajectory ends, lists every achieved natural-language goal and labels each action as relevant or irrelevant; the agent is then trained with masked SFT on the reweighted relabeled pairs, optionally mixed with ordinary expert data.

Load-bearing premise

That a fixed, zero-shot auxiliary language model can accurately recover every achieved goal and the relevance of each action from partial observations alone.

What would settle it

Manually audit a large sample of relabeled trajectories; if goal labels or relevance masks are systematically wrong, or if removing the relabeled buffer erases the reported gains over SFT/DPO, the central claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces Hindsight Supervised Learning (HSL) for post-training LLM agents in partially observable, long-horizon settings. An auxiliary LLM reviews completed agent trajectories and relabels them with all natural-language goals the agent actually achieved (including uninstructed intermediate goals), yielding successful demonstrations for supervised fine-tuning. Two techniques—irrelevant-action masking and demonstration reweighting—address suboptimality of the hindsight data. A theoretical upper bound (Theorem 1) relates agent–expert total-variation discrepancy to SFT losses on expert and hindsight data, plus occupancy coverage κ_E←H and hindsight optimality δ_E; a corollary argues that fitting hindsight data strictly tightens the bound at equal expert fit. Empirically, HSL is added to SFT and DPO (ETO) on ALFWorld, PlanCraft, and WebShop with a Llama-3.2-1B agent and Llama-3.3-70B relabeler. It improves both pipelines (roughly 8–32% success-rate gains on ALFWorld), is more helpful on long-horizon tasks with diverse multi-goal structure, and is sample-efficient: with about one quarter of ALFWorld ground-truth demonstrations it surpasses full-data SFT/DPO baselines. Ablations (RelabelFailure, UniWeight, NoMask), multi-seed tables, goal-type analysis, and a 90% human audit of 200 ALFWorld relabels support the design choices.

Significance. If the results hold, HSL is a practical, complementary source of supervision that turns noisy agent rollouts into usable demonstrations without extra environment interaction or hand-crafted intermediate rewards. The sample-efficiency claim on ALFWorld is especially valuable given the cost of expert demos for agentic tasks. The work usefully adapts HER/GCSL-style goal relabeling to LLM agents under partial observability, multi-goal mining along full trajectories, and continuous on-policy buffer refresh—distinguishing it from fixed offline synthesis (BAGEL, BehaviorClone). Strengths that should be credited include: an explicit theory-to-technique link (masking and reweighting motivated by Theorem 1), strong baselines that also use the 70B model, ablations that remove multi-goal mining / reweighting / masking and lose gains, multi-seed reporting (Table 2), and a human audit of relabeling quality. Limitations (predefined goal templates, residual need for some expert demos, compute cost of the relabeler) are acknowledged and do not invert the reported numbers on the evaluated benchmarks.

major comments (3)
  1. §3.4, Theorem 1 and Corollary 1: The analysis shows that SFT on hindsight data can tighten an upper bound on Δ(θ) when expert fit is held fixed, via κ_E←H and δ_E. The manuscript never reports empirical estimates of occupancy coverage, hindsight–expert TV, or agent–expert TV before vs. after HSL (or vs. fixed-synthesis baselines). Without these diagnostics it is difficult to judge how tight the bound is or how much of the empirical gain is explained by the stated mechanism versus other factors (e.g., distillation from the 70B relabeler). A short measurement of these intermediate quantities on ALFWorld would make the theory–experiment link load-bearing rather than motivational.
  2. §4.4, Figures 6–7: Relabeling fidelity is the weakest assumption of the pipeline. The human audit (200 samples, 90% accuracy) is only on ALFWorld and already shows goal-type variation and incomplete-goal errors (e.g., “cool tomato” accepted without placement in the fridge). PlanCraft and WebShop have no analogous audit, yet the abstract and Table 1 treat HSL gains as general. Reporting accuracy and failure modes on the other two benchmarks (even on a smaller sample) is needed to support the claim that zero-shot hindsight recovery is reliably easier than acting.
  3. §4.1–4.2, Table 1 and Figure 3: All agent results use a 1B policy with a fixed 70B relabeler. Controls (REACT, BEHAVIORCLONE, BAGEL) correctly use the same 70B model, but the paper does not test (i) a weaker/cheaper relabeler or (ii) an agent closer in scale to the relabeler. The sample-efficiency and “compatible with SFT/DPO” claims are therefore demonstrated only in a large teacher / small student regime. At least one additional setting (weaker relabeler or larger agent) is needed before the central efficiency claim can be treated as pipeline-general rather than scale-asymmetric.
minor comments (6)
  1. Notation inconsistency: S(I′_k) in §3.3 vs. S_M(I′) in Eq. (1) and the appendix; unify the first-achievement operator.
  2. Figure 3 and Figure 4: axis labels and legend are dense; success-rate percentages overlaid on bars help, but the “Full (>3200)” tick is easy to misread as a continuous scale—consider categorical ticks only.
  3. §3.4, Eq. (3)–(4): P(d′) uses w′_d in the numerator while the text defines w_d; clarify whether reweighting is applied only to the relabeled term or also renormalized after masking.
  4. Appendix C prompts are thorough and reproducible; consider moving a short excerpt of the ALFWorld goal templates into the main text so readers need not open the appendix to understand the allowed goal space.
  5. Table 2 multi-seed results are reported only for ALFWorld and WebShop; PlanCraft remains single-seed (seed 42). A one-sentence note on variance for PlanCraft would match the care given to the other two.
  6. Related work §2.2: GCSL and Cideron et al. are correctly distinguished; a brief pointer to more recent language-conditioned HER variants in robotics would help readers place the multi-goal, partial-observation contribution.

Circularity Check

0 steps flagged

No significant circularity: HSL is an empirical post-training method with an independent imitation-learning-style bound and external success-rate evaluation.

full rationale

The paper's central claims (sample-efficient gains of HSL over SFT/DPO on ALFWorld/PlanCraft/WebShop success rates and task scores; larger gains on long-horizon diverse-goal tasks) are measured against held-out interactive environment metrics that are independent of the training objective and of any free parameters fitted inside the paper. Theorem 1 is a standard finite-horizon occupancy/TV/KL discrepancy bound (with Pinsker and a simulation lemma) relating the agent to an expert and to a hindsight expert induced by the agent's own rollouts plus an external zero-shot relabeler; the bound is derived rather than assumed, and the corollary simply notes that matching the hindsight expert (under coverage) tightens it. Relabeling accuracy is audited by human inspection (90% on 200 samples) rather than being defined by the loss. Masking and reweighting are explicit heuristics motivated by the bound's coverage and optimality terms; they are ablated, not smuggled. On-policy buffer refresh is intentional design, not a definitional loop. No uniqueness theorem, ansatz, or fitted scalar is imported via self-citation and then re-presented as a prediction. The derivation chain is therefore self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 2 invented entities

The central empirical claim rests on standard POMDP agent modeling, access to a known textual goal space for prompting, the premise that zero-shot LLM hindsight labeling is accurate enough to act as a hindsight expert, and a handful of training hyperparameters (mixture weight, reweighting exponent, buffer size). No new physical entities; invented constructs are the hindsight expert and the HSL pipeline itself.

free parameters (3)
  • mixture weight λ (ground-truth vs relabeled loss) = 0.3
    Set to 0.3 by authors; balances expert SFT against hindsight SFT in Eq. (4) and directly affects reported gains.
  • reweighting exponent α = 0.8
    Set to 0.8; controls how strongly optimally solved / longer demos are upweighted in sampling from D′.
  • relabeled buffer size N and batch of new trajectories b = N=100, b=18
    Queue size 100 and b=18 per step; control on-policy refresh rate and coverage of π_H.
axioms (4)
  • domain assumption Agent tasks are finite-horizon POMDPs with natural-language instructions and a known textual description of the goal space G used in relabeler prompts.
    Stated in §3.1 and used throughout relabeling prompts (Appendix C); open-world goals without templates are out of scope (Limitations).
  • ad hoc to paper Zero-shot auxiliary LLM can identify all achieved goals and binary action relevance from full trajectories more easily than the agent can solve the original task.
    Core methodological premise in Introduction and §3.3; supported by 90% audit but not proved in general.
  • standard math Finite-horizon simulation / occupancy mismatch inequalities from imitation learning (Ross & Bagnell style) apply with C_T = 2(T−1).
    Lemma 1 and Theorem 1 in Appendix D; standard TV/KL error propagation under finite horizon.
  • domain assumption Some ground-truth demonstrations remain necessary for stable learning (support alignment with the expert).
    Stated in §3.4 and Limitations; shared with many agent post-training methods.
invented entities (2)
  • Hindsight expert π_H no independent evidence
    purpose: Defines the action distribution induced by agent trajectories conditioned on first achievement of a relabeled goal; used to bound expert–agent discrepancy.
    Definition Eq. (1)/Appendix D; analytical construct, not an external physical entity.
  • Hindsight Supervised Learning (HSL) pipeline no independent evidence
    purpose: On-policy collect–relabel–mask–reweight–SFT loop combined with existing SFT/DPO.
    The proposed method; validated only by the paper’s experiments.

pith-pipeline@v1.1.0-grok45 · 27609 in / 3514 out tokens · 37110 ms · 2026-07-11T20:44:36.991299+00:00 · methodology

0 comments
read the original abstract

Large language model agents operate in partially observable, long-horizon settings where obtaining supervision remains a major bottleneck. We address this by utilizing a source of supervision overlooked in existing post-training methods: unintended yet successful goals embedded within agent rollouts. Specifically, we introduce Hindsight Supervised Learning (HSL), where an auxiliary LLM reviews each completed trajectory and relabels it with all of the natural-language goals the agent actually achieved. HSL then pairs the trajectory with its relabeled goals and uses these pairs for additional fine-tuning. To mitigate suboptimality in the relabeled data, we propose two learning techniques for HSL, irrelevant-action masking and sample reweighting. Our experiments show that HSL is flexible and compatible with existing post-training pipelines. It improves both SFT and DPO, with larger gains on long-horizon tasks with more diverse goal spaces. Moreover, HSL is sample-efficient: on ALFWorld, it surpasses baselines trained on the full dataset while using only one quarter of the ground-truth demonstrations.

Figures

Figures reproduced from arXiv: 2607.04235 by Gang Wu, Jihyung Kil, Ruiyi Zhang, Ryan A. Rossi, Vlad I Morariu, Wanrong Zhu, Zichao Li, Zichao Wang.

Figure 1
Figure 1. Figure 1: Toy example. Given the instruction “Go to the blue flag”, the agent may instead reach the red flag or visit the yellow flag along the way. While these trajectories are incorrect or not optimal for the original instruction, they can be relabeled as successful demonstrations for the goals the agent actually achieved. Instead, the problem reduces to inferring which tasks were actually accomplished by reasonin… view at source ↗
Figure 2
Figure 2. Figure 2: Left: Overview of the existing training pipeline with HSL. Right: Example of the relabeling [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Sample efficiency of HSL with different post-training methods. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Ablation studies on ALFWorld [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Top: Distribution of the goal types in relabeled and ground-truth demonstrations. Bottom: [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Accuracy of relabeling per goal type. For clean, heat, and cool, the pro￾portions in the relabeled data are close to those in the ground truth, and the agent achieves success rates above 80%. In con￾trast, examine and put-two are under￾represented in the relabeled demos. While put and put-two appear at similar rates in the ground truth, the relabeled demos contain roughly twice as many put as put-two. More… view at source ↗
Figure 7
Figure 7. Figure 7: Two examples (correct and incorrect) of relabeled demonstrations in ALFWorld. Each [PITH_FULL_IMAGE:figures/full_fig_p010_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Goal inference prompt for ALFWorld 20 [PITH_FULL_IMAGE:figures/full_fig_p020_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Action relevance labeling prompt for ALFWorld Goal inference prompt for PlanCraft You are a recipe-inference assistant for MineCraft. Given a crafted item, its recipes and a sequence of actions, the change of slots (inventory, crafting grid, output) as effects of the action, derive which recipe is actually used for crafting the item. Slot legend: • [0] is the output slot where the crafted result appears. •… view at source ↗
Figure 10
Figure 10. Figure 10: Goal inference prompt for PlanCraft 21 [PITH_FULL_IMAGE:figures/full_fig_p021_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Action relevance labeling prompt for PlanCraft 22 [PITH_FULL_IMAGE:figures/full_fig_p022_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Goal inference prompt for WebShop 23 [PITH_FULL_IMAGE:figures/full_fig_p023_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Action relevance labeling prompt for WebShop 24 [PITH_FULL_IMAGE:figures/full_fig_p024_13.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

46 extracted references · 13 linked inside Pith

  1. [1]

    arXiv preprint arXiv:2010.03768 , year=

    Alfworld: Aligning text and embodied environments for interactive learning , author=. arXiv preprint arXiv:2010.03768 , year=

    Pith/arXiv arXiv 2010

  2. [2]

    arXiv e-prints , pages=

    On the Opportunities and Risks of Foundation Models , author=. arXiv e-prints , pages=

  3. [3]

    International Conference on Machine Learning , pages=

    GPT-4V (ision) is a Generalist Web Agent, if Grounded , author=. International Conference on Machine Learning , pages=. 2024 , organization=

    2024

  4. [4]

    Advances in Neural Information Processing Systems , volume=

    Toolqa: A dataset for llm question answering with external tools , author=. Advances in Neural Information Processing Systems , volume=

  5. [5]

    arXiv preprint arXiv:2310.06770 , year=

    Swe-bench: Can language models resolve real-world github issues? , author=. arXiv preprint arXiv:2310.06770 , year=

    Pith/arXiv arXiv

  6. [6]

    Advances in Neural Information Processing Systems , volume=

    Minedojo: Building open-ended embodied agents with internet-scale knowledge , author=. Advances in Neural Information Processing Systems , volume=

  7. [7]

    Advances in Neural Information Processing Systems , volume=

    Embodied agent interface: Benchmarking llms for embodied decision making , author=. Advances in Neural Information Processing Systems , volume=

  8. [8]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Trial and Error: Exploration-Based Trajectory Optimization of LLM Agents , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  9. [9]

    Advances in neural information processing systems , volume=

    Direct preference optimization: Your language model is secretly a reward model , author=. Advances in neural information processing systems , volume=

  10. [10]

    arXiv preprint arXiv:1810.08272 , year=

    Babyai: A platform to study the sample efficiency of grounded language learning , author=. arXiv preprint arXiv:1810.08272 , year=

    Pith/arXiv arXiv

  11. [11]

    Advances in Neural Information Processing Systems , volume=

    Webshop: Towards scalable real-world web interaction with grounded language agents , author=. Advances in Neural Information Processing Systems , volume=

  12. [12]

    2024 , booktitle=

    WebArena: A Realistic Web Environment for Building Autonomous Agents , author=. 2024 , booktitle=

    2024

  13. [13]

    Advances in Neural Information Processing Systems , volume=

    Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments , author=. Advances in Neural Information Processing Systems , volume=

  14. [14]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Os agents: A survey on mllm-based agents for computer, phone and browser use , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  15. [15]

    arXiv preprint arXiv:1707.06347 , year=

    Proximal policy optimization algorithms , author=. arXiv preprint arXiv:1707.06347 , year=

    Pith/arXiv arXiv

  16. [16]

    Advances in Neural Information Processing Systems , volume=

    Reflexion: Language agents with verbal reinforcement learning , author=. Advances in Neural Information Processing Systems , volume=

  17. [17]

    Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    Watch Every Step! LLM Agent Learning via Iterative Step-level Process Refinement , author=. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    2024

  18. [18]

    Advances in neural information processing systems , volume=

    Hindsight experience replay , author=. Advances in neural information processing systems , volume=

  19. [19]

    arXiv preprint arXiv:1809.02070 , year=

    Archer: Aggressive rewards to counter bias in hindsight experience replay , author=. arXiv preprint arXiv:1809.02070 , year=

    Pith/arXiv arXiv

  20. [20]

    Advances in neural information processing systems , volume=

    Curriculum-guided hindsight experience replay , author=. Advances in neural information processing systems , volume=

  21. [21]

    International Conference on Learning Representations , year=

    DHER: Hindsight experience replay for dynamic goals , author=. International Conference on Learning Representations , year=

  22. [22]

    arXiv preprint arXiv:1912.06088 , year=

    Learning to reach goals via iterated supervised learning , author=. arXiv preprint arXiv:1912.06088 , year=

    Pith/arXiv arXiv 1912

  23. [23]

    2020 IEEE Symposium Series on Computational Intelligence (SSCI) , pages=

    Higher: Improving instruction following with hindsight generation for experience replay , author=. 2020 IEEE Symposium Series on Computational Intelligence (SSCI) , pages=. 2020 , organization=

    2020

  24. [24]

    , title =

    Zhang, Tianjun and Liu, Fangchen and Wong, Justin and Abbeel, Pieter and Gonzalez, Joseph E. , title =. 2023 , booktitle =

    2023

  25. [25]

    Transactions on Machine Learning Research , issn=

    Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models , author=. Transactions on Machine Learning Research , issn=. 2023 , url=

    2023

  26. [26]

    2025 , eprint=

    Agent-RLVR: Training Software Engineering Agents via Guidance and Environment Rewards , author=. 2025 , eprint=

    2025

  27. [27]

    2023 , html =

    Yao, Shunyu and Zhao, Jeffrey and Yu, Dian and Du, Nan and Shafran, Izhak and Narasimhan, Karthik and Cao, Yuan , booktitle =. 2023 , html =

    2023

  28. [28]

    arXiv preprint arXiv:2402.03300 , year=

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models , author=. arXiv preprint arXiv:2402.03300 , year=

    Pith/arXiv arXiv

  29. [29]

    AISTATS , year=

    Efficient reductions for imitation learning , author=. AISTATS , year=

  30. [30]

    AISTATS , year=

    A reduction of imitation learning and structured prediction to no-regret online learning , author=. AISTATS , year=

  31. [31]

    International Conference on Machine Learning , pages=

    BAGEL: Bootstrapping Agents by Guiding Exploration with Language , author=. International Conference on Machine Learning , pages=. 2024 , organization=

    2024

  32. [32]

    EACL 2023-17th Conference of the European Chapter of the Association for Computational Linguistics, Proceedings of the Conference , year=

    Self-imitation learning for action generation in text-based games , author=. EACL 2023-17th Conference of the European Chapter of the Association for Computational Linguistics, Proceedings of the Conference , year=

    2023

  33. [33]

    Findings of the Association for Computational Linguistics ACL 2024 , pages=

    AgentTuning: Enabling Generalized Agent Abilities for LLMs , author=. Findings of the Association for Computational Linguistics ACL 2024 , pages=

    2024

  34. [34]

    arXiv e-prints , pages=

    The llama 3 herd of models , author=. arXiv e-prints , pages=

  35. [35]

    Proceedings of the ACM on Web Conference 2025 , pages=

    Symagent: A neural-symbolic self-learning agent framework for complex reasoning over knowledge graphs , author=. Proceedings of the ACM on Web Conference 2025 , pages=

    2025

  36. [36]

    arXiv preprint arXiv:2503.04957 , year=

    Safearena: Evaluating the safety of autonomous web agents , author=. arXiv preprint arXiv:2503.04957 , year=

    Pith/arXiv arXiv

  37. [37]

    arXiv preprint arXiv:1711.05101 , year=

    Decoupled weight decay regularization , author=. arXiv preprint arXiv:1711.05101 , year=

    Pith/arXiv arXiv

  38. [38]

    arXiv preprint arXiv:2412.09605 , year=

    Agenttrek: Agent trajectory synthesis via guiding replay with web tutorials , author=. arXiv preprint arXiv:2412.09605 , year=

    Pith/arXiv arXiv

  39. [39]

    arXiv preprint arXiv:2412.19723 , year=

    Os-genesis: Automating gui agent trajectory construction via reverse task synthesis , author=. arXiv preprint arXiv:2412.19723 , year=

    Pith/arXiv arXiv

  40. [40]

    Advances in Neural Information Processing Systems , volume=

    Learning formal mathematics from intrinsic motivation , author=. Advances in Neural Information Processing Systems , volume=

  41. [41]

    arXiv preprint arXiv:2412.21033 , year=

    Plancraft: an evaluation dataset for planning with LLM agents , author=. arXiv preprint arXiv:2412.21033 , year=

    Pith/arXiv arXiv

  42. [42]

    Advances in neural information processing systems , volume=

    Learning to summarize with human feedback , author=. Advances in neural information processing systems , volume=

  43. [43]

    The Twelfth International Conference on Learning Representations , year=

    Let's verify step by step , author=. The Twelfth International Conference on Learning Representations , year=

  44. [44]

    arXiv preprint arXiv:2210.10047 , year=

    From play to policy: Conditional behavior generation from uncurated robot data , author=. arXiv preprint arXiv:2210.10047 , year=

    Pith/arXiv arXiv

  45. [45]

    Conference on robot learning , pages=

    Learning latent plans from play , author=. Conference on robot learning , pages=. 2020 , organization=

    2020

  46. [46]

    ArXiv , year=

    Plancraft: an evaluation dataset for planning with LLM agents , author=. ArXiv , year=