Pith. sign in

REVIEW 3 major objections 6 minor 33 references

GenPlan: Generative Sequence Models as Adaptive Planners

T0 review · 3 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read GenPlan is a discrete-flow planner that jointly denoises states, actions, and goals; the paper reports it outperforms prior planners by over 10 percentage points on adaptive planning tasks where an agent trained on simple demos must solve…

desk verdict Promising DFM-based planner with a clean idea, but a load-bearing inconsistency in the uniform-noise variant and loose energy-guidance description make the experimental claims hard to verify as written. read the letter →

arxiv 2412.08565 v2 pith:TYXHNXND submitted 2024-12-11 cs.LG cs.AI

Akash Karthikeyan , Yash Vardhan Pant This is my paper · ORCID
classification cs.LGcs.AI
keywords offlinereinforcementlearningsequencemodelingdiscreteflowmodelscontinuous-timeMarkovchainsenergy-basedplanningadaptivezero-shotgeneralizationBabyAI
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

GenPlan turns planning into a generative sequence-modeling problem: a discrete flow model iteratively denoises a corrupted trajectory of states, actions, and goals until it forms a plan. The paper trains an energy-based denoiser with a lower bound on entropy so the planner keeps exploring instead of collapsing onto repeated actions from the demonstrations. Its central claim is that this recipe lets an agent trained only on simple single-goal demonstrations adapt zero-shot to harder missions involving multiple goals, locked doors, and obstacles. A sympathetic reader should care because the method removes the need for a simulator-given goal oracle at test time: the goal sequence is generated together with the actions, which is what prior planners required.

What carries the argument

The load-bearing object is the joint denoising model $p^\theta_{1|t}$ together with the reverse CTMC sampler of the discrete flow model. The denoiser is trained to minimize the negative log-likelihood of clean tokens at masked positions, subject to $\mathbb{E}[H(p^\theta_{1|t})] \geq \beta$; this entropy constraint is what lets the planner propose actions and goals that were not in the training demonstrations. The reverse process uses a rate matrix $R_t$ with masking or uniform interpolants to evolve a corrupted trajectory $\tau^t$ back to a clean plan $\tau^1$. All three trajectory components, states, actions, and goals, are denoised jointly by a bidirectional transformer conditioned on FiLM-encoded image, instruction, and position observations, so goal discovery is part of planning rather than an input.

What would settle it

Train GenPlan only on single-goal demonstrations in a planar environment, then evaluate it in a novel maze where the mission requires an unblocking or door-opening skill that never appeared in training, with no simulator-provided goal oracle. If its success rate does not substantially exceed LEAP without goal conditioning and Decision Transformer under identical conditions, the claimed generalization of the energy function fails. A more direct check is to compute, in a held-out maze, whether the learned energy assigns lower values to expert trajectories than to stalling or repeatedly-forward trajectories; if it does not, the energy objective is not the mechanism carrying the adaptation.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that a maximum-likelihood objective is the wrong training target for adaptive planning and that an entropy-regularized energy objective over whole trajectories is the right one. GenPlan models a trajectory as discrete tokens, corrupts it with either masking or uniform noise, and trains a bidirectional transformer denoiser $p^\theta_{1|t}$ to predict the clean states, actions, and goals. The energy of a sequence is the accumulated negative log-likelihood the denoiser assigns to it, and the entropy constraint keeps the action distribution from collapsing. At inference, the reverse continuous-time Markov chain turns noise into a plan, jointly producing goals and actions rather than taking goal positions from a simulator. In the BabyAI adaptive-planning suite, GenPlan-M reaches 52.3% mean success versus 38.4% for LEAP and 18.2% for Decision Transformer, and it is the only method to make progress on key-collection and obstacle-unblocking missions; in continuous PushT and Kitchen benchmarks it is competitive with, though not ahead of, diffusion-policy and VQ-BeT baselines.

Load-bearing premise

The load-bearing premise is that the energy function learned on simple single-goal training environments keeps assigning low energy to successful trajectories in harder, unseen test environments; the paper states this as a hypothesis (Section 1: 'as long as the objective function remains consistent'; Section 4: 'provided the energy function is generalizable') rather than proving it.

Editorial extensions

If this is right

  • A planner can be trained on one simple task distribution and evaluated on substantially harder tasks with zero fine-tuning, since the goal-generation module proposes sub-goals during denoising.
  • Simulator-based goal oracles, which LEAP and Decision Transformer rely on, become unnecessary; this matters for real deployments where goal positions are not known in advance.
  • Order-critical long-horizon missions, such as collecting a key before opening a door, become feasible for demonstration-trained planners, where behavior-cloning baselines stall or loop.
  • The entropy regularizer provides a mechanism for discovering skills such as door opening and obstacle unblocking that are absent from the training data, not just for stitching known skills.
  • Masking and uniform noise interpolants both work, with masking training faster, so the planner can trade exploration for convergence by choosing the corruption schedule.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct but untested extension is to treat GenPlan's entropy-regularized discrete-flow objective as a general sequence-generation prior: any domain where low-energy sequences are rare and mode collapse is fatal, such as instruction following or program synthesis, could inherit the same zero-shot refinement behavior.
  • The reported adaptive-planning margin is established in discrete grid-world planning; in the paper's continuous manipulation benchmarks GenPlan is competitive but not dominant, which suggests the advantage is tied to discrete, order-structured planning rather than low-level control.
  • One could test the generalization mechanism directly by measuring whether the learned energy ranks expert trajectories below stalling or repeatedly-forward trajectories in novel mazes; if it does not, the energy objective would need a different explanation for the observed success.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes GenPlan, a generative planner built on discrete flow models (CTMC-based denoising) that jointly denoises sequences of actions, states, and goals. Training uses a negative log-likelihood objective with an entropy lower bound, and planning is done by iterative reverse-CTMC sampling from a learned denoiser. The authors evaluate GenPlan on BabyAI trajectory planning, instruction completion, and adaptive planning tasks, plus continuous PushT and Franka Kitchen tasks, and report success-rate improvements, including AP results showing GenPlan outperforming LEAP by roughly 13.8 percentage points.

Significance. If the reported results are reproducible, GenPlan is a useful contribution: it demonstrates that a discrete flow model with joint goal/state/action denoising and entropy regularization can transfer from simple single-goal demonstrations to novel multi-goal, door-opening, and obstacle-unblocking tasks without oracle goal conditioning. The manuscript is also commendable for releasing code, comparing against energy-based and diffusion baselines, and including ablations of the joint prediction, entropy, and noise-schedule components. However, the uniform-noise variant appears to be untrainable under the published objective, the central empirical claims lack error bars, and the advertised 'energy-guided sampling' is not what Algorithm 2 actually does; these issues currently prevent the results from being verified as described.

major comments (3)
  1. [§3–§4, Eqs. (3b), (5), Algorithm 1] The uniform interpolant and the training loss are mutually inconsistent. Under Eq. (3b), x_t takes values only in the ordinary token set, so the mask state [M] never appears; consequently the Kronecker delta in Eq. (5) is identically zero for every position, making L_x identically zero, and Algorithm 1 would train only the entropy regularizer. Since Tables 1, 2, and 9 report GenPlan-U success rates (and the AP mean in Table 2 is a load-bearing part of the abstract's 'over 10%' claim), the paper must either specify the actual loss minimized for GenPlan-U (e.g., a loss over positions where x_t differs from x_1) or state explicitly that the uniform variant is trained with a masking schedule and only sampled with a uniform rate matrix. As written, the reported GenPlan-U results cannot be reproduced.
  2. [§5, Tables 1, 2, and 9] The headline comparisons are single numbers with no estimates of variance. Success rates are averaged over 250 environments, but no standard errors, confidence intervals, or number of seeds are reported for the main tables; Figure 10A reports three seeds for only one environment. Without variance information, the observed differences (e.g., GP-U 46.8% vs. LEAP 38.4% on adaptive planning) cannot be assessed for statistical significance, and some ablation differences in Table 3 are as small as 0.8 percentage points. Please report mean ± standard deviation (or confidence intervals) across seeds, or at minimum across environment resamplings.
  3. [§4, Energy-Guided Denoising Model vs. Algorithm 2] The text repeatedly states that sampling is energy-guided and that the optimization 'stays at an energy minimum at each step,' but Algorithm 2 contains no energy evaluation and no energy-gradient or energy-based accept/reject step; it only samples from the rate matrix R^θ_t that is built from the learned denoiser p^θ_{1|t}. If the energy function E(a) is only an implicit construction of the training objective, the paper should say so and should not claim energy-guided sampling without either placing an energy term in Algorithm 2 or explaining precisely how the energy landscape shapes the CTMC sampling. This is not just a wording issue, because Section 1 and the limitations discussion attribute the generalization gains to the energy landscape.
minor comments (6)
  1. [§4, Eq. (4a)] The expectation in Eq. (4a) is written over a0 and o but the loss involves a1; the intended distribution over a1 (presumably the data distribution) should be stated explicitly.
  2. [Algorithm 1, line 9] The update rule for the Lagrangian multiplier λ is not fully specified: the step size for the λ update is omitted, and the entropy term in line 6 is evaluated at τ0,o while Eq. (4b) is stated over a0,o; please clarify.
  3. [Table 3] The column 'GenPlan-M Reduction' is confusing: it reports the reduced success rate and the decrease relative to a baseline, but it is not clear whether the baseline is the full GenPlan-M from Tables 1–2 or a different run; please state the reference value explicitly.
  4. [§5, Tables 1–2] The parenthetical deltas such as 'TP (7.6 ↑)' and 'AP (13.84 ↑)' are not defined in the table captions or text; specify which baseline(s) these deltas are computed against.
  5. [Appendix A, Tables 19–20] The uniform-interpolant example in Table 20 shows a token '0' that is not in the action set {0,...,5} defined elsewhere; please clarify how the uniform noise state space relates to the actual token vocabulary.
  6. [§5, Results section] The AP discussion says GenPlan 'can eventually find a plan given enough timesteps as long as the energy function generalizes well,' which is a conditional statement and not a demonstrated property; please mark this clearly as an assumption rather than a conclusion.

Circularity Check

1 steps flagged · score 4.0 of 10

GenPlan-M's central results are held-out and non-circular, but the uniform variant GenPlan-U is untrainable as specified: Eq. (5)'s mask-delta loss is identically zero under Eq. (3b), so the reported GP-U numbers are not derivable from the published method.

  1. other [Section 4 (Training Objective, Eq. 5 vs Eq. 3b); Algorithm 1 lines 4-7; Tables 1, 2, 9]
    "L_x = [ - Σ_{k=1}^H δ{x^t_k, [M]} log pθ_{1|t}(x1|x^t,o) ] (5) ... p^unif_{t|1}(x_t | x_1) = C(tδ{x_1,x_t} + (1−t) 1/|X|) (3b)"

    Under Eq. (3b) every corrupted token is drawn from the ordinary token set {1,...,|X|}; the mask state [M] never appears. Therefore δ{x^t_k,[M]} ≡ 0, so L_x ≡ 0 for GenPlan-U. Algorithm 1's LNLL ← La + Ls + Lg then supplies no learning signal; the only remaining term is the entropy regularizer, which a uniform output trivially satisfies. Tables 1, 2, and 9 nevertheless report GenPlan-U success rates close to GenPlan-M. By the paper's own equations, these GP-U results cannot be produced by the stated training objective and require an undisclosed different loss, making the uniform-variant numbers a by-construction artifact rather than a prediction of the described method.

full rationale

The central GenPlan-M results are held-out successes on novel BabyAI and continuous tasks, so the main 'over 10%' claim does not reduce to a fitted parameter or to the training objective; GenPlan-M alone beats LEAP by 13.84 points on AP mean (52.3% vs 38.5%), preserving the headline claim. The energy-landscape observation in Appendix B that clean sequences receive low energy is by construction of the NLL objective, but the paper presents it only as a sanity check, not as evidence for OOD generalization. There is no load-bearing self-citation: the DFM formalism is cited to Campbell et al. as external work, and no uniqueness theorem is imported from the present authors. However, the uniform-noise variant is internally inconsistent as specified: Eq. (5) scores only positions corrupted to [M], while Eq. (3b) never produces [M]. This is a severe method-description gap in a reported variant, though not a circularity of the central GenPlan-M model, so the score is 4 rather than 0-2 or 6+.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the discrete flow model machinery from Campbell et al. as a black box, on the untested assumption that the learned energy landscape transfers to unseen environments, on the assumption of near-optimal demonstrations, and on a manually tuned entropy bound beta. No new physical entities are introduced.

free parameters (3)
  • Entropy lower bound beta = 0.3 (TP/IC), 0.7 (AP)
    Manually specified per task paradigm (Table 8); acknowledged as a limitation in the paper.
  • Number of denoising iterations Imax = 10-30 depending on task
    Chosen per environment (Table 8); affects success rates (Figure 10A).
  • Context length = 1 or 10
    Used as memory in KeyCorridor and BlockUnlock tasks (Table 8).
assumptions (4)
  • standard math Discrete flow model (CTMC) framework of Campbell et al. (2024) correctly describes the forward and reverse generative process
    Used as the backbone for training and sampling (Section 3, Preliminaries).
  • domain assumption The learned energy function generalizes to out-of-distribution environments
    Section 4 states auxiliary modules 'can generalize to out-of-distribution tasks, provided the energy function is generalizable'.
  • domain assumption Near-optimal demonstration trajectories are available for training
    Stated in Limitations in Section 6.
  • ad hoc to paper Entropy lower bound beta induces exploration and task discovery
    The mechanism is hypothesized (Section 5), not proven.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GenPlan: Generative Sequence Models as Adaptive Planners." pith.science (2026). https://pith.science/paper/TYXHNXND

@misc{pith2026241208565,
  author       = {Pith},
  title        = {Pith review of: GenPlan: Generative Sequence Models as Adaptive Planners},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TYXHNXND}},
  note         = {Machine review of arXiv:2412.08565}
}
read the original abstract

Sequence models have demonstrated remarkable success in behavioral planning by leveraging previously collected demonstrations. However, solving multi-task missions remains a significant challenge, particularly when the planner must adapt to unseen constraints and tasks, such as discovering goals and unlocking doors. Such behavioral planning problems are challenging to solve due to: a) agents failing to adapt beyond the single task learned through their reward function, and b) inability to generalize to new environments, e.g., those with walls and locked doors, when trained only in planar environments. Consequently, state-of-the-art decision-making methods are limited to missions where the required tasks are well-represented in the training demonstrations and can be solved within a short (temporal) planning horizon. To address this, we propose GenPlan: a stochastic and adaptive planner that leverages discrete-flow models for generative sequence modeling, enabling sample-efficient exploration and exploitation. This framework relies on an iterative denoising procedure to generate a sequence of goals and actions. This approach captures multi-modal action distributions and facilitates goal and task discovery, thereby generalizing to out-of-distribution tasks and environments, i.e., missions not part of the training data. We demonstrate the effectiveness of our method through multiple simulation environments. Notably, GenPlan outperforms state-of-the-art methods by over 10% on adaptive planning tasks, where the agent adapts to multi-task missions while leveraging demonstrations from single-goal-reaching tasks. Our code is available at https://github.com/CL2-UWaterloo/GenPlan.

Figures

Figures reproduced from arXiv: 2412.08565 by the authors.

Figure 1
Figure 1. Overview. GenPlan is a generative, multi-step planner that optimizes energy landscape to adapt to complex tasks and iteratively refine long-horizon missions. Goals are highlighted in yellow, and distractors are marked in red. while avoiding getting stuck, unblocking paths, and navigat￾ing around walls to complete the mission. Learning such behaviors from demonstrations is challeng￾ing and often requires a diverse da… view at source ↗
Figure 2
Figure 2. Method Overview. GenPlan, trained on offline data (A), learns to jointly model action, goal, and state distributions. In (B), the joint denoising model (see Section 4) takes in a corrupted trajectory τ t and predicts the clean trajectory τ 1 . (C) Demon￾strates the joint inference of goals and actions by simulating the reverse CTMC, as detailed in Algorithm 2. erative model’s adaptability to new tasks and environmen… view at source ↗
Figure 3
Figure 3. Energy Landscape. GenPlan, when conditioned on sub-goals, implicitly assigns minimal energy to neces￾sary sub-goals (e.g., picking up keys, opening doors) for task completion. States closer to the white region are more likely to be transitioned into. LEAP, in contrast, does not prioritize these sub-tasks. ios—the goal generation module in GenPlan dynamically updates sub-goals, enabling better state coverage (see Ap￾… view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Trajectory Planning (TP). The agent is randomly initialized and must navigate through a maze-like environment to reach the goal(s). In each evaluation, the map layout, the agent’s initial position, and goal positions are varied. Note that in the case of GenPlan, the ag…
Figure 5
Figure 5. Figure 5: Instruction Completion IC. These tasks are more complex than TP, requiring the agent to perform additional actions such as pickup, drop, and open in addition to navigation. For example, in DoorsOpen, the agent must open the specified doors in the correct order without …
Figure 6
Figure 6. Figure 6: Adaptive Planning AP. The model is trained on simpler tasks and tested for adaptability by progressively increasing task difficulty. In GotoObjMazeClose and GoToSeqUn, we evaluate whether the model can demonstrate novel skills, such as unblocking obstacles and opening …
Figure 7
Figure 7. Figure 7: GenPlan I/O. During planning, the model takes in a corrupted trajectory along with the current state and past observations (if available), and iteratively recovers the clean trajectory. Energy Landscape. We investigate the energy assignment to trajectories with varying…
Figure 8
Figure 8. Figure 8: State Coverage (AP). State visit frequency is evaluated across 10 unseen maze layouts with varying goal positions (Rooms 1-9), starting from a fixed agent position (Room 1). Success Rate: 52%, Coverage: 85.47% GoToObjMazeS7G1 GenPlan-M LEAP DT Success Rate: 14%, Covera…
Figure 9
Figure 9. Figure 9: State Coverage (TP). Coverage is evaluated in larger environments. Results are reported across 50 map variations from a fixed start position (Room 1). (A) (B) [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: GenPlan Analysis. (A) Success rate as a function of the number of denoising iterations, reported using 3 different training seeds evaluated on 250 environments, specifically for GoToObjMazeS4N3G1 (AP) and GoToObjMazeS4G1 (TP). (B) Energy landscape learned by GenPlan d…
Figure 11
Figure 11. Figure 11: GenPlan Configurations. (A) Training progression of GenPlan, showing that GenPlan-M converges faster than GenPlan-U. (B) Success rate as a function of entropy. Stochasticity in Environments. In this setup, the agent operates in an environment where it has a 20% chance…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

33 extracted references · 14 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Campbell, A.; Benton, J.; De Bortoli, V.; Rainforth, T.; Deligiannidis, G.; and Doucet, A. 2022. A continuous time framework for discrete denoising models. Advances in Neural Information Processing Systems

    work page 2022

  4. [4]

    Campbell, A.; Yim, J.; Barzilay, R.; Rainforth, T.; and Jaakkola, T. 2024. Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Design. arXiv:2402.04997

    arXiv 2024

  5. [5]

    B.; and Vela, P

    Chen, H.; Du, Y.; Chen, Y.; Tenenbaum, J. B.; and Vela, P. A. 2023. Planning with Sequence Models through Iterative Energy Minimization. In ICLR

    work page 2023

  6. [6]

    Chen, L.; Lu, K.; Rajeswaran, A.; Lee, K.; Grover, A.; Laskin, M.; Abbeel, P.; Srinivas, A.; and Mordatch, I. 2021. Decision Transformer: Reinforcement Learning via Sequence Modeling. In Advances in Neural Information Processing Systems, volume 34, 15084--15097

    work page 2021

  7. [7]

    H.; and Bengio, Y

    Chevalier-Boisvert, M.; Bahdanau, D.; Lahlou, S.; Willems, L.; Saharia, C.; Nguyen, T. H.; and Bengio, Y. 2019. Baby AI : First Steps Towards Grounded Language Learning With a Human In the Loop. In International Conference on Learning Representations

    2019

  8. [8]

    Chi, C.; Feng, S.; Du, Y.; Xu, Z.; Cousineau, E.; Burchfiel, B.; and Song, S. 2023. Diffusion Policy : Visuomotor Policy Learning via Action Diffusion . ArXiv:2303.04137 [cs]

    arXiv 2023

Show all 33 references
  1. [9]

    Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

    2018 arXiv

  2. [10]

    Du, Y.; Mao, J.; and Tenenbaum, J. B. 2024. Learning Iterative Reasoning through Energy Diffusion. In Forty-first International Conference on Machine Learning

    2024

  3. [11]

    Emmons, S.; Eysenbach, B.; Kostrikov, I.; and Levine, S. 2022. RvS: What is Essential for Offline RL via Supervised Learning? In International Conference on Learning Representations

    2022

  4. [12]

    Eysenbach, B.; Zhang, T.; Levine, S.; and Salakhutdinov, R. 2022. Contrastive Learning as Goal-Conditioned Reinforcement Learning. In Oh, A. H.; Agarwal, A.; Belgrave, D.; and Cho, K., eds., Advances in Neural Information Processing Systems

    2022

  5. [13]

    Florence, P.; Lynch, C.; Zeng, A.; Ramirez, O.; Wahid, A.; Downs, L.; Wong, A.; Lee, J.; Mordatch, I.; and Tompson, J. 2021. Implicit Behavioral Cloning. Conference on Robot Learning (CoRL)

    2021

  6. [14]

    Fujimoto, S.; Meger, D.; and Precup, D. 2019. Off-Policy Deep Reinforcement Learning without Exploration. In International Conference on Machine Learning, 2052--2062

    2019

  7. [15]

    Furuta, H.; Matsuo, Y.; and Gu, S. S. 2022. Generalized Decision Transformer for Offline Hindsight Information Matching. In International Conference on Learning Representations

    2022

  8. [16]

    Goyal, K.; Dyer, C.; and Berg-Kirkpatrick, T. 2021. Exposing the Implicit Energy Networks behind Masked Language Models via Metropolis--Hastings. arXiv preprint arXiv:2106.02736

    2021 arXiv

  9. [17]

    Gupta, A.; Kumar, V.; Lynch, C.; Levine, S.; and Hausman, K. 2019. Relay policy learning: Solving long-horizon tasks via imitation and reinforcement learning. arXiv preprint arXiv:1910.11956

    2019 arXiv

  10. [18]

    Haarnoja, T.; Tang, H.; Abbeel, P.; and Levine, S. 2017. Reinforcement Learning with Deep Energy-Based Policies. arXiv:1702.08165

    2017 arXiv

  11. [19]

    Ho, J.; Jain, A.; and Abbeel, P. 2020. Denoising Diffusion Probabilistic Models. arXiv:2006.11239

    2020 arXiv

  12. [20]

    B.; and Levine, S

    Janner, M.; Du, Y.; Tenenbaum, J. B.; and Levine, S. 2022. Planning with Diffusion for Flexible Behavior Synthesis. arXiv:2205.09991

    2022 arXiv

  13. [21]

    Janner, M.; Li, Q.; and Levine, S. 2021. Offline Reinforcement Learning as One Big Sequence Modeling Problem

    2021

  14. [22]

    Kostrikov, I.; Nair, A.; and Levine, S. 2021. Offline reinforcement learning with implicit q-learning. arXiv preprint arXiv:2110.06169

    2021 arXiv

  15. [23]

    Kumar, A.; Fu, J.; Soh, M.; Tucker, G.; and Levine, S. 2019. Stabilizing off-policy q-learning via bootstrapping error reduction. Advances in Neural Information Processing Systems, 32

    2019

  16. [24]

    Lambert, N.; Pister, K.; and Calandra, R. 2022. Investigating Compounding Prediction Errors in Learned Dynamics Models. arXiv:2203.09637

    2022 arXiv

  17. [25]

    J.; Shafiullah, N

    Lee, S.; Wang, Y.; Etukuru, H.; Kim, H. J.; Shafiullah, N. M. M.; and Pinto, L. 2024. Behavior Generation with Latent Actions. arXiv preprint arXiv:2403.03181

    2024 arXiv

  18. [26]

    Levine, S.; Kumar, A.; Tucker, G.; and Fu, J. 2020. Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems. arXiv:2005.01643

    2020 arXiv

  19. [27]

    Meng, L.; Wen, M.; Yang, Y.; Le, C.; Li, X.; Zhang, W.; Wen, Y.; Zhang, H.; Wang, J.; and Xu, B. 2022. Offline Pre-trained Multi-Agent Decision Transformer: One Big Sequence Model Tackles All SMAC Tasks. arXiv:2112.02845

    2022 arXiv

  20. [28]

    Perez, E.; Strub, F.; De Vries, H.; Dumoulin, V.; and Courville, A. 2018. Film: Visual reasoning with a general conditioning layer. In Proceedings of the AAAI Conference on Artificial Intelligence

    2018

  21. [29]

    Schmidhuber, J. 2020. Reinforcement Learning Upside Down: Don't Predict Rewards -- Just Map Them to Actions. arXiv:1912.02875

    2020 arXiv

  22. [30]

    Schmied, T.; Hofmarcher, M.; Paischer, F.; Pascanu, R.; and Hochreiter, S. 2023. Learning to Modulate pre-trained Models in RL . In Thirty-seventh Conference on Neural Information Processing Systems

    2023

  23. [31]

    Sun, J.; Huang, D.-A.; Lu, B.; Liu, Y.-H.; Zhou, B.; and Garg, A. 2022. PlaTe: Visually-Grounded Planning With Transformers in Procedural Tasks. IEEE Robotics and Automation Letters, 7(2): 4924--4930

    2022

  24. [32]

    Yu, T.; Thomas, G.; Yu, L.; Ermon, S.; Zou, J.; Levine, S.; Finn, C.; and Ma, T. 2020. MOPO: Model-based Offline Policy Optimization. arXiv preprint arXiv:2005.13239

    2020 arXiv

  25. [33]

    Zheng, Q.; Zhang, A.; and Grover, A. 2022. Online decision transformer. In International Conference on Machine Learning. PMLR

    2022

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.