Pith. sign in

REVIEW 3 major objections 4 minor 61 references

Prompting Decision Transformers for Zero-Shot Reach-Avoid Policies

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A decision transformer learns reach-avoid policies from random data and dodges unseen hazard boxes at evaluation time.

desk verdict The paired hindsight relabeling may teach the model to ignore avoid prompts, so the zero-shot results need a direct prompt-dependence test before the central claim holds. read the letter →

arxiv 2505.19337 v2 pith:36FEO7O7 submitted 2025-05-25 cs.LG cs.AIq-bio.QM

Kevin Li , Marinka Zitnik This is my paper
classification cs.LGcs.AIq-bio.QM
keywords offlinereinforcementlearningdecisiontransformerreach-avoidhindsightrelabelingpromptconditioningzero-shotgeneralizationgoal-conditionedRLsafesequentialdecision-making
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces RADT, a decision-transformer model that treats reach and avoid specifications as prompt tokens: a goal state and any number of box-shaped avoid regions are fed to the model as tokens, so the same trained network can be asked at evaluation time to avoid hazards it never saw during training. The authors aim to show that this prompting design removes the need for reward and cost engineering in offline reach-avoid learning, and that a model trained only on suboptimal random-policy trajectories can match or beat prior methods that are retrained for each new hazard configuration. They report that in one zero-shot setting RADT improves normalized cost by 35.7% over the best retrained baseline while keeping goal-reaching success high, and that the same approach reduces visits to unsafe intermediate states in a discrete, stochastic model of cell reprogramming.

What carries the argument

The load-bearing mechanism is a prompt made of six token types: an avoid-success indicator z, an avoid-start token, one box vector per avoid region (lower and upper bounds in each state dimension), a goal-start token, the goal state, and an end-of-prompt token. On top of this, the training procedure applies two-pass hindsight avoid relabeling: each random trajectory is labeled with a sampled set of avoid boxes it happens to miss (z=1), and a copy is made with boxes resampled until the trajectory violates one (z=0). This paired dataset, combined with a language-modeling loss and an auxiliary box-awareness loss, is what teaches the transformer what 'avoid this box' means without any cost signal.

What would settle it

Compare paired training copies with the same start and goal: if the z=1 copy reliably has shorter or straighter paths than its z=0 copy, then the avoid token could be correlated with path geometry; training on a dataset where that correlation is removed (e.g., by matching trajectory lengths across the pair) and observing whether zero-shot avoidance disappears would settle whether the model learned avoidance itself.

Watch

Extended reading notes

Core claim

RADT's central claim is that reach-avoid behavior can be learned entirely offline, without reward or cost functions, by relabeling random trajectories in hindsight and by conditioning a causal transformer on explicit goal and avoid tokens. Avoid regions are encoded as box corner coordinates, and the prompt carries an avoid-success bit z; training data is built in two passes so that each trajectory has a paired copy with the opposite z, isolating the concept of avoidance from trajectory content. At evaluation the model is always prompted with z=1 and with whatever boxes the user wants, so a single checkpoint generalizes to out-of-distribution hazard sizes and counts. The paper reports experiments in two simulated robot tasks and in a Boolean-network model of heart-field reprogramming, where RADT either finds alternate paths around the specified state or, when the state is unavoidable, spends less time in it.

Load-bearing premise

The method assumes that a random trajectory that happens to miss a randomly placed box is a valid demonstration of how to avoid that box, so the paired z=1/z=0 copies teach the avoid concept rather than some unrelated difference between the copies.

Editorial extensions

If this is right

  • A single trained model can be redeployed to environments with new hazard counts, locations, and sizes by changing only the prompt tokens, with no retraining or fine-tuning.
  • Reach-avoid policies become learnable from cheap, purely random data, which matters in safety-critical domains where expert demonstrations and online exploration are unavailable.
  • The same prompt-conditioned policy transfers to discrete, stochastic state spaces, as shown by reduced visitation of unsafe gene-expression states in the Boolean-network reprogramming model.
  • The avoid-success bit provides a clean, interpretable interface for specifying avoidance strength at test time, since the model is always conditioned on z=1 when deployed.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves non-box hazard shapes unimplemented; a natural extension is to represent arbitrary regions as sets or unions of box tokens, which the prompt format could in principle support without architectural change.
  • The paired z=0/z=1 construction risks teaching a spurious correlate of avoidance if the two copies differ systematically in path length or geography; an ablation that matches paired trajectories on those covariates would test whether the model truly learns avoidance.
  • A testable extension beyond the reported benchmarks is to perturb the trajectory distribution, for example using straight-line random walks with no circumnavigation geometry, and checking whether zero-shot avoidance collapses.
  • The cell-reprogramming case suggests a broader use: specifying 'do not visit high-risk intermediate states' as prompt tokens could allow safety constraints to be adapted per patient without retraining.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces RADT, a decision-transformer model that encodes goals and box-shaped avoid regions as prompt tokens and is trained offline on random-policy trajectories using hindsight goal relabeling and a two-pass hindsight avoid-region relabeling scheme. At evaluation, the model is conditioned on a binary avoid-success token and on arbitrary avoid-box coordinates, with the claimed ability to generalize zero-shot to out-of-distribution avoid-region sizes and counts. The empirical sections report results on FetchReachObstacle, MazeObstacle, and a Boolean-network cell-reprogramming environment, comparing RADT against RbSL, AM-Lag, and WGCSL. The core claims are that RADT learns reach-avoid behavior from suboptimal, reward-free data and that it matches or outperforms baselines that are retrained for each test configuration.

Significance. If the central mechanism is real, the contribution is significant: RADT would provide a flexible, prompt-based way to specify reach-avoid objectives at test time without reward or cost engineering, using only suboptimal offline data. The evaluation design has real strengths: the OOD box sizes and counts are genuinely held out, the baselines are retrained for each test condition, and results are reported with three-seed standard deviations. However, the paper does not currently demonstrate that action selection is actually driven by the avoid prompt, and the main evaluation metric is vulnerable to a trajectory-length confound. Until those points are addressed, the empirical tables do not establish the paper's central claim.

major comments (3)
  1. [Section 4.2 and Appendix B.2] The two-pass relabeling creates paired examples (tau_orig, tau_copy) with identical state-action sequences, different avoid-box tokens, and opposite z labels. Because the action-prediction loss L_action in Appendix B.2 uses the same target actions for both members of every pair, the action head can minimize L_action by ignoring the box and z prompt entirely; any prompt-dependent action change would increase loss on one of the two paired examples. The only prompt-dependent term, L_avoid_awareness, is a per-step binary classification of whether the current state lies inside any avoid box; it does not reward actions that keep future states out of the specified box. The zero-shot results in Tables 1 and 2 therefore do not establish that the model's behavior is prompt-driven. Please provide a direct test of prompt-conditioned action variation (for example, evaluating the same state history with different avoid boxes or with z=0 versus z=1), or modify the training objective so that the action head is explicitly incentivized to condition on the avoid specification.
  2. [Appendix C.1 and Tables 1-2] The main cost metric MNC is length-normalized, so a policy that takes longer or more meandering trajectories can reduce MNC without genuinely avoiding the specified regions. The paper motivates this metric with the 'rushed policy' versus 'cautious policy' example, but the robotics experiments report only MNC and success rate, not trajectory length or absolute cost. The cell-reprogramming Scenario 1 explicitly reports that avoidance is accompanied by significantly longer trajectories. To confirm that the reported MNC improvements reflect avoidance rather than dilution, please report absolute cost and trajectory length for the robotics experiments, or use a metric that controls for path duration.
  3. [Section 5.2 and Table 2] The MazeObstacle training protocol is ambiguous. The text says the authors 'begin by training all models using data generated in environments with a single avoid region,' then 'train and evaluate all models in environments with three avoid regions,' and then state that RADT is evaluated zero-shot without retraining. It is unclear whether RADT was trained on one or three avoid regions for the 4-7 avoid-region tests, and whether row 3 of Table 2 is in-distribution or out-of-distribution for RADT. This ambiguity matters for the zero-shot count-generalization claim. Please specify the exact RADT checkpoint and training distribution used for each row of Table 2.
minor comments (4)
  1. [Section 5.1, Table 1] At the in-distribution width 0.16, RADT's MNC is 0.049 +/- 0.016 and AM-Lag's is 0.011 +/- 0.003; the statement that RADT 'performs comparably to AM-Lag in MNC' is not supported by these numbers and should be qualified or supported with a statistical comparison.
  2. [Tables 1 and 2] Many OOD comparisons have overlapping error bars across three seeds (for example, Maze rows 4-7), so the claim that RADT 'outperforms' or 'matches' retrained baselines should be accompanied by significance tests, effect sizes, or a clear statement of which differences are within noise.
  3. [Appendix A.1 and Appendix D] The manuscript states that datasets and code 'will be released' but currently provides no links; the experiments are therefore not independently reproducible from the paper as submitted.
  4. [Throughout] There are several formatting and typographical issues: 'raDT' appears in Figure 1, 'Cariogenesis' appears in Appendix A.2.2, the symbol '∫⊔' appears where s_t is meant in Appendix B.2, and Figure 7 reports an 'obstacle-avoiding success rate' that is never defined in the main text.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the OOD evaluations are genuinely held out, and no fitted parameter or training label encodes the test configurations.

full rationale

RADT's central claims are that (i) avoid-region hindsight relabeling lets a decision transformer trained on random-policy trajectories produce reach-avoid behavior, and (ii) this behavior transfers zero-shot to out-of-distribution box sizes and counts. Neither claim reduces to its own inputs by construction. The training labels z and k_t are generated by deterministically checking whether sampled box coordinates intersect trajectory states (Section 4.2 and Appendix A.2.1), and L_avoid_awareness supervises exactly this membership indicator (Appendix B.2). The evaluation metric MNC is the average of the same indicator over rollouts. This metric-label alignment is standard task-aligned supervision, not circular fitting: the model weights are not fit to any parameter encoding the test box sizes or counts (widths 0.18-0.24 in Table 1; 4-7 avoid regions in Table 2). Those configurations are held out, while RADT is evaluated with a fixed model; baselines are retrained per configuration, which if anything biases the comparison against RADT. Checkpoint and hyperparameter selection use in-distribution evaluations (Appendix B.2, Stopping Criteria and Table 3), not the OOD test configurations. The citations to MGPO [49] and RbSL [3] are architectural or baseline references, not load-bearing self-citations, and no uniqueness theorem is imported from the authors' prior work. The skeptic's concern that the two-pass paired relabeling gives L_action no incentive to make action predictions prompt-dependent is an empirical/correctness question about whether the observed avoidance is truly prompt-driven, not a logical reduction of the claimed prediction to its inputs. Under the required standard of quoting a specific reduction, no circular step is identifiable; a non-finding with score 0 is therefore the appropriate verdict.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim depends on a few design choices: the maximum relabeled box width that defines the training distribution, attention-boosting bias, architecture selected by HPO, and, for the biology case, the Boolean network model taken as ground truth. The method introduces no new physical entities; the paired relabeling is an algorithmic construct. The most fragile input is the assumption that random trajectories, after relabeling, contain a usable avoidance signal.

free parameters (6)
  • w_max (maximum hindsight-relabeled avoid box width) = 0.16 for FetchReachObstacle; 0.4 (circumscribing box for radius-0.2 circles) for MazeObstacle
    Controls the training distribution of avoid box sizes. OOD test widths in Fetch are 0.18 to 0.24, i.e., 1.125x to 1.5x w_max. Not fitted to test results, but a design choice that defines what out-of-distribution means.
  • adelta (attention-boosting bias for prompt tokens) = 2 (Fetch), 1 (Maze), 1 (Cardiogenesis)
    Tuned by hyperparameter optimization; appendix Figure 7 shows it raises goal-reaching success rate.
  • architecture hyperparameters (n_head, n_layer, embed_dim) = {4,4,256} for Fetch; {6,6,384} for Maze and Cardiogenesis
    Chosen via RayTune random search per environment, not derived from first principles.
  • alpha1 (loss weight for avoid-awareness BCE) = 1
    Authors state it has no significant effect after enough training, so it is fixed to 1 in all experiments.
  • epsilon (box margin for discrete avoid states) = 0.001
    Arbitrary margin used to convert discrete cardiogenesis states into boxes; the authors call it arbitrary.
  • k (asynchronous Boolean network updates per transition) = 10
    Affects transition stochasticity in the Cardiogenesis environment; chosen by the authors, not swept.
assumptions (5)
  • standard math A GPT-2-style causal transformer can represent a conditional policy from prompt and trajectory tokens.
    Relies on the standard Transformer architecture and gradient training, as described in Section 4 and Appendix B.
  • domain assumption The 15-gene Boolean network model of mouse cardiogenesis is a valid stand-in for cell reprogramming dynamics.
    The biology application treats this model as ground-truth environment dynamics, drawn from cited literature [15, 36] and used in Section 5.3 and Appendix C.2.
  • ad hoc to paper Random-policy trajectories plus hindsight relabeling contain enough signal to learn goal-reaching and avoidance.
    The method's training signal depends on this assumption; the contour-based sampling in A.2.2 is introduced to compensate for insufficient signal.
  • domain assumption Length-normalized cost MNC is the right measure of avoidance quality.
    Discussed in Appendix C.1 versus absolute cost; it favors cautious over rushed policies, which is a value judgment.
  • domain assumption Avoid regions can be faithfully represented as axis-aligned boxes.
    Used everywhere in the method; explicitly acknowledged as a limitation in the Limitations section.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Prompting Decision Transformers for Zero-Shot Reach-Avoid Policies." pith.science (2026). https://pith.science/paper/36FEO7O7

@misc{pith2026250519337,
  author       = {Pith},
  title        = {Pith review of: Prompting Decision Transformers for Zero-Shot Reach-Avoid Policies},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/36FEO7O7}},
  note         = {Machine review of arXiv:2505.19337}
}
read the original abstract

Offline goal-conditioned reinforcement learning methods have shown promise for reach-avoid tasks, where an agent must reach a target state while avoiding undesirable regions of the state space. Existing approaches typically encode avoid-region information into an augmented state space and cost function, which prevents flexible, dynamic specification of novel avoid-region information at evaluation time. They also rely heavily on well-designed reward and cost functions, limiting scalability to complex or poorly structured environments. We introduce RADT, a decision transformer model for offline, reward-free, goal-conditioned, avoid region-conditioned RL. RADT encodes goals and avoid regions directly as prompt tokens, allowing any number of avoid regions of arbitrary size to be specified at evaluation time. Using only suboptimal offline trajectories from a random policy, RADT learns reach-avoid behavior through a novel combination of goal and avoid-region hindsight relabeling. We benchmark RADT against 3 existing offline goal-conditioned RL models across 11 tasks, environments, and experimental settings. RADT generalizes in a zero-shot manner to out-of-distribution avoid region sizes and counts, outperforming baselines that require retraining. In one such zero-shot setting, RADT achieves 35.7% improvement in normalized cost over the best retrained baseline while maintaining high goal-reaching success. We apply RADT to cell reprogramming in biology, where it reduces visits to undesirable intermediate gene expression states during trajectories to desired target states, despite stochastic transitions and discrete, structured state dynamics.

Figures

Figures reproduced from arXiv: 2505.19337 by the authors.

Figure 1
Figure 1. (a) An ideal reach-avoid model should learn to avoid arbitrarily specified regions of varying number and size at evaluation time, using only suboptimal, random-policy training data. (b) RADT is the only model that satisfies all criteria for an ideal reach-avoid learner (Section 2). These settings require flexible and interpretable models that support zero-shot generalization to unseen goal and avoid specifications w… view at source ↗
Figure 2
Figure 2. (a) RADT receives goal states and avoid regions as prompt inputs. (b) Avoid regions are defined as boxes in the state space and encoded as vectors of bounding box corner coordinates. (c) For each offline trajectory, we generate two versions: one that violates a sampled avoid region and one that avoids it. Both are labeled with an avoid success token z. (d) Prior models encode avoid regions via augmented state vector… view at source ↗
Figure 3
Figure 3. (a) Visualization of the FetchReachObstacle environment. The red point is the goal; the blue box is the avoid region. (b) Unlike prior setups, the robot arm can pass through avoid boxes, allowing training data to include violations. (c) RADT and AM-Lag achieve state-of-the-art reach-avoid performance on in-distribution box sizes, measured by MNC and SR. (d) RADT generalizes zero-shot to out-of-distribution avoid box… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: (a) Visualization of the MazeObstacle environment, with red goal, blue avoid regions, and green agent. (b) RADT outperforms all baselines on MNC and SR in the in-distribution single-avoid setting. (c) RADT generalizes zero-shot to out-of-distribution numbers of avoid r…
Figure 5
Figure 5. Figure 5: (a) Cell reprogramming involves sequential gene perturbations to reach a target expression state while avoiding unsafe intermediate states. (b) Evaluation pipeline: RADT is first run without an avoid token. The most frequently visited intermediate state (e.g., gray cel…
Figure 6
Figure 6. Figure 6: a) 2D depiction of the contour-based centroid sampling strategy for avoiding region relabeling. The convex and concave hulls are calculated for the set of data points in the state space for a training trajectory. Points that are part of the concave hull but not the con…
Figure 7
Figure 7. Figure 7: The maximum goal-reaching success rate for RADT trained on the FetchReachObstacle task improves with increasing the attention boosting bias adelta to the prompt tokens. The maximum obstacle-avoiding success rate, on the other hand, seems to be unaffected by the value o…
Figure 8
Figure 8. Figure 8: A policy that rushes directly through the avoid region to get to the goal as quickly as possible may earn a low absolute cost return but a high length-normalized cost return. On the other hand, a slower, more cautious policy that takes more timesteps to reach the goal …
Figure 9
Figure 9. Figure 9: a) A diagram representing the 15-gene boolean network model for mouse cardiogenesis. b) A depiction of one state-action-state transition simulated in the Cardiogenesis environment, where k = 4. 24 [PITH_FULL_IMAGE:figures/full_fig_p024_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

61 extracted references · 47 canonical work pages

  1. [1]

    A review of reward functions for reinforcement learning in the context of autonomous driving

    Ahmed Abouelazm, Jonas Michel, and J Marius Zöllner. A review of reward functions for reinforcement learning in the context of autonomous driving. In2024 IEEE Intelligent Vehicles Symposium (IV). IEEE, June 2024

    work page 2024

  2. [2]

    Hindsight experience replay

    Marcin Andrychowicz, Filip Wolski, Alex Ray, Jonas Schneider, Rachel Fong, Peter Welinder, Bob McGrew, Josh Tobin, Pieter Abbeel, and Wojciech Zaremba. Hindsight experience replay. 2017

    work page 2017

  3. [3]

    Offline Goal-Conditioned Reinforcement Learning for Safety-Critical Tasks with Recovery Policy

    Chenyang Cao, Zichen Yan, Renhao Lu, Junbo Tan, and Xueqian Wang. Offline goal- conditioned reinforcement learning for safety-critical tasks with recovery policy.arXiv preprint arXiv:2403.01734, 2024

    work page Pith review arXiv 2024

  4. [4]

    Actionable models: Unsupervised offline reinforcement learning of robotic skills.arXiv preprint arXiv:2104.07749, 2021

    Yevgen Chebotar, Karol Hausman, Yao Lu, Ted Xiao, Dmitry Kalashnikov, Jake Varley, Alex Irpan, Benjamin Eysenbach, Ryan Julian, Chelsea Finn, and Sergey Levine. Actionable models: Unsupervised offline reinforcement learning of robotic skills.arXiv preprint arXiv:2104.07749, 2021

    arXiv 2021

  5. [5]

    Decision transformer: Reinforcement learning via sequence modeling.arXiv preprint arXiv:2106.01345, 2021

    Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Michael Laskin, Pieter Abbeel, Aravind Srinivas, and Igor Mordatch. Decision transformer: Reinforcement learning via sequence modeling.arXiv preprint arXiv:2106.01345, 2021

    arXiv 2021

  6. [6]

    Gymnasium robotics, 2024

    Rodrigo de Lazcano, Kallinteris Andreas, Jun Jet Tai, Seungjae Ryan Lee, and Jordan Terry. Gymnasium robotics, 2024

    2024

  7. [7]

    Contrastive learning as goal-conditioned reinforcement learning.arXiv preprint arXiv:2206.07568, 2022

    Benjamin Eysenbach, Tianjun Zhang, Ruslan Salakhutdinov, and Sergey Levine. Contrastive learning as goal-conditioned reinforcement learning.arXiv preprint arXiv:2206.07568, 2022

    arXiv 2022

  8. [8]

    Safe multi-agent navigation guided by goal- conditioned safe reinforcement learning, 2025

    Meng Feng, Viraj Parimi, and Brian Williams. Safe multi-agent navigation guided by goal- conditioned safe reinforcement learning, 2025

    work page 2025

Show all 61 references
  1. [9]

    Curriculum reinforcement learning for complex reward functions

    Kilian Freitag, Kristian Ceder, Rita Laezza, Knut Åkesson, and Morteza Haghir Chehreghani. Curriculum reinforcement learning for complex reward functions. 2024

    2024

  2. [10]

    Off-policy deep reinforcement learning without exploration

    Scott Fujimoto, David Meger, and Doina Precup. Off-policy deep reinforcement learning without exploration. InInternational Conference on Machine Learning, pages 2052–2062, 2019

    2019

  3. [11]

    Integrating domain knowledge for handling limited data in offline RL

    Briti Gangopadhyay, Zhao Wang, Jia-Fong Yeh, and Shingo Takamatsu. Integrating domain knowledge for handling limited data in offline RL. 2024

    2024

  4. [12]

    Learning to reach goals without reinforcement learning.ArXiv, abs/1912.06088, 2019

    Dibya Ghosh, Abhishek Gupta, Justin Fu, Ashwin Reddy, Coline Devin, Benjamin Eysen- bach, and Sergey Levine. Learning to reach goals without reinforcement learning.ArXiv, abs/1912.06088, 2019

    1912 arXiv

  5. [13]

    Bullet-safety-gym: A framework for constrained reinforcement learning

    Sven Gronauer. Bullet-safety-gym: A framework for constrained reinforcement learning. Technical report, mediaTUM, 2022

    2022

  6. [14]

    Chemical reprogramming of human somatic cells to pluripotent stem cells.Nature, 605(7909):325–331, May 2022

    Jingyang Guan, Guan Wang, Jinlin Wang, Zhengyuan Zhang, Yao Fu, Lin Cheng, Gaofan Meng, Yulin Lyu, Jialiang Zhu, Yanqin Li, Yanglu Wang, Shijia Liuyang, Bei Liu, Zirun Yang, Huanjing He, Xinxing Zhong, Qijing Chen, Xu Zhang, Shicheng Sun, Weifeng Lai, Yan Shi, Lulu Liu, Lipeng...

    2022

  7. [15]

    A boolean model of the cardiac gene regulatory network determining first and second heart field identity

    Franziska Herrmann, Alexander Groß, Dao Zhou, Hans A Kestler, and Michael Kühl. A boolean model of the cardiac gene regulatory network determining first and second heart field identity. PLoS One, 7(10):e46798, October 2012

    2012

  8. [16]

    Tomlin, and Jaime F

    Kai-Chieh Hsu ∗, Vicenç Rubies-Royo ∗, Claire J. Tomlin, and Jaime F. Fisac. Safety and liveness guarantees through reach-avoid reinforcement learning. InProceedings of Robotics: Science and Systems, Virtual, July 2021

    2021

  9. [17]

    Offline reinforcement learning as one big sequence modeling problem

    Michael Janner, Qiyang Li, and Sergey Levine. Offline reinforcement learning as one big sequence modeling problem. InAdvances in Neural Information Processing Systems, 2021. 11

    2021

  10. [18]

    Bradley Knox, Alessandro Allievi, Holger Banzhaf, Felix Schmitt, and Peter Stone

    W. Bradley Knox, Alessandro Allievi, Holger Banzhaf, Felix Schmitt, and Peter Stone. Reward (mis)design for autonomous driving.Artificial Intelligence, 316:103829, 2023

    2023

  11. [19]

    Bradley Knox and James MacGlashan

    W. Bradley Knox and James MacGlashan. How to specify reinforcement learning objectives. InFinding the Frame: An RLC Workshop for Examining Conceptual Frameworks, 2024

    2024

  12. [20]

    Offline reinforcement learning with implicit q-learning

    Ilya Kostrikov, Ashvin Nair, and Sergey Levine. Offline reinforcement learning with implicit q-learning. 2021

    2021

  13. [21]

    Stabilizing off-policy q-learning via bootstrapping error reduction

    Aviral Kumar, Justin Fu, George Tucker, and Sergey Levine. Stabilizing off-policy q-learning via bootstrapping error reduction

  14. [22]

    Should i run offline reinforcement learning or behavioral cloning? InInternational Conference on Learning Representations, 2022

    Aviral Kumar, Joey Hong, Anikait Singh, and Sergey Levine. Should i run offline reinforcement learning or behavioral cloning? InInternational Conference on Learning Representations, 2022

    2022

  15. [23]

    Batch policy learning under constraints

    Hoang Le, Cameron V oloshin, and Yisong Yue. Batch policy learning under constraints. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors,Proceedings of the 36th International Conference on Machine Learning, volume 97 ofProceedings of Machine Learning Research, pages 3703–...

    2019

  16. [24]

    COptiDICE: Offline constrained reinforcement learning via stationary distribution correction estimation

    Jongmin Lee, Cosmin Paduraru, Daniel J Mankowitz, Nicolas Heess, Doina Precup, Kee-Eung Kim, and Arthur Guez. COptiDICE: Offline constrained reinforcement learning via stationary distribution correction estimation. InInternational Conference on Learning Representations, 2022

    2022

  17. [25]

    Possible strategies to reduce the tumorigenic risk of reprogrammed normal and cancer cells.Int

    Ying-Chu Lin, Cha-Chien Ku, Kenly Wuputra, Chung-Jung Liu, Deng-Chyang Wu, Maki Satou, Yukio Mitsui, Shigeo Saito, and Kazunari K Yokoyama. Possible strategies to reduce the tumorigenic risk of reprogrammed normal and cancer cells.Int. J. Mol. Sci., 25(10):5177, May 2024

    2024

  18. [26]

    Datasets and benchmarks for offline safe reinforcement learning.Journal of Data-centric Machine Learning Research, 2024

    Zuxin Liu, Zijian Guo, Haohong Lin, Yihang Yao, Jiacheng Zhu, Zhepeng Cen, Hanjiang Hu, Wenhao Yu, Tingnan Zhang, Jie Tan, and Ding Zhao. Datasets and benchmarks for offline safe reinforcement learning.Journal of Data-centric Machine Learning Research, 2024

    2024

  19. [27]

    Learning latent plans from play.Conference on Robot Learning (CoRL), 2019

    Corey Lynch, Mohi Khansari, Ted Xiao, Vikash Kumar, Jonathan Tompson, Sergey Levine, and Pierre Sermanet. Learning latent plans from play.Conference on Robot Learning (CoRL), 2019

    2019

  20. [28]

    Offline goal-conditioned reinforcement learning via $f$-advantage regression

    Yecheng Jason Ma, Jason Yan, Dinesh Jayaraman, and Osbert Bastani. Offline goal-conditioned reinforcement learning via $f$-advantage regression. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors,Advances in Neural Information Processing Systems, 2022

    2022

  21. [29]

    Offline reinforcement learning with domain-unlabeled data

    Soichiro Nishimori, Xin-Qiang Cai, Johannes Ackermann, and Masashi Sugiyama. Offline reinforcement learning with domain-unlabeled data. 2024

    2024

  22. [30]

    Partial cellular reprogramming: A deep dive into an emerging rejuvenation technology.Aging Cell, 23(2):e14039, February 2024

    Patrick T Paine, Ada Nguyen, and Alejandro Ocampo. Partial cellular reprogramming: A deep dive into an emerging rejuvenation technology.Aging Cell, 23(2):e14039, February 2024

    2024

  23. [31]

    Chemical reprogramming takes the fast lane.Cell Stem Cell, 30(4):335–337, April 2023

    Emily J Park, Srikanth Kodali, and Bruno Di Stefano. Chemical reprogramming takes the fast lane.Cell Stem Cell, 30(4):335–337, April 2023

    2023

  24. [32]

    Ogbench: Benchmarking offline goal-conditioned rl

    Seohong Park, Kevin Frans, Benjamin Eysenbach, and Sergey Levine. Ogbench: Benchmarking offline goal-conditioned rl. InInternational Conference on Learning Representations (ICLR), 2025

    2025

  25. [33]

    HIQL: Offline goal- conditioned RL with latent states as actions

    Seohong Park, Dibya Ghosh, Benjamin Eysenbach, and Sergey Levine. HIQL: Offline goal- conditioned RL with latent states as actions. InThirty-seventh Conference on Neural Information Processing Systems, 2023

    2023

  26. [34]

    Epigenetic reprogramming as a key to reverse ageing and increase longevity.Ageing Res

    Beatriz Pereira, Francisca P Correia, Inês A Alves, Margarida Costa, Mariana Gameiro, Ana P Martins, and Jorge A Saraiva. Epigenetic reprogramming as a key to reverse ageing and increase longevity.Ageing Res. Rev., 95(102204):102204, March 2024. 12

    2024

  27. [35]

    Benchmarking Safe Exploration in Deep Reinforcement Learning

    Alex Ray, Joshua Achiam, and Dario Amodei. Benchmarking Safe Exploration in Deep Reinforcement Learning. 2019

    2019

  28. [36]

    Optimizing sequential gene expression modulation for cellular reprogramming - coupled boolean modeling and reinforcement learning based method

    Vivek Singh. Optimizing sequential gene expression modulation for cellular reprogramming - coupled boolean modeling and reinforcement learning based method. March 2024

    2024

  29. [37]

    Solving minimum-cost reach avoid using reinforcement learning

    Oswin So, Cheng Ge, and Chuchu Fan. Solving minimum-cost reach avoid using reinforcement learning. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  30. [38]

    Responsive safety in reinforcement learning by PID lagrangian methods

    Adam Stooke, Joshua Achiam, and Pieter Abbeel. Responsive safety in reinforcement learning by PID lagrangian methods. 2020

    2020

  31. [39]

    Induction of pluripotent stem cells from mouse embryonic and adult fibroblast cultures by defined factors.Cell, 126(4):663–676, August 2006

    Kazutoshi Takahashi and Shinya Yamanaka. Induction of pluripotent stem cells from mouse embryonic and adult fibroblast cultures by defined factors.Cell, 126(4):663–676, August 2006

    2006

  32. [40]

    Direct neuronal reprogramming: Bridging the gap between basic science and clinical application

    Lakshmy Vasan, Eunjee Park, Luke Ajay David, Taylor Fleming, and Carol Schuurmans. Direct neuronal reprogramming: Bridging the gap between basic science and clinical application. Front. Cell Dev. Biol., 9:681087, July 2021

    2021

  33. [41]

    Strategies and mechanisms of neuronal reprogramming.Brain Res

    Yue Wan and Yan Ding. Strategies and mechanisms of neuronal reprogramming.Brain Res. Bull., 199(110661):110661, July 2023

    2023

  34. [42]

    Safe decision transformer with learning-based constraints

    Ruhan Wang and Dongruo Zhou. Safe decision transformer with learning-based constraints. In Neurips Safe Generative AI Workshop 2024, 2024

    2024

  35. [43]

    Elastic decision transformer

    Yueh-Hua Wu, Xiaolong Wang, and Masashi Hamaya. Elastic decision transformer. 2023

    2023

  36. [44]

    Prevention of tumor risk associated with the reprogramming of human pluripotent stem cells.J

    Kenly Wuputra, Chia-Chen Ku, Deng-Chyang Wu, Ying-Chu Lin, Shigeo Saito, and Kazunari K Yokoyama. Prevention of tumor risk associated with the reprogramming of human pluripotent stem cells.J. Exp. Clin. Cancer Res., 39(1):100, June 2020

    2020

  37. [45]

    Constraints penalized q-learning for safe offline reinforcement learning.Proc

    Haoran Xu, Xianyuan Zhan, and Xiangyu Zhu. Constraints penalized q-learning for safe offline reinforcement learning.Proc. Conf. AAAI Artif. Intell., 36(8):8753–8760, June 2022

    2022

  38. [46]

    Joshua Tenenbaum, and Chuang Gan

    Mengdi Xu, Yikang Shen, Shun Zhang, Yuchen Lu, Ding Zhao, B. Joshua Tenenbaum, and Chuang Gan. Prompting decision transformer for few-shot policy generalization. InThirty-ninth International Conference on Machine Learning, 2022

    2022

  39. [47]

    Rethinking goal-conditioned supervised learning and its connection to offline RL

    Rui Yang, Yiming Lu, Wenzhe Li, Hao Sun, Meng Fang, Yali Du, Xiu Li, Lei Han, and Chongjie Zhang. Rethinking goal-conditioned supervised learning and its connection to offline RL. In International Conference on Learning Representations, 2022

    2022

  40. [48]

    Swapped goal-conditioned offline reinforcement learning

    Wenyan Yang, Huiling Wang, Dingding Cai, Joni Pajarinen, and Joni-Kristen Kämäräinen. Swapped goal-conditioned offline reinforcement learning. 2023

    2023

  41. [49]

    Pre-trained multi-goal transformers with prompt optimization for efficient online adaptation

    Haoqi Yuan, Yuhui Fu, Feiyang Xie, and Zongqing Lu. Pre-trained multi-goal transformers with prompt optimization for efficient online adaptation. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  42. [50]

    Online decision transformer.CoRR, abs/2202.05607, 2022

    Qinqing Zheng, Amy Zhang, and Aditya Grover. Online decision transformer.CoRR, abs/2202.05607, 2022

    2022 arXiv

  43. [51]

    attempting

    Yinan Zheng, Jianxiong Li, Dongjie Yu, Yujie Yang, Shengbo Eben Li, Xianyuan Zhan, and Jingjing Liu. Safe offline reinforcement learning with feasibility-guided diffusion model. In The Twelfth International Conference on Learning Representations, 2024. 13 A Data Prep A.1 Datas...

    2024

  44. [52]

    Constrained markov decision processes with total cost criteria: Occupation measures and primal LP.Math

    Eitan Altman. Constrained markov decision processes with total cost criteria: Occupation measures and primal LP.Math. Methods Oper. Res. (Heidelb.), 43(1):45–72, February 1996

    1996

  45. [53]

    An efficient algorith for determining the convex hull of a finite planar set.Inf

    R L Graham. An efficient algorith for determining the convex hull of a finite planar set.Inf. Process. Lett., 1(4):132–133, June 1972

    1972

  46. [54]

    Cosine annealing with warmup for pytorch

    Naoki Katsura and Federico Baldassarre. Cosine annealing with warmup for pytorch. https: //github.com/katsura-jp/pytorch-cosine-annealing-with-warmup.git, 2021

    2021

  47. [55]

    Tune: A research platform for distributed model selection and training.arXiv preprint arXiv:1807.05118, 2018

    Richard Liaw, Eric Liang, Robert Nishihara, Philipp Moritz, Joseph E Gonzalez, and Ion Stoica. Tune: A research platform for distributed model selection and training.arXiv preprint arXiv:1807.05118, 2018

    2018 arXiv

  48. [56]

    A new concave hull algorithm and concaveness measure for n-dimensional datasets.Journal of Information Science and Engineering, 29:379–392, 03 2013

    J.-S Park and S.-J Oh. A new concave hull algorithm and concaveness measure for n-dimensional datasets.Journal of Information Science and Engineering, 29:379–392, 03 2013

    2013

  49. [57]

    Language models are unsupervised multitask learners

    Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. 2019

    2019

  50. [58]

    Pay attention to what matters

    Pedro Luiz Silva, Fadhel Ayed, Antonio De Domenico, and Ali Maatouk. Pay attention to what matters. InMINT: Foundation Model Interventions, 2024

    2024

  51. [59]

    concave_hull.https://github.com/cubao/concave_hull.git, 2022

    Zhixiong Tang. concave_hull.https://github.com/cubao/concave_hull.git, 2022

    2022

  52. [60]

    11th EAI International Conference, ICCASA 2022 Vinh Long, Vietnam, October 27–28, 2022 Proceedings.04 2023

    Phan Vinh and Nguyen Dung.Context-Aware Systems and Applications. 11th EAI International Conference, ICCASA 2022 Vinh Long, Vietnam, October 27–28, 2022 Proceedings.04 2023

    2022

  53. [61]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mari...

    2020

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.