Pith. sign in

REVIEW 4 major objections 5 minor 30 references

RAD: Retrieval High-quality Demonstrations to Enhance Decision-making

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read RAD shows offline RL can generalize by retrieving high-return states and planning toward them with a condition-guided diffusion model.

desk verdict Promising retrieval-plus-diffusion idea for offline RL, but the paper as written is incomplete: the key guidance function in Eq. 7 is never defined or trained, and the reported average is below two baselines. read the letter →

arxiv 2507.15356 v3 pith:GCQDM5WN submitted 2025-07-21 cs.AI

Lu Guo , Yixiang Shan , Zhengbang Zhu , Qifan Liang , Lichang Song , Ting Long , Weinan Zhang , Yi Chang This is my paper · ORCID
classification cs.AI
keywords offlinereinforcementlearningretrieval-augmentedplanningdiffusionmodelstrajectorystitchinggoal-conditionedgenerationD4RLbenchmarkMuJoColocomotion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the limited generalization of offline reinforcement learning can be addressed by retrieving high-return states from the fixed dataset and planning toward them with a diffusion model, instead of augmenting the dataset with synthetic transitions. If true, an agent can escape low-reward or out-of-distribution states by stitching its current trajectory to a high-return demonstration without extra environment interaction. RAD's design is evaluated on D4RL MuJoCo tasks, where it is reported to achieve competitive or superior normalized returns relative to prior diffusion- and augmentation-based methods. The load-bearing premise is that reachable high-return states exist in the dataset and can be found by similarity search.

What carries the argument

The central object is the conditioning pair cond = $\{(0, s_t), (\hat{i}, s_g)\}$: the current state anchored at position 0 and the retrieved high-return target anchored at the predicted step count $\hat{i}$. The target selection module scores candidate states by cosine similarity (Eq. 9), keeps those whose returns lie within a tolerance of the best candidate (Eq. 10), and picks the one with the longest remaining trajectory (Eq. 11). The step estimation module is a multilayer perceptron $f_{\psi}$ that predicts the step span (Eq. 12). The condition-guided diffusion model then denoises a trajectory segment with these anchors while a return predictor $J_{\phi}$ steers generation (Eq. 7); training uses pseudo-targets at random offsets along demonstrations (Eq. 14) so the model learns variable-horizon planning.

What would settle it

A direct test: build an offline navigation dataset where high-return goal states are cosine-similar to the starting states but separated by a barrier, such as a wall that requires a long detour. If RAD plans straight toward those targets and its advantage over a non-retrieval diffusion baseline shrinks or vanishes, the reachability premise is doing the work; if it still succeeds, the method is achieving stitching beyond its stated retrieval rationale.

Watch

Extended reading notes

Core claim

On its own terms, RAD's discovery is that offline RL can be turned into a retrieval-augmented generative planning problem: rather than trying to cover the state space with generated data, the agent looks up a high-return state in the fixed dataset, predicts how many steps it will take to get there, and asks a conditional diffusion model to fill in the intervening trajectory. Because the retrieved target sits on a high-return trajectory, once the agent reaches it, the remaining expert actions carry it to high reward. This turns trajectory stitching into a retrieval-and-generation operation that adapts at decision time, and the paper reports that this matches or exceeds static augmentation and diffusion baselines on D4RL MuJoCo tasks.

Load-bearing premise

The central bet is that for every low-reward or out-of-distribution state the agent visits, the offline dataset contains a high-return state that is reachable and whose remaining actions are still valuable; retrieval checks similarity and return but never verifies reachability.

Editorial extensions

If this is right

  • Offline RL agents can generalize to states not covered by the training data by planning toward retrieved high-return targets, without generating synthetic transitions.
  • Trajectory stitching becomes a retrieval-and-generation step: the agent escapes low-reward regions by reaching a state on a high-return trajectory and then following its remaining actions.
  • The method adapts at decision time, so new or out-of-distribution states encountered during deployment receive fresh guidance rather than relying on a fixed augmented dataset.
  • The step-estimation module keeps generated trajectories temporally coherent when the distance from current state to target varies.
  • On the evaluated D4RL MuJoCo tasks, the approach matches or exceeds existing diffusion and augmentation baselines, supporting retrieval-guided generation as a usable offline RL strategy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A dynamics-aware distance or reachability score could replace cosine similarity in the retrieval module; the gap in performance between the two would quantify how much the paper's reachability assumption drives RAD's success.
  • The same retrieval-and-conditioning pattern could serve as a subgoal generator for hierarchical or model-based offline RL, where the low-level policy or planner executes the segment to the retrieved state.
  • The method's sensitivity to the similarity threshold suggests that a learned embedding trained on transition statistics, rather than raw state features, would make retrieval more robust in noisy environments.
  • A testable extension is to compare RAD with a variant that validates targets with a learned dynamics model; if the validated variant improves in stochastic environments, it would confirm that reachability, not similarity, is the operative constraint.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes RAD, a method for offline reinforcement learning that retrieves high-return states from a static dataset and uses a condition-guided diffusion model to plan trajectories toward those states. The target selection module uses cosine similarity and trajectory return, and a learned step-estimation module predicts how many steps the plan should take to reach the target. The authors evaluate RAD on nine D4RL MuJoCo tasks, report ablations of the retrieval, conditioning, and step-estimation components, and claim competitive or superior performance relative to several offline RL baselines. The core algorithmic claim is that retrieval-guided diffusion planning can improve generalization beyond the training distribution by stitching OOD states to high-return regions.

Significance. The idea of combining non-parametric retrieval with diffusion-based planning is a reasonable and potentially useful direction for offline RL, and the paper makes an empirical contribution by demonstrating the approach on standard D4RL benchmarks. Strengths include the availability of code, experiments on nine tasks, and ablations that show the retrieval and step-estimation modules contribute to performance. However, the paper's central algorithmic component is underspecified, and the headline empirical claim is not fully supported by the reported numbers. If the method were fully specified and the claims calibrated, the paper would be of interest to the offline RL community; in its current form, the reproducibility of the described method is the main obstacle.

major comments (4)
  1. [§4.1, Eq. (7)] The guidance function J_phi is never defined. The text states that J_phi(·,·) predicts trajectory return, but §4.4 gives no architecture, training objective, or optimizer update for J_phi; only L_RAD(θ) in Eq. (14) and L_step(ψ) in Eq. (15) are specified. The gradient ∇J_phi in Eq. (7) therefore cannot be computed from the paper alone, and the 'condition-guided' component of RAD is not reproducible as written. The scaling factor ρ is also not specified. This is the central load-bearing part of the method, and it must be fully described, or the guidance term must be removed and the method presented without it.
  2. [§4.2, Eqs. (9)–(11) and Table 2] The retrieval procedure is incompletely specified. Eq. (9) selects the top-k candidates by cosine similarity, Eq. (10) filters by a return tolerance η, and Eq. (11) picks the target with the longest remaining trajectory. However, η is never given a value or a tuning procedure, and the similarity threshold δ from Table 2 does not appear in Eqs. (9)–(11) or anywhere in §4.2. The reader cannot determine how δ is applied during retrieval, for example as a cutoff before top-k selection. These parameters are needed to reproduce the method.
  3. [§5.1, Table 1] The empirical claim of 'superiority' is not supported by Table 1. RAD's average normalized score is 81.2, below ReDiffuser (82.6) and Decision Diffuser (81.8). Per task, RAD is the best method only on Hopper-Medium-Expert, and it is not within 5% of the best score on HalfCheetah-Medium, Hopper-Medium, Walker2d-Medium, or Walker2d-Medium-Replay. The abstract and contributions claim 'superior performance' or 'superiority'; the paper should either use 'competitive' consistently or identify a statistically meaningful sense in which RAD is superior.
  4. [§4.1, Eq. (8)] The anchor mechanism is underspecified. After setting st at position 0 and sg at position i in the noisy trajectory, the paper does not state how these anchors are enforced during the reverse denoising process. If the anchors are only in the initial noise, the final trajectory τ^0_t in Eq. (8) is not guaranteed to start at st or end at sg; if they are enforced by inpainting after each denoising step, that operation should be described. Without this detail, the planned trajectory and the executed first action are not uniquely determined by the equations.
minor comments (5)
  1. [§5.4, Table 2] The similarity threshold δ is reported as a key hyperparameter, but it is not defined in the method section; the notation should be introduced at Eq. (9) and used consistently throughout.
  2. [§5.3, Figures 3 and 4] The ablation results in Figure 3 are described only qualitatively, with no numeric values; they should be reported in a table or with explicit scores so the claimed contributions of each component can be quantified.
  3. [Abstract and §6] The abstract states that RAD 'retrieves high-return and reachable states,' but no reachability check appears in the algorithm; the limitations section in §6 acknowledges this, but the abstract should indicate these are intended targets rather than guaranteed reachable states.
  4. [Throughout] There are multiple typos and grammatical issues, such as 'targe states' in §5.3 and 'which act as the goal' in §4.1; the paper needs a careful proofreading pass.
  5. [Table 1] Baseline scores are reported without standard deviations, while RAD reports standard deviation over three seeds; without baseline errors, the statistical significance of the comparisons is unclear, and baseline errors should be included or a reference given.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: RAD is an empirical, externally benchmarked method; the only same-author citation is a non-load-bearing related-work reference, and the missing guidance function J_phi is a specification gap rather than a circular reduction.

full rationale

RAD's central claim is empirical: retrieve high-return states from the offline dataset and use condition-guided diffusion to plan toward them, with results evaluated on external D4RL MuJoCo benchmarks. The retrieval module (Sec. 4.2, Eqs. 9-11) labels candidate states by cosine similarity and dataset return, but that is a method-design choice, not a prediction identical to its fit; the reported performance comes from environment rollouts and is not forced by the retrieval equations. The only same-author citation is [25] (Bootstrapped Transformer, co-authored by Weinan Zhang), which appears in a related-work list of trajectory augmentation methods and is not used to justify RAD's mechanism or to exclude alternatives, so it is not load-bearing. Hyperparameter selection of k and delta (Table 2) on the same evaluation tasks is a validation/reporting weakness, not a circular reduction. The paper does leave Eq. (7) underspecified: it invokes a return-guidance function J_phi with no training objective in Sec. 4.4, and no optimizer update or loss for it is given. This is a reproducibility and correctness gap, but it is not an equation-level equivalence between the claim and its inputs. The stated limitation that retrieval may fail in noisy or stochastic settings is an environment assumption, not a circular step. No derivational chain in the paper reduces to its own inputs.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

RAD introduces no new physical entities. It does introduce a step estimation module and a pseudo target training strategy, but these are algorithmic components, not postulated entities. The main ledger items are hyperparameters tuned on the evaluation benchmark and two strong domain assumptions about dataset reachability and state-similarity semantics.

free parameters (6)
  • Similarity threshold delta = 0.9
    Tuned on D4RL evaluation tasks; Table 2 shows best performance at delta = 0.9.
  • Top-k retrieval count k = 6
    Tuned on D4RL evaluation tasks; Table 2 shows k = 6 gives the best results.
  • Return tolerance eta = not reported
    Threshold in Eq 10 that filters candidate targets by closeness to the best return; no numerical value is given.
  • Guidance scale rho = not reported
    Scaling factor for the return-guidance term in Eq 7; no numerical value is reported.
  • Planning horizon H = 32
    The generation range for MuJoCo tasks is set to 32 in implementation details.
  • Denoising steps N = 20
    The denoising step length is set to 20 in implementation details.
assumptions (4)
  • domain assumption The offline dataset contains reachable high-return states near any current state where planning is needed.
    The method retrieves targets from the dataset, and the Limitations section states: 'RAD also assumes that such target states exist in the dataset.'
  • ad hoc to paper Cosine similarity between states is a valid proxy for reachability and task progress.
    Eq 9 uses cosine similarity to select candidate target states; the paper does not validate this against a dynamics-based reachability measure.
  • domain assumption A diffusion model trained on trajectory segments can stitch a path from the current state to a retrieved target.
    This is the standard assumption of Diffuser-style generative planning, used in Eq 7 and Eq 8.
  • ad hoc to paper A well-trained return-guidance function J_phi is available during inference.
    J_phi appears in Eq 7 as a gradient guidance term, but Section 4.4 does not describe its architecture, training data, or loss.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RAD: Retrieval High-quality Demonstrations to Enhance Decision-making." pith.science (2026). https://pith.science/paper/GCQDM5WN

@misc{pith2026250715356,
  author       = {Pith},
  title        = {Pith review of: RAD: Retrieval High-quality Demonstrations to Enhance Decision-making},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GCQDM5WN}},
  note         = {Machine review of arXiv:2507.15356}
}
read the original abstract

Offline reinforcement learning (RL) learns policies from fixed datasets, thereby avoiding costly or unsafe environment interactions. However, its reliance on finite static datasets inherently restricts the ability to generalize beyond the training distribution. Prior solutions based on synthetic data augmentation often fail to generalize to unseen scenarios in the (augmented) dataset. To address these challenges, we propose Retrieval High-quAlity Demonstrations (RAD) for decision-making, which innovatively introduces a retrieval mechanism into offline RL. Specifically, RAD retrieves high-return and reachable states from the offline dataset as target states, and leverages a generative model to generate sub-trajectories conditioned on these targets for planning. Since the targets are high-return states, once the agent reaches such a target, it can continue to obtain high returns by following the associated high-return actions, thereby improving policy generalization. Extensive experiments confirm that RAD achieves competitive or superior performance compared to baselines across diverse benchmarks, validating its effectiveness. Our code is available at https://github.com/LeahGL/RAD.

Figures

Figures reproduced from arXiv: 2507.15356 by the authors.

Figure 1
Figure 1. Illustration of different strategies for trajectory generalization in offline RL. Suppose [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overall framework of RAD. where each reverse step is parameterized as a Gaussian: pθ(τ i−1 | τ i ) = N (µθ(τ i , i), Σ i ). (3) Training is performed by minimizing a denoising objective, where the model learns to predict the noise ϵ added to a clean trajectory sample τ 0 : Ldenoise(θ) = Ei,τ0,ϵ ∥ϵ − ϵθ(τ i , i)∥ 2 [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Results of ablation experiments on different variants. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Performance of our network architecture with and without retrieval. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 12 canonical work pages

  1. [1]

    Is conditional generative modeling all you need for decision-making?arXiv preprint arXiv:2211.15657, 2022

    Anurag Ajay, Yilun Du, Abhi Gupta, Joshua Tenenbaum, Tommi Jaakkola, and Pulkit Agrawal. Is conditional generative modeling all you need for decision-making?arXiv preprint arXiv:2211.15657, 2022

    arXiv 2022

  2. [2]

    Decision transformer: Reinforcement learning via sequence modeling.Advances in neural information processing systems, 34:15084–15097, 2021

    Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Misha Laskin, Pieter Abbeel, Aravind Srinivas, and Igor Mordatch. Decision transformer: Reinforcement learning via sequence modeling.Advances in neural information processing systems, 34:15084–15097, 2021

    2021

  3. [3]

    Bail: Best-action imitation learning for batch deep reinforcement learning.Advances in Neural Information Processing Systems, 33:18353–18363, 2020

    Xinyue Chen, Zijian Zhou, Zheng Wang, Che Wang, Yanqiu Wu, and Keith Ross. Bail: Best-action imitation learning for batch deep reinforcement learning.Advances in Neural Information Processing Systems, 33:18353–18363, 2020

    work page 2020

  4. [4]

    Semi-markov offline reinforcement learning for healthcare

    Mehdi Fatemi, Mary Wu, Jeremy Petch, Walter Nelson, Stuart J Connolly, Alexander Benz, Anthony Carnicelli, and Marzyeh Ghassemi. Semi-markov offline reinforcement learning for healthcare. In Conference on Health, Inference, and Learning, pages 119–137. PMLR, 2022

    2022

  5. [5]

    D4rl: Datasets for deep data-driven reinforcement learning.arXiv preprint arXiv:2004.07219, 2020

    Justin Fu, Aviral Kumar, Ofir Nachum, George Tucker, and Sergey Levine. D4rl: Datasets for deep data-driven reinforcement learning.arXiv preprint arXiv:2004.07219, 2020

    arXiv 2004

  6. [6]

    Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor

    Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference on machine learning, pages 1861–1870. Pmlr, 2018

    work page 2018

  7. [7]

    Rediffuser: Reliable decision-making using a diffuser with confidence estimation

    Nantian He, Shaohui Li, Zhi Li, Yu Liu, and You He. Rediffuser: Reliable decision-making using a diffuser with confidence estimation. InForty-first International Conference on Machine Learning, 2024

    work page 2024

  8. [8]

    Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020. 10

    work page 2020

Show all 30 references
  1. [9]

    Planning with diffusion for flexible behavior synthesis.arXiv preprint arXiv:2205.09991, 2022

    Michael Janner, Yilun Du, Joshua B Tenenbaum, and Sergey Levine. Planning with diffusion for flexible behavior synthesis.arXiv preprint arXiv:2205.09991, 2022

    2022 arXiv

  2. [10]

    Offline reinforcement learning as one big sequence modeling problem.Advances in neural information processing systems, 34:1273–1286, 2021

    Michael Janner, Qiyang Li, and Sergey Levine. Offline reinforcement learning as one big sequence modeling problem.Advances in neural information processing systems, 34:1273–1286, 2021

    2021

  3. [11]

    Mt-opt: Continuous multi-task robotic reinforcement learning at scale.arXiv preprint arXiv:2104.08212, 2021

    Dmitry Kalashnikov, Jacob Varley, Yevgen Chebotar, Benjamin Swanson, Rico Jonschkowski, Chelsea Finn, Sergey Levine, and Karol Hausman. Mt-opt: Continuous multi-task robotic reinforcement learning at scale.arXiv preprint arXiv:2104.08212, 2021

    2021 arXiv

  4. [12]

    Morel: Model-based offline reinforcement learning.Advances in neural information processing systems, 33:21810–21823, 2020

    Rahul Kidambi, Aravind Rajeswaran, Praneeth Netrapalli, and Thorsten Joachims. Morel: Model-based offline reinforcement learning.Advances in neural information processing systems, 33:21810–21823, 2020

    2020

  5. [13]

    Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980, 2014

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  6. [14]

    Offline reinforcement learning with implicit q-learning

    Ilya Kostrikov, Ashvin Nair, and Sergey Levine. Offline reinforcement learning with implicit q-learning. arXiv preprint arXiv:2110.06169, 2021

    2021 arXiv

  7. [15]

    Offline reinforcement learning: Tutorial, review, and perspectives on open problems.arXiv preprint arXiv:2005.01643, 2020

    Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. Offline reinforcement learning: Tutorial, review, and perspectives on open problems.arXiv preprint arXiv:2005.01643, 2020

    2005 arXiv

  8. [16]

    Ceil: Generalized contextual imitation learning.Advances in Neural Information Processing Systems, 36:75491–75516, 2023

    Jinxin Liu, Li He, Yachen Kang, Zifeng Zhuang, Donglin Wang, and Huazhe Xu. Ceil: Generalized contextual imitation learning.Advances in Neural Information Processing Systems, 36:75491–75516, 2023

    2023

  9. [17]

    Synthetic experience replay

    Cong Lu, Philip Ball, Yee Whye Teh, and Jack Parker-Holder. Synthetic experience replay. Advances in Neural Information Processing Systems, 36:46323–46344, 2023

    2023

  10. [18]

    Double check your state before trusting it: Confidence- aware bidirectional offline model-based imagination.Advances in Neural Information Processing Systems, 35:38218–38231, 2022

    Jiafei Lyu, Xiu Li, and Zongqing Lu. Double check your state before trusting it: Confidence- aware bidirectional offline model-based imagination.Advances in Neural Information Processing Systems, 35:38218–38231, 2022

    2022

  11. [19]

    Awac: Accelerating online reinforcement learning with offline datasets.arXiv preprint arXiv:2006.09359, 2020

    Ashvin Nair, Abhishek Gupta, Murtaza Dalal, and Sergey Levine. Awac: Accelerating online reinforcement learning with offline datasets.arXiv preprint arXiv:2006.09359, 2020

    2006 arXiv

  12. [20]

    A survey on offline reinforcement learning: Taxonomy, review, and open problems.IEEE Transactions on Neural Networks and Learning Systems, 2023

    Rafael Figueiredo Prudencio, Marcos ROA Maximo, and Esther Luna Colombini. A survey on offline reinforcement learning: Taxonomy, review, and open problems.IEEE Transactions on Neural Networks and Learning Systems, 2023

    2023

  13. [21]

    Offline reinforcement learning for autonomous driving with safety and exploration enhancement.arXiv preprint arXiv:2110.07067, 2021

    Tianyu Shi, Dong Chen, Kaian Chen, and Zhaojian Li. Offline reinforcement learning for autonomous driving with safety and exploration enhancement.arXiv preprint arXiv:2110.07067, 2021

    2021 arXiv

  14. [22]

    Deep unsuper- vised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. InInternational conference on machine learning, pages 2256–2265. pmlr, 2015

    2015

  15. [23]

    Efficient exploration in continuous-time model-based reinforcement learning

    Lenart Treven, Jonas Hübotter, Bhavya Sukhija, Florian Dorfler, and Andreas Krause. Efficient exploration in continuous-time model-based reinforcement learning. Advances in Neural Information Processing Systems, 36:42119–42147, 2023

    2023

  16. [24]

    Offline reinforcement learning with reverse model-based imagination.Advances in Neural Information Processing Systems, 34:29420–29432, 2021

    Jianhao Wang, Wenzhe Li, Haozhe Jiang, Guangxiang Zhu, Siyuan Li, and Chongjie Zhang. Offline reinforcement learning with reverse model-based imagination.Advances in Neural Information Processing Systems, 34:29420–29432, 2021. 11

    2021

  17. [25]

    Boot- strapped transformer for offline reinforcement learning

    Kerong Wang, Hanye Zhao, Xufang Luo, Kan Ren, Weinan Zhang, and Dongsheng Li. Boot- strapped transformer for offline reinforcement learning. Advances in Neural Information Processing Systems, 35:34748–34761, 2022

    2022

  18. [26]

    Critic regularized regression

    Ziyu Wang, Alexander Novikov, Konrad Zolna, Josh S Merel, Jost Tobias Springenberg, Scott E Reed, Bobak Shahriari, Noah Siegel, Caglar Gulcehre, Nicolas Heess, et al. Critic regularized regression. Advances in Neural Information Processing Systems, 33:7768–7778, 2020

    2020

  19. [27]

    Combo: Conservative offline model-based policy optimization

    Tianhe Yu, Aviral Kumar, Rafael Rafailov, Aravind Rajeswaran, Sergey Levine, and Chelsea Finn. Combo: Conservative offline model-based policy optimization. Advances in neural information processing systems, 34:28954–28967, 2021

    2021

  20. [28]

    Mopo: Model-based offline policy optimization.Advances in Neural Information Processing Systems, 33:14129–14142, 2020

    Tianhe Yu, Garrett Thomas, Lantao Yu, Stefano Ermon, James Y Zou, Sergey Levine, Chelsea Finn, and Tengyu Ma. Mopo: Model-based offline policy optimization.Advances in Neural Information Processing Systems, 33:14129–14142, 2020

    2020

  21. [29]

    Uncertainty-driven trajectory truncation for data augmentation in offline reinforcement learning

    Junjie Zhang, Jiafei Lyu, Xiaoteng Ma, Jiangpeng Yan, Jun Yang, Le Wan, and Xiu Li. Uncertainty-driven trajectory truncation for data augmentation in offline reinforcement learning. In ECAI 2023, pages 3018–3025. IOS Press, 2023

    2023

  22. [30]

    Decision stacks: Flexible reinforcement learning via modular generative models

    Siyan Zhao and Aditya Grover. Decision stacks: Flexible reinforcement learning via modular generative models. Advances in Neural Information Processing Systems, 36:80306–80323, 2023. 12

    2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.