Pith. sign in

REVIEW 2 major objections 9 minor 2 cited by

Offline Adaptation of Quadruped Locomotion using Diffusion Models

T0 review · 2 major / 9 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Classifier-free guidance lets a diffusion policy adapt a quadruped to new velocity commands after training, entirely on the onboard CPU.

desk verdict Useful empirical combo of CFG diffusion and quadruped hardware, but the Table I comparison is cherry-picked over lambda and the skill interpolation is only qualitative; worth reviewing nonetheless. read the letter →

arxiv 2411.08832 v3 pith:JWXVEWYK submitted 2024-11-13 cs.RO cs.AIcs.LG

Reece O'Mahoney , Alexander L. Mitchell , Wanming Yu , Ingmar Posner , Ioannis Havoutis This is my paper · ORCID
classification cs.ROcs.AIcs.LG
keywords diffusionmodelsquadrupedlocomotionclassifier-freeguidanceofflinereinforcementlearningskillinterpolationANYmalscore-basedgenerativeadaptation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that a single diffusion-based policy can do two things at once for quadruped locomotion: switch between distinct gaits (walking and crawling) with smooth interpolation, and adapt to new velocity-tracking rewards after training without retraining or access to the original velocity commands. The adaptation is achieved by treating a discounted return computed from the robot's velocity as a conditioning variable and applying classifier-free guidance at inference time to push sampled action trajectories toward maximum-return behaviour. On the ANYmal platform the guided policy reaches velocity-tracking rewards comparable to an expert model that was given the true commands during training, while running entirely on the onboard CPU with three denoising steps. The significance is that the same trained model can be reshaped at test time, removing the need for hierarchical skill policies or relabelled datasets.

What carries the argument

The central object is the score function of a diffusion model over action trajectories, trained with denoising score matching and cast in the k-diffusion SDE framework so that a probability-flow ODE enables fast sampling. The mechanism that carries the adaptation claim is classifier-free guidance: the conditioning variable y is defined as a discounted sum of future velocity-tracking rewards, exponentially rescaled and normalised to [0,1], and the model is trained with y randomly masked so that one network can provide both ∇x log p(x) and ∇x log p(x|y). At test time the user sets y = 1 and chooses λ above 1 to trade dynamics consistency for reward maximisation; the paper's ablations show λ ≈ 1.5–2 gives the best reward with fewest terminations.

What would settle it

Run the model over a skill switch while recording base height and foot-contact phase, and check whether the generated trajectories form a continuous bridge between walk and crawl; a bimodal distribution with an empty gap between the two modes would falsify the interpolation claim.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that classifier-free guidance turns an unlabelled offline dataset of quadruped trajectories into a policy that can be adapted after training to a new reward. The method labels the dataset with a velocity-tracking reward, scales the discounted returns to [0,1], randomly masks the return conditioning during training so the model also learns the unconditional distribution, and then at test time sets the return to 1 and uses λ in the guidance formula to steer generation. With the DDIM sampler needing only three denoising steps, the resulting policy tracks commanded velocities at levels statistically indistinguishable from an expert that was conditioned on the true commands, and does so on real hardware. The paper further claims that the same model interpolates between walking and crawling despite the two modes having been collected as separate datasets with no transitions, and that this interpolation survives when reward guidance is applied.

Load-bearing premise

The load-bearing premise is that the score function can bridge two disjoint modes in trajectory space: the training data contains separate walking and crawling trajectories with no transitions between them, yet the model is claimed to interpolate continuously and stably between the two skills on hardware.

Editorial extensions

If this is right

  • A single diffusion policy can handle multiple locomotion skills, so hierarchical skill-switching layers become unnecessary for gaits with similar state distributions.
  • New locomotion behaviours can be specified after training by designing a reward function and scaling its returns, without collecting new data or retraining.
  • Because the model runs in three denoising steps with deterministic sampling, deployment on quadruped hardware does not require a GPU.
  • Guidance strength is a test-time dial: changing λ from 1.5 to 5 trades reward for stability, giving the operator a live safety and performance knob.
  • Return-distribution scaling is a first-order design choice; a properly normalised, broad return range is necessary for guidance to work without falls.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not test whether the same test-time steering works when the target reward is far outside the training return distribution; its own conclusion says the model cannot extrapolate, so a natural next check is how gracefully performance degrades as the target approaches the distribution's edge.
  • If the skill-interpolation claim holds in general, the same mechanism could be used to blend more than two gaits or to morph between speeds continuously, potentially replacing gait schedulers in existing locomotion stacks.
  • The return-labelling scheme suggests a broader recipe: any offline locomotion dataset, including motion-capture clips without command labels, could be post-hoc steered toward user-defined objectives by computing a reward and applying the same guidance.
  • Since λ is chosen after training, one could vary λ online based on terrain or operator input, using the same weights for conservative and aggressive behaviour.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 9 minor

Summary. The paper proposes a diffusion-based policy for quadruped locomotion trained entirely offline on datasets collected by two RL policies, one for walking and one for crawling. The policy is a transformer denoiser trained with return conditioning and classifier-free guidance; at inference, the return is set to its maximum and CFG adapts the generated action trajectories to a velocity-tracking reward defined after data collection. The authors evaluate the method in simulation for four target velocities against an expert policy conditioned on ground-truth velocity commands (Table I), ablate guidance strength and return-temperature/scaling (Sections IV-D and IV-E), and demonstrate skill transitions and CPU-only inference with three denoising steps on an ANYmal. The central claim is that CFG yields velocity tracking comparable to a command-conditioned expert without ever observing velocity commands at training or inference time.

Significance. The combination of SDE-based diffusion, return-conditioned CFG, multi-skill interpolation, and CPU-only hardware deployment is a useful and timely systems contribution for legged locomotion. The paper is honest about its limitations (no out-of-distribution extrapolation, need for broad data) and releases code and videos. I also agree that the conditioning signal is a return computed from a hand-designed reward rather than a constant fit to the evaluation metric, so this is standard offline RL and not circular. If the comparison in Table I can be made without oracle selection of the guidance strength, the simulation evidence would support the main claim; the skill-interpolation claim, however, needs substantially more methodological and quantitative support.

major comments (2)
  1. [Section IV-B, Table I] The caption states that results are 'taken at the value of lambda that gave the best rewards.' Because lambda is a free method hyperparameter and the evaluation metric is exactly the reward used for selection, the reported parity with the expert model is an oracle-selected comparison. Please report the lambda value chosen for each of the four reward functions, choose lambda on a held-out set independent of the test evaluation, and include per-lambda curves for all tasks (Figure 4 provides this only for the forward-velocity reward). This is necessary to support the claim of 'comparable velocity tracking' in Section IV-B.
  2. [Section IV-C] The claim that the model 'was able to learn interpolations between these two skills' is not adequately supported. The datasets contain 'no transitions present,' and the manuscript never specifies how skill interpolation is implemented at inference: the conditioning includes a one-hot skill vector and no interpolation scheme for it is described. The only evidence is the qualitative snapshots in Figure 3. Please describe the inference-time mechanism and provide quantitative measures of intermediate behavior (e.g., commanded continuous sweeps of body height or velocity, with tracking errors) to distinguish true interpolation from discrete switching.
minor comments (9)
  1. [Global] The abstract uses 'adaption' while the title and other sections use 'adaptation'; please standardize the terminology.
  2. [Section IV-C] The phrase 'have have' appears in the sentence beginning 'Previous works have have looked at'; please remove the duplicate.
  3. [Section IV-E] In the first paragraph, 'the mode would struggle' should read 'the model would struggle.'
  4. [Figure 4] The termination axis is not defined; please state whether the plotted quantity is a count, a rate, or a normalized frequency.
  5. [Section III-C] The statement that the return is 'linearly scale[d] between [0,1] based on the batch statistics' is underspecified; state whether these statistics come from the full dataset or from each minibatch and how they are applied at inference.
  6. [Section IV-C] The hardware evaluation is only qualitative (snapshots in Figure 3); quantitative hardware tracking metrics would strengthen the deployment claim.
  7. [Table I] The DPM++ (2M) row is clearly outside the expert's error bars, so the 'comparable' conclusion should be explicitly scoped to the DDPM and DDIM samplers rather than stated in broad terms.
  8. [Section IV-D] The remark that the optimal lambda decreases from about 10 to 1.5-2 during training is not supported by any curve or protocol; please provide evidence or remove the observation.
  9. [Section II] The sentence 'While one other prior work has looked at performing Offline RL using CFG' does not explicitly name that work; please add a citation there.

Circularity Check

0 steps flagged · score 2.0 of 10

No derivation step reduces to its inputs by construction; the central offline-adaptation claim is standard return-conditioned guidance. Minor self-citations and an optimistic lambda selection are noted but do not constitute circularity.

full rationale

The paper's central derivation is self-contained in the relevant sense. Section III-B applies the standard classifier-free guidance identity (Eq. 6), which follows from Bayes rule and is not defined in terms of the paper's results. The offline-adaptation signal in Section III-C is a hand-designed velocity-tracking reward (Eq. 7), converted to a return and used as a conditioning variable; the model is never given the target command, so producing high-return trajectories requires generalization from the return scalar to actions. The fact that the evaluation metric (Table I, 'velocity tracking reward') coincides with the reward whose return is used for conditioning is the intended objective of offline RL, not a circular reduction: the model was not fit to the four specific v_target values or to the expert's performance. The skill-interpolation claim is empirical and is not asserted from any self-referential equation. Self-citations are present — [35] provides the RL policies used to collect the dataset, [18] is cited as related work, and [2,3] are earlier Oxford results — but none carries a load-bearing argument that reduces a claim to a citation: [35] is a data source, not a theorem invoked to force the outcome. The one legitimate caveat is in the Table I and Table II captions: 'Results are taken at the value of lambda that gave the best rewards.' Choosing lambda by the evaluation reward makes the reported numbers an oracle-selected maximum over guidance strengths, and the paper does not report fixed lambda values or held-out episodes. This inflates the headline comparison and should be weighed as a methodological limitation, but it is not a constructional circularity: lambda is an inference-time hyperparameter, not a fitted coefficient that encodes the evaluation answer in the model. No step in the derivation is equivalent to its input by definition, so the score is low.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

No new physical entities are introduced. The method rests on standard diffusion and CFG theory plus a hand-designed reward. Free parameters lambda and A are tuned on the evaluation reward, which weakens the strength of the reported results but does not sink the method.

free parameters (4)
  • Guidance strength lambda (Equation 6) = 1.5 to 2.0 per reward function (best on eval)
    Selected per reward function after evaluating test reward ('Results are taken at the value of lambda that gave the best rewards'), so the reported performance is a selected maximum.
  • Return temperature A (Equation 8) = 10
    Chosen empirically as producing the widest return distribution and best results (Section IV-E); other values A=1 and A=100 degrade performance.
  • Reward sharpness coefficient in Eq. 7 (3 in exponent) = 3
    Hand-chosen constant defining the velocity tracking reward; not ablated.
  • Number of denoising steps = 3
    Chosen for real-time inference; the paper states additional steps give marginal improvement but no supporting ablation is shown.
assumptions (5)
  • standard math Score-based diffusion models can generate samples from a data distribution via a probability flow ODE (Eq. 3), and the k-diffusion preconditioning (Eq. 4) stabilizes training.
    Imported from Karras et al. [25] and Song et al. [24]; accepted background for the method.
  • domain assumption Classifier-free guidance (Eq. 6) with lambda > 1 improves sample quality over the exact conditional score.
    Empirical claim from Ho and Salimans [14]; the paper relies on it to extract high-return trajectories.
  • ad hoc to paper The reward in Eq. 7, a discounted sum of velocity-tracking rewards over 50 steps, is a sufficient objective for recovering goal-conditioned locomotion behavior.
    The reward is hand-designed; no evidence that it is the right objective for all target behaviors, though the authors validate it on four velocity commands.
  • domain assumption The prior RL policies from [35] generate a dataset diverse enough (in velocities and physical parameters) for the diffusion model to learn a broad multi-modal distribution, and the data transfer to the real ANYmal robot.
    Dataset collection relies on the authors' prior RL framework [35]; the sim-to-real transfer is demonstrated but not analyzed.
  • ad hoc to paper The diffusion model can bridge disjoint skill modes (no transition examples in data) to produce stable interpolated gaits.
    Central to the skill-switching claim; only qualitative hardware evidence is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Offline Adaptation of Quadruped Locomotion using Diffusion Models." pith.science (2026). https://pith.science/paper/JWXVEWYK

@misc{pith2026241108832,
  author       = {Pith},
  title        = {Pith review of: Offline Adaptation of Quadruped Locomotion using Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JWXVEWYK}},
  note         = {Machine review of arXiv:2411.08832}
}
read the original abstract

We present a diffusion-based approach to quadrupedal locomotion that simultaneously addresses the limitations of learning and interpolating between multiple skills and of (modes) offline adapting to new locomotion behaviours after training. This is the first framework to apply classifier-free guided diffusion to quadruped locomotion and demonstrate its efficacy by extracting goal-conditioned behaviour from an originally unlabelled dataset. We show that these capabilities are compatible with a multi-skill policy and can be applied with little modification and minimal compute overhead, i.e., running entirely on the robots onboard CPU. We verify the validity of our approach with hardware experiments on the ANYmal quadruped platform.

Figures

Figures reproduced from arXiv: 2411.08832 by the authors.

Figure 1
Figure 1. Our diffusion-based approach possesses two capa [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Method Overview: a) A reinforcement learning agent is pre-trained with a hand crafted policy that generates reference [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Velocity tracking reward, height, and snapshots from a model trained with the forward velocity reward as it switches [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Velocity tracking reward and number of termina [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. GRoQ-LoCO: Generalist and Robot-agnostic Quadruped Locomotion Control using Offline Datasets

    cs.RO 2025-05 conditional novelty 6.0 of 10

    A single offline-trained attention model, using no robot-specific encoding, transfers walking and stair-climbing across multiple quadruped robots, including hardware runs.

  2. Discovery of skill switching criteria for learning agile quadruped locomotion

    cs.RO 2025-02 conditional novelty 6.0 of 10

    A hierarchical reinforcement learning framework lets a quadruped robot automatically switch between trotting, bounding, galloping, and fall recovery based on distance to the goal, with switch distances tuned by CMA-ES.

Reference graph

Works this paper leans on

35 extracted references · 21 canonical work pages · cited by 2 Pith papers

  1. [35]

    Learning low- frequency motion control for robust and dynamic robot locomotion,

    S. Gangapurwala, L. Campanaro, and I. Havoutis, “Learning low- frequency motion control for robust and dynamic robot locomotion,” in2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023, pp. 5085–5091

    work page 2023

  2. [1]

    Learning agile and dynamic motor skills for legged robots,

    J. Hwangbo, J. Lee, A. Dosovitskiy, D. Bellicoso, V . Tsounis, V . Koltun, and M. Hutter, “Learning agile and dynamic motor skills for legged robots,”Science Robotics, vol. 4, no. 26, 2019

    2019

  3. [2]

    Guided constrained policy optimization for dynamic quadrupedal robot locomotion,

    S. Gangapurwala, A. Mitchell, and I. Havoutis, “Guided constrained policy optimization for dynamic quadrupedal robot locomotion,”IEEE Robot. Automat. Lett. (RA-L), vol. 5, no. 2, pp. 3642–3649, 2020

    work page 2020

  4. [3]

    Rloc: Terrain-aware legged locomotion using reinforcement learning and optimal control,

    S. Gangapurwala, M. Geisert, R. Orsolino, M. Fallon, and I. Havoutis, “Rloc: Terrain-aware legged locomotion using reinforcement learning and optimal control,”IEEE Transactions on Robotics, vol. 38, no. 5, pp. 2908–2927, 2022

    2022

  5. [4]

    Learning to walk in minutes using massively parallel deep reinforcement learning,

    N. Rudin, D. Hoeller, P. Reist, and M. Hutter, “Learning to walk in minutes using massively parallel deep reinforcement learning,” 2022

    work page 2022

  6. [5]

    Anymal parkour: Learning agile navigation for quadrupedal robots,

    D. Hoeller, N. Rudin, D. Sako, and M. Hutter, “Anymal parkour: Learning agile navigation for quadrupedal robots,” 2023

    work page 2023

  7. [6]

    Barkour: Benchmarking animal-level agility with quadruped robots,

    K. Caluwaerts, A. Iscen, J. C. Kew, W. Yu, T. Zhang, D. Freeman, K.-H. Lee, L. Lee, S. Saliceti, V . Zhuang, N. Batchelor, S. Bohez, F. Casarini, J. E. Chen, O. Cortes, E. Coumans, A. Dostmohamed, G. Dulac-Arnold, A. Escontrela, E. Frey, R. Hafner, D. Jain, B. Jyenis, Y . Kuang, E. Lee, L. Luu, O. Nachum, K. Oslund, J. Powell, D. Reyes, F. Romano, F. Sade...

    work page 2023

  8. [7]

    Amp: Adversarial motion priors for stylized physics-based character con- trol,

    X. B. Peng, Z. Ma, P. Abbeel, S. Levine, and A. Kanazawa, “Amp: Adversarial motion priors for stylized physics-based character con- trol,”ACM Transactions on Graphics (ToG), vol. 40, no. 4, pp. 1–20, 2021

    2021

Show all 35 references
  1. [8]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,”Advances in neural information processing systems, vol. 33, pp. 6840–6851, 2020

    2020

  2. [9]

    Diffuseloco: Real-time legged locomotion control with diffusion from offline datasets,

    X. Huang, Y . Chi, R. Wang, Z. Li, X. B. Peng, S. Shao, B. Nikolic, and K. Sreenath, “Diffuseloco: Real-time legged locomotion control with diffusion from offline datasets,”arXiv preprint arXiv:2404.19264, 2024

    2024 arXiv

  3. [10]

    Generative adversarial imitation learning,

    J. Ho and S. Ermon, “Generative adversarial imitation learning,” CoRR, vol. abs/1606.03476, 2016. [Online]. Available: http://arxiv. org/abs/1606.03476

    2016 arXiv

  4. [11]

    Estimation of non-normalized statistical models by score matching,

    A. Hyvarinen, “Estimation of non-normalized statistical models by score matching,” 2005

    2005

  5. [12]

    Generative modeling by estimating gradients of the data distribution,

    Y . Song and S. Ermon, “Generative modeling by estimating gradients of the data distribution,” 2020

    2020

  6. [13]

    Planning with diffusion for flexible behavior synthesis,

    M. Janner, Y . Du, J. B. Tenenbaum, and S. Levine, “Planning with diffusion for flexible behavior synthesis,” 2022

    2022

  7. [14]

    Classifier-free diffusion guidance,

    J. Ho and T. Salimans, “Classifier-free diffusion guidance,” 2022

    2022

  8. [15]

    Is conditional generative modeling all you need for decision- making?

    A. Ajay, Y . Du, A. Gupta, J. Tenenbaum, T. Jaakkola, and P. Agrawal, “Is conditional generative modeling all you need for decision- making?” 2023

    2023

  9. [16]

    Offline reinforcement learning: Tutorial, review, and perspectives on open problems,

    S. Levine, A. Kumar, G. Tucker, and J. Fu, “Offline reinforcement learning: Tutorial, review, and perspectives on open problems,” 2020. [Online]. Available: https://arxiv.org/abs/2005.01643

    2020 arXiv

  10. [17]

    ANYmal - a highly mobile and dynamic quadrupedal robot,

    M. Hutter, C. Gehring, D. Jud, A. Lauber, C. D. Bellicoso, V . Tsounis, J. Hwangbo, K. Bodie, P. Fankhauser, M. Bloesch, R. Diethelm, S. Bachmann, A. Melzer, and M. Hoepflinger, “ANYmal - a highly mobile and dynamic quadrupedal robot,” inIEEE/RSJ Int. Conf. Intell. Rob. Sys. (...

    2016

  11. [18]

    Gaitor: Learning a unified representation across gaits for real-world quadruped locomotion,

    A. L. Mitchell, W. Merkt, A. Papatheodorou, I. Havoutis, and I. Posner, “Gaitor: Learning a unified representation across gaits for real-world quadruped locomotion,” 2024

    2024

  12. [19]

    Advanced skills through multiple adversarial motion priors in reinforcement learning,

    E. V ollenweider, M. Bjelonic, V . Klemm, N. Rudin, J. Lee, and M. Hutter, “Advanced skills through multiple adversarial motion priors in reinforcement learning,” 03 2022

    2022

  13. [20]

    A reduction of imitation learning and structured prediction to no-regret online learning,

    S. Ross, G. Gordon, and D. Bagnell, “A reduction of imitation learning and structured prediction to no-regret online learning,” in Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, ser. Proceedings of Machine Learning Research, G...

    2011

  14. [21]

    High-resolution image synthesis with latent diffusion models,

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” 2022

    2022

  15. [22]

    Diffusion policy: Visuomotor policy learning via action diffusion,

    C. Chi, Z. Xu, S. Feng, E. Cousineau, Y . Du, B. Burchfiel, R. Tedrake, and S. Song, “Diffusion policy: Visuomotor policy learning via action diffusion,” 2024

    2024

  16. [23]

    Goal-conditioned imi- tation learning using score-based diffusion policies,

    M. Reuss, M. Li, X. Jia, and R. Lioutikov, “Goal-conditioned imi- tation learning using score-based diffusion policies,”arXiv preprint arXiv:2304.02532, 2023

    2023 arXiv

  17. [24]

    Score-based generative modeling through stochastic differential equations,

    Y . Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-based generative modeling through stochastic differential equations,” in9th International Conference on Learning Representations, ICLR, 2021

    2021

  18. [25]

    Elucidating the design space of diffusion-based generative models,

    T. Karras, M. Aittala, T. Aila, and S. Laine, “Elucidating the design space of diffusion-based generative models,” 2022

    2022

  19. [26]

    Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models,

    C. Lu, Y . Zhou, F. Bao, J. Chen, C. Li, and J. Zhu, “Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models,” 2023

    2023

  20. [27]

    Conservative q-learning for offline reinforcement learning,

    A. Kumar, A. Zhou, G. Tucker, and S. Levine, “Conservative q-learning for offline reinforcement learning,” inAdvances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., vol. 33. Curran Associates, Inc., 2020, pp. 1179–...

    2020

  21. [28]

    A minimalist approach to offline reinforcement learning,

    S. Fujimoto and S. S. Gu, “A minimalist approach to offline reinforcement learning,” inAdvances in Neural Information Processing Systems, M. Ranzato, A. Beygelzimer, Y . Dauphin, P. Liang, and J. W. Vaughan, Eds., vol. 34. Curran Associates, Inc., 2021, pp. 20 132–20 145. [Onl...

    2021

  22. [29]

    Diffusion policies as an expressive policy class for offline reinforcement learning,

    Z. Wang, J. J. Hunt, and M. Zhou, “Diffusion policies as an expressive policy class for offline reinforcement learning,” inThe Eleventh International Conference on Learning Representations, 2023. [Online]. Available: https://openreview.net/forum?id=AHvFDPi-FA

    2023

  23. [30]

    Offline reinforcement learning with implicit q-learning,

    I. Kostrikov, A. Nair, and S. Levine, “Offline reinforcement learning with implicit q-learning,” inInternational Conference on Learning Representations, 2022. [Online]. Available: https: //openreview.net/forum?id=68n2s9ZJWF8

    2022

  24. [31]

    Idql: Implicit q-learning as an actor-critic method with diffusion policies,

    P. Hansen-Estruch, I. Kostrikov, M. Janner, K. Grudzien, and S. Levine, “Idql: Implicit q-learning as an actor-critic method with diffusion policies,” Master’s thesis, EECS Department, University of California, Berkeley, May 2023. [Online]. Available: http: //www2.eecs.berkele...

    2023

  25. [32]

    A real-world quadrupedal locomotion benchmark for offline reinforcement learning,

    H. Zhang, S. Yang, and D. Wang, “A real-world quadrupedal locomotion benchmark for offline reinforcement learning,” 2023. [Online]. Available: https://arxiv.org/abs/2309.16718

    2023 arXiv

  26. [33]

    A connection between score matching and denoising autoencoders,

    P. Vincent, “A connection between score matching and denoising autoencoders,” 2010

    2010

  27. [34]

    Sliced score matching: A scalable approach to density and score estimation,

    Y . Song, S. Garg, J. Shi, and S. Ermon, “Sliced score matching: A scalable approach to density and score estimation,” 2019

    2019

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.