Pith. sign in

REVIEW 4 major objections 6 minor 42 references

This paper claims that time-domain-only diffusion models for offline reinforcement learning introduce shifts in low-frequency trajectory components, and that a wavelet-Fourier diffuser that models low- and high-frequency parts separately el

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A wavelet-Fourier conditioning scheme for trajectory diffusion improves offline RL returns on most D4RL tasks by modeling low- and high-frequency components separately.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection The architecture is genuinely new and directionally effective, but the frequency-shift mechanism is not established by the evidence as presented. the 4 major comments →

arxiv 2509.19305 v2 pith:H3GUSHYP submitted 2025-09-04 cs.LG cs.AIeess.SP

Wavelet Fourier Diffuser: Frequency-Aware Diffusion Model for Reinforcement Learning

Yifu Luo , Yongzhe Chang , Xueqian Wang This is my paper
classification cs.LG cs.AIeess.SP
keywords offline reinforcement learningdiffusion modelswavelet transformFourier transformfrequency shifttrajectory generationD4RL benchmarkcross attention
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that diffusion-based offline reinforcement learning methods, which model trajectories purely in the time domain, inadvertently distort the low-frequency components of trajectories. These low-frequency parts carry the overall trend and stability, so the distortion causes jerky, unstable trajectories and lower returns. To fix this, the paper proposes WFDiffuser, which uses a discrete wavelet transform to split each trajectory into low- and high-frequency sub-trajectories, trains separate diffusion models on each, and lets them exchange information through a Short-Time Fourier Transform and cross-attention block. On the D4RL benchmark, WFDiffuser outperforms the standard Decision Diffuser baseline, with the largest gains on the long-horizon Kitchen tasks.

Core claim

The central claim is that frequency-domain analysis is a useful and missing axis for generative decision-making. The authors observe that the training loss of a time-domain-only trajectory diffuser is concentrated in low frequencies, and that a typical state trajectory also has most of its energy in low frequencies; they interpret the loss concentration as an introduced shift, not a faithful mirror of the data. WFDiffuser then targets this issue directly: a Haar-wavelet decomposition separates each trajectory into low-frequency (global trend) and high-frequency (local variation) components, separate conditional diffusion models generate each, and a Cross Fourier Fusion Conditioner uses STFT

What carries the argument

The load-bearing object is the Discrete Wavelet Transform with Haar filters, which splits a trajectory into τ_low and τ_high without losing information (biorthogonal property). On top of that, the Cross Fourier Fusion Conditioner (CFFC) computes amplitude and phase spectra via STFT, refines them with feed-forward networks, and then applies cross attention using high-frequency features as queries and low-frequency features as keys, producing conditions for both diffusion blocks. The separate low-frequency and high-frequency diffusion models are the generative engines, and the inverse wavelet transform is the recombination step.

Load-bearing premise

The paper's mechanism assumes that the low-frequency concentration in Decision Diffuser's training loss is an artifact introduced by time-domain-only modeling, rather than simply a reflection of the naturally low-frequency data spectrum.

What would settle it

Train WFDiffuser and Decision Diffuser on a version of a D4RL dataset whose trajectories have been pre-whitened so the energy spectrum is flat across frequencies. If the low-frequency loss concentration and the performance gap between the two methods both disappear, the shift is data-driven; if the gap persists on flat-spectrum data, the low-frequency shift is indeed a modeling artifact.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Trajectory-level diffusion models that ignore frequency structure are leaving a measurable source of instability on the table; frequency-aware modeling is a broadly applicable improvement.
  • The method's gains are largest on long-horizon, multi-stage tasks like Kitchen, suggesting frequency-aware decomposition especially helps when trajectories must stay coherent over many steps.
  • Since low-frequency components carry the global trend, generating them with dedicated modeling should make diffusion-based planners more reliable in real-world settings with non-smooth or noisy dynamics.
  • The cross-frequency conditioning mechanism ensures the high-frequency sub-trajectory is guided by low-frequency context, which the ablation suggests is the direction that matters most.
  • The wavelet decomposition and STFT conditioning introduce no reliance on action-space modeling, so the approach is compatible with any diffusion-based planner that predicts states and uses inverse dynamics.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's own diagnosis could be sharpened by comparing the loss spectrum against a no-shift reference; if the low-frequency concentration simply mirrors the data spectrum, the gains would then be attributable to added conditioning capacity rather than to shift mitigation.
  • A direct test of the mechanism would be to run WFDiffuser on a dataset whose frequency spectrum is deliberately flattened or inverted; if low-frequency modeling is the true driver, performance should degrade when the data no longer concentrates energy at low frequencies.
  • The same wavelet-Fourier decomposition could plausibly help other generative sequence models beyond RL, such as time-series forecasting or imitation learning, wherever physical smoothness makes low-frequency fidelity critical.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Wavelet Fourier Diffuser (WFDiffuser), an offline-RL trajectory-generation method that decomposes state trajectories with a Haar discrete wavelet transform into low- and high-frequency sub-trajectories, trains separate diffusion models on each component, and uses a Cross Fourier Fusion Conditioner (CFFC) that combines Short-Time Fourier Transform features with cross-attention to condition the two diffusion models. The authors report D4RL locomotion and kitchen results that are competitive with or better than Decision Diffuser (locomotion average 84.0 vs. 81.8; kitchen average 66.8 vs. 61.0), and they include ablations showing that the CFFC block and the Haar wavelet contribute positively. The paper frames the contribution as mitigating a 'frequency shift' that time-domain-only diffusion models allegedly introduce in low-frequency trajectory components.

Significance. If the frequency-shift diagnosis were established, the paper would offer a useful new axis for analyzing and improving diffusion-based decision-making models. The empirical work is grounded in external D4RL benchmarks, and the CFFC ablation (Fig. 7) provides some evidence that cross-frequency conditioning is beneficial. The main strengths are the breadth of D4RL evaluations and the ablation isolating CFFC. However, the central mechanistic claim is not yet supported: no no-shift reference spectrum is defined, and the diagnostic in Fig. 6 is self-referential. Moreover, baseline columns in Table I lack error bars, making several reported improvements hard to interpret. The contribution is promising, but the explanation needs to be made testable and the statistical evidence needs strengthening.

major comments (4)
  1. [Section I, Figs. 1–2 and abstract] The paper's central claim is that time-domain-only diffusion models 'inadvertently introduce shifts in the low-frequency components' of trajectories. The evidence is Fig. 1, showing Decision Diffuser's training loss concentrated at low frequencies, and Fig. 2, showing trajectory energy also concentrated at low frequencies. For a model trained to fit a signal, the squared-error spectrum is generally proportional to the signal power spectrum, so Fig. 1 could simply mirror the data's spectral envelope. Without a no-shift reference (e.g., loss normalized by per-frequency data power, or the loss spectrum of the same architecture trained on spectrally white targets), the observed concentration cannot distinguish an introduced defect from faithful modeling of the data. The 'frequency shift' phenomenon is therefore not yet falsifiably defined.
  2. [Section IV.B, Fig. 6 and Eq. (8)] Fig. 6 defines frequency shift through the ratio of losses in the first versus last 10 DWT modes of the training loss. This ratio is not compared against any no-shift reference, nor against the data's spectral envelope, and it is not shown to correlate with trajectory stability or returns. Since the same ratio is used both to define the problem and to validate the method, the argument is circular. Please specify the ratio's computation precisely, include a data-power reference, and report trajectory-level stability metrics (e.g., state-jump statistics or action smoothness) to support the claim of 'smoother, more stable trajectories.'
  3. [Table I] Baseline columns in Table I list point estimates only; only WFDiffuser has standard errors, and several differences are small (e.g., Walker2d Med-Expert: 108.6 vs. 108.8; HalfCheetah Med-Replay: 38.1 vs. 39.3). As reported, the reader cannot assess whether the headline average improvement of 2.2 points in locomotion or 5.8 points in kitchen is significant. Please provide standard errors for baselines from the original papers or rerun them, state the number of seeds for all entries, and report a pairwise significance test between WFDiffuser and Decision Diffuser.
  4. [Section IV.C, Fig. 7] The ablation removes the CFFC conditioning but does not control for the added parameters and computation introduced by the CFFC block. A stronger comparison would replace CFFC with a same-capacity conditioning module (e.g., a cross-attention module without the Fourier frequency features) to attribute the gain to frequency-domain fusion rather than to extra capacity. Additionally, Fig. 7 reports no numerical values or error bars, which makes the claimed 'outperforms all variants in all cases' hard to verify.
minor comments (6)
  1. [Algorithm 1] In lines 9–10, both low- and high-frequency samples are drawn from LFD; the second should likely be HFD.
  2. [Section III.B/IV.A and Algorithm 1] There are typographical inconsistencies: 'CSFC' appears instead of 'CFFC' in Section III.B, 'SFFC' appears in Section IV.A, 'descrete' appears in Algorithm 1, 'Finaly' in Section III.B, and 'Decison' in the conclusion.
  3. [Eq. (8)] The inverse DFT formula is mis-written: the notation 'F^{-1}(Flow(i))' is inconsistent with the index variables, and the constant 1/n is ambiguous. Please define n and all indices clearly.
  4. [Section III.A] The symbol τ is used both for the transition dynamics in the MDP tuple and for the trajectory sequence. This is confusing and should be disambiguated.
  5. [Section III.E] Low-temperature sampling is mentioned, but no temperature value or schedule is given, and the classifier-free guidance weight ω in Eq. (15) is not specified in the experimental setup.
  6. [Section III.C/IV.A] The number of wavelet decomposition levels is not stated; this affects the shapes of τ_low and τ_high and the design of the diffusion models. The STFT window length and hop size are also unspecified.

Circularity Check

0 steps flagged

No significant circularity: the main D4RL results are external benchmarks and no fitted input is renamed as a prediction; the frequency-shift narrative is an interpretation, not a constructional identity.

full rationale

The paper's central quantitative claim—improved offline RL performance—is evaluated on the external D4RL benchmark (Table I) and compared against independently published baselines. No parameter is fitted to the reported returns, and no equation in the paper reduces a prediction to an input by construction. The 'frequency shift' diagnosis in Section I and Figs. 1–2 is an interpretive claim about Decision Diffuser's loss spectrum and trajectory energy density; while it lacks a no-shift reference spectrum, that is a validity weakness, not circularity. The Fig. 6 loss-ratio metric is self-defined as an auxiliary diagnostic, but the method's success is not defined by that metric: the main results and ablations are based on task returns. There are no load-bearing self-citations (all references are external prior work), no imported uniqueness theorem, and no ansatz smuggled in via the authors' own prior publications. Accordingly, the derivation chain is self-contained with respect to its claimed empirical outcomes, and the appropriate circularity score is 0.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 1 invented entities

The paper's central contribution rests on five hand-chosen or unreported hyperparameters (guidance scale, temperature, diffusion steps, wavelet level, and the loss-spectrum mode threshold used for the Fig. 6 diagnostic), on standard math (Haar perfect reconstruction, Gaussian diffusion), and on two domain assumptions that are asserted rather than derived: that low-frequency components are the causally important part of trajectories for returns, and that the observed low-frequency loss concentration is an introduced defect. The 'frequency shift' phenomenon itself is a postulated entity evidenced only by the paper's own diagnostics.

free parameters (5)
  • Classifier-free guidance scale omega = not reported
    Eq. 15; hand-chosen hyperparameter controlling condition amplification. The value is not stated in the paper, so its influence on the reported scores cannot be assessed.
  • Low-temperature sampling temperature = not reported
    Section III.E says low-temperature sampling follows previous work, but the temperature/variance scaling value is not given. This affects the sharpness of the sampled distribution.
  • Number of diffusion timesteps T = not reported
    The diffusion horizon is not stated in Section III.E. It presumably follows Decision Diffuser, but is not specified.
  • Wavelet decomposition level = 1 level (implied: H=96 becomes H/2=48 per sub-trajectory)
    Section III.C defines tau_low and tau_high each of length H/2, implying a single-level Haar DWT. The choice of decomposition depth is hand-chosen and not ablated beyond level implied.
  • Low/high frequency mode threshold (10 modes) = 10 frequency modes
    Section IV.B and Fig. 6 define the first and last 10 frequency modes as low/high frequency for the shift diagnostic. This threshold is chosen by hand and drives the evidence for the central mechanism claim.
axioms (6)
  • standard math Haar DWT is biorthogonal with perfect reconstruction
    Section III.C: 'Due to the biorthogonal property of DWT, the sub-trajectories retain all information despite down-sampling.' Standard wavelet theory; correct for orthogonal Haar.
  • standard math Gaussian forward noising and score-based reverse denoising correctly model trajectory distributions
    Section III.E, Eq. 12; inherited from DDPM (Ho et al. 2020, cited as [28]) and prior diffusion-based planning work.
  • domain assumption Low-frequency components carry the trend/stability of trajectories and are essential for optimal trajectories
    Section I and Fig. 2: 'low-frequency components capture the overall trend and stability of trajectories, which are essential for smooth, regular, and optimal trajectories.' This motivates the design but is asserted, not derived.
  • ad hoc to paper Physical energy conservation justifies trajectory smoothness and low-frequency concentration
    Section I: 'This pattern reflects the inherent continuity and smoothness in natural physical phenomena and robotic motor motion, which stem from the physical principle of energy conservation.' A loose physical analogy; no formal argument connects energy conservation to the spectral loss of a diffusion model.
  • ad hoc to paper The low-frequency loss concentration in Decision Diffuser is an introduced 'shift' rather than a data property
    Section I, Fig. 1; the load-bearing diagnostic interpretation. No no-shift baseline is defined, so the 'shift' is asserted from a single observed spectrum.
  • domain assumption Separate diffusion models on tau_low and tau_high with CFFC conditioning suffice to model the joint trajectory distribution
    Sections III.B-III.E: the only coupling between the two component models is the CFFC cross-attention condition. The paper does not quantify the information loss from factorizing a trajectory into two independently denoised components before IDWT recombination.
invented entities (1)
  • Frequency shift phenomenon no independent evidence
    purpose: Postulated defect of time-domain-only trajectory diffusion models: inadvertent shifts in low-frequency components that cause trajectory instability and degraded performance; WFDiffuser is designed to mitigate it.
    Section I and Fig. 6. The only evidence is the paper's own loss-spectrum analysis (Figs. 1 and 6). There is no falsifiable handle outside this paper, and the causal link from loss spectrum to returns is not quantified.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Wavelet Fourier Diffuser: Frequency-Aware Diffusion Model for Reinforcement Learning." pith.science (2026). https://pith.science/paper/H3GUSHYP

@misc{pith2026250919305,
  author       = {Pith},
  title        = {Pith review of: Wavelet Fourier Diffuser: Frequency-Aware Diffusion Model for Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H3GUSHYP}},
  note         = {Machine review of arXiv:2509.19305}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Diffusion probability models have shown significant promise in offline reinforcement learning by directly modeling trajectory sequences. However, existing approaches primarily focus on time-domain features while overlooking frequency-domain features, leading to frequency shift and degraded performance according to our observation. In this paper, we investigate the RL problem from a new perspective of the frequency domain. We first observe that time-domain-only approaches inadvertently introduce shifts in the low-frequency components of the frequency domain, which results in trajectory instability and degraded performance. To address this issue, we propose Wavelet Fourier Diffuser (WFDiffuser), a novel diffusion-based RL framework that integrates Discrete Wavelet Transform to decompose trajectories into low- and high-frequency components. To further enhance diffusion modeling for each component, WFDiffuser employs Short-Time Fourier Transform and cross attention mechanisms to extract frequency-domain features and facilitate cross-frequency interaction. Extensive experiment results on the D4RL benchmark demonstrate that WFDiffuser effectively mitigates frequency shift, leading to smoother, more stable trajectories and improved decision-making performance over existing methods.

Figures

Figures reproduced from arXiv: 2509.19305 by Xueqian Wang, Yifu Luo, Yongzhe Chang.

Figure 1
Figure 1. Figure 1: Using the Discrete Fourier Transform (DFT), we convert the training [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Energy density (normalized to 0% ∼ 100%) in the frequency domain of a trajectory state sequence. It is observed that the energy density distribution is predominantly concentrated in the low-frequency components (center of X￾axis), which shows the importance of low-frequency components to a stable trajectory. The original time domain data are the joint angle trajectory of Hopper-expert-v2 in D4RL dataset [1… view at source ↗
Figure 3
Figure 3. Figure 3: The overall WFDiffuser framework. During training, we apply DWT to split trajectories into low- and a high-frequency sub-trajectories. Diffusion [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: The structure of Cross Fourier Fusion Conditioner (CFFC) block. [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: WFDiffuser performs better than existing diffuser-based (Decision [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: WFDiffuser successfully mitigates the frequency shift in the low [PITH_FULL_IMAGE:figures/full_fig_p006_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Ablation study on the Hopper. The performance score refers to the [PITH_FULL_IMAGE:figures/full_fig_p007_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

42 extracted references · 17 canonical work pages · 1 internal anchor

  1. [1]

    Offline reinforcement learning: Tutorial, review, and perspectives on open problems,

    S. Levine, A. Kumar, G. Tucker, and J. Fu, “Offline reinforcement learning: Tutorial, review, and perspectives on open problems,” arXiv preprint arXiv:2005.01643, 2020

    Pith/arXiv arXiv 2005

  2. [2]

    A survey on offline reinforcement learning: Taxonomy, review, and open problems,

    R. F. Prudencio, M. R. Maximo, and E. L. Colombini, “A survey on offline reinforcement learning: Taxonomy, review, and open problems,” IEEE Transactions on Neural Networks and Learning Systems , 2023

    work page 2023

  3. [3]

    Semi-markov offline reinforcement learning for healthcare,

    M. Fatemi, M. Wu, J. Petch, W. Nelson, S. J. Connolly, A. Benz, A. Carnicelli, and M. Ghassemi, “Semi-markov offline reinforcement learning for healthcare,” in Conference on Health, Inference, and Learn- ing. PMLR, 2022, pp. 119–137

    work page 2022

  4. [4]

    Human-centric dialog training via offline reinforcement learning,

    N. Jaques, J. H. Shen, A. Ghandeharioun, C. Ferguson, A. Lapedriza, N. Jones, S. S. Gu, and R. Picard, “Human-centric dialog training via offline reinforcement learning,” arXiv preprint arXiv:2010.05848, 2020

    Pith/arXiv arXiv 2010

  5. [5]

    Mastering atari, go, chess and shogi by planning with a learned model,

    J. Schrittwieser, I. Antonoglou, T. Hubert, K. Simonyan, L. Sifre, S. Schmitt, A. Guez, E. Lockhart, D. Hassabis, T. Graepel et al. , “Mastering atari, go, chess and shogi by planning with a learned model,” Nature, vol. 588, no. 7839, pp. 604–609, 2020

    work page 2020

  6. [6]

    Offline reinforcement learning for autonomous driving with safety and exploration enhancement,

    T. Shi, D. Chen, K. Chen, and Z. Li, “Offline reinforcement learning for autonomous driving with safety and exploration enhancement,” arXiv preprint arXiv:2110.07067, 2021

    Pith/arXiv arXiv 2021

  7. [7]

    Rt-2: Vision-language- action models transfer web knowledge to robotic control,

    A. Brohan, N. Brown, J. Carbajal, Y . Chebotar, X. Chen, K. Choroman- ski, T. Ding, D. Driess, A. Dubey, C. Finnet al., “Rt-2: Vision-language- action models transfer web knowledge to robotic control,” arXiv preprint arXiv:2307.15818, 2023

    Pith/arXiv arXiv 2023

  8. [8]

    Conservative q-learning for offline reinforcement learning,

    A. Kumar, A. Zhou, G. Tucker, and S. Levine, “Conservative q-learning for offline reinforcement learning,” Advances in Neural Information Processing Systems, vol. 33, pp. 1179–1191, 2020

    work page 2020

  9. [9]

    Behavior regularized offline reinforcement learning,

    Y . Wu, G. Tucker, and O. Nachum, “Behavior regularized offline reinforcement learning,” arXiv preprint arXiv:1911.11361 , 2019

    Pith/arXiv arXiv 1911

  10. [10]

    Offline reinforcement learning with implicit q-learning,

    I. Kostrikov, A. Nair, and S. Levine, “Offline reinforcement learning with implicit q-learning,” arXiv preprint arXiv:2110.06169 , 2021

    Pith/arXiv arXiv 2021

  11. [11]

    Offline re- inforcement learning with fisher divergence critic regularization,

    I. Kostrikov, R. Fergus, J. Tompson, and O. Nachum, “Offline re- inforcement learning with fisher divergence critic regularization,” in International Conference on Machine Learning . PMLR, 2021, pp. 5774–5783

    work page 2021

  12. [12]

    Offline rl policies should be trained to be adaptive,

    D. Ghosh, A. Ajay, P. Agrawal, and S. Levine, “Offline rl policies should be trained to be adaptive,” in International Conference on Machine Learning. PMLR, 2022, pp. 7513–7530

    work page 2022

  13. [13]

    Offline reinforcement learning with pseudometric learning,

    R. Dadashi, S. Rezaeifar, N. Vieillard, L. Hussenot, O. Pietquin, and M. Geist, “Offline reinforcement learning with pseudometric learning,” in International Conference on Machine Learning . PMLR, 2021, pp. 2307–2318

    work page 2021

  14. [14]

    Reinforcement learning: An introduction,

    R. S. Sutton, “Reinforcement learning: An introduction,” A Bradford Book, 2018

    work page 2018

  15. [15]

    Decision transformer: Reinforcement learning via sequence modeling,

    L. Chen, K. Lu, A. Rajeswaran, K. Lee, A. Grover, M. Laskin, P. Abbeel, A. Srinivas, and I. Mordatch, “Decision transformer: Reinforcement learning via sequence modeling,” Advances in neural information pro- cessing systems, vol. 34, pp. 15 084–15 097, 2021

    work page 2021

  16. [16]

    Offline reinforcement learning as one big sequence modeling problem,

    M. Janner, Q. Li, and S. Levine, “Offline reinforcement learning as one big sequence modeling problem,” Advances in neural information processing systems, vol. 34, pp. 1273–1286, 2021

    work page 2021

  17. [17]

    Planning with diffu- sion for flexible behavior synthesis,

    M. Janner, Y . Du, J. B. Tenenbaum, and S. Levine, “Planning with diffu- sion for flexible behavior synthesis,” arXiv preprint arXiv:2205.09991 , 2022

    Pith/arXiv arXiv 2022

  18. [18]

    Is conditional generative modeling all you need for decision-making?

    A. Ajay, Y . Du, A. Gupta, J. Tenenbaum, T. Jaakkola, and P. Agrawal, “Is conditional generative modeling all you need for decision-making?” arXiv preprint arXiv:2211.15657 , 2022

    Pith/arXiv arXiv 2022

  19. [19]

    D4rl: Datasets for deep data-driven reinforcement learning,

    J. Fu, A. Kumar, O. Nachum, G. Tucker, and S. Levine, “D4rl: Datasets for deep data-driven reinforcement learning,” arXiv preprint arXiv:2004.07219, 2020

    Pith/arXiv arXiv 2004

  20. [20]

    Hierarchical text-conditional image generation with clip latents,

    A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen, “Hierarchical text-conditional image generation with clip latents,” arXiv preprint arXiv:2204.06125, vol. 1, no. 2, p. 3, 2022

    Pith/arXiv arXiv 2022

  21. [21]

    High- resolution image synthesis with latent diffusion models,

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High- resolution image synthesis with latent diffusion models,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 10 684–10 695

    2022

  22. [22]

    Attention is all you need,

    A. Vaswani, “Attention is all you need,” Advances in Neural Information Processing Systems, 2017

    2017

  23. [23]

    Structured denoising diffusion models in discrete state-spaces,

    J. Austin, D. D. Johnson, J. Ho, D. Tarlow, and R. Van Den Berg, “Structured denoising diffusion models in discrete state-spaces,” Ad- vances in Neural Information Processing Systems , vol. 34, pp. 17 981– 17 993, 2021

    2021

  24. [24]

    An overview on principles for energy efficient robot locomotion,

    N. Kashiri, A. Abate, S. J. Abram, A. Albu-Schaffer, P. J. Clary, M. Daley, S. Faraji, R. Furnemont, M. Garabini, H. Geyer et al. , “An overview on principles for energy efficient robot locomotion,” Frontiers in Robotics and AI , vol. 5, p. 129, 2018

    work page 2018

  25. [25]

    A behavior regularized implicit policy for offline reinforcement learning,

    S. Yang, Z. Wang, H. Zheng, Y . Feng, and M. Zhou, “A behavior regularized implicit policy for offline reinforcement learning,” arXiv preprint arXiv:2202.09673, 2022

    Pith/arXiv arXiv 2022

  26. [26]

    The importance of pessimism in fixed-dataset policy optimization,

    J. Buckman, C. Gelada, and M. G. Bellemare, “The importance of pessimism in fixed-dataset policy optimization,” arXiv preprint arXiv:2009.06799, 2020

    Pith/arXiv arXiv 2009

  27. [27]

    Deep unsupervised learning using nonequilibrium thermodynamics,

    J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli, “Deep unsupervised learning using nonequilibrium thermodynamics,” in International conference on machine learning . PMLR, 2015, pp. 2256–2265

    2015

  28. [28]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in neural information processing systems , vol. 33, pp. 6840– 6851, 2020

    2020

  29. [29]

    Diffusion policies as an expres- sive policy class for offline reinforcement learning,

    Z. Wang, J. J. Hunt, and M. Zhou, “Diffusion policies as an expres- sive policy class for offline reinforcement learning,” arXiv preprint arXiv:2208.06193, 2022

    Pith/arXiv arXiv 2022

  30. [30]

    Energy-guided diffusion sampling for offline-to-online reinforcement learning,

    X.-H. Liu, T.-S. Liu, S. Jiang, R. Chen, Z. Zhang, X. Chen, and Y . Yu, “Energy-guided diffusion sampling for offline-to-online reinforcement learning,” arXiv preprint arXiv:2407.12448 , 2024

    Pith/arXiv arXiv 2024

  31. [31]

    Adapt- diffuser: Diffusion models as adaptive self-evolving planners,

    Z. Liang, Y . Mu, M. Ding, F. Ni, M. Tomizuka, and P. Luo, “Adapt- diffuser: Diffusion models as adaptive self-evolving planners,” arXiv preprint arXiv:2302.01877, 2023

    Pith/arXiv arXiv 2023

  32. [32]

    DIDI: Diffusion-Guided Diversity for Offline Behavioral Generation

    J. Liu, X. Guo, Z. Zhuang, and D. Wang, “Didi: Diffusion-guided diver- sity for offline behavioral generation,” arXiv preprint arXiv:2405.14790, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  33. [33]

    Glide: Towards photorealistic image gen- eration and editing with text-guided diffusion models,

    A. Nichol, P. Dhariwal, A. Ramesh, P. Shyam, P. Mishkin, B. McGrew, I. Sutskever, and M. Chen, “Glide: Towards photorealistic image gen- eration and editing with text-guided diffusion models,” arXiv preprint arXiv:2112.10741, 2021

    Pith/arXiv arXiv 2021

  34. [34]

    Photorealistic text-to-image diffusion models with deep language understanding,

    C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. L. Denton, K. Ghasemipour, R. Gontijo Lopes, B. Karagol Ayan, T. Salimans et al., “Photorealistic text-to-image diffusion models with deep language understanding,” Advances in neural information processing systems , vol. 35, pp. 36 479–36 494, 2022

    work page 2022

  35. [35]

    Implicit generation and modeling with energy based models,

    Y . Du and I. Mordatch, “Implicit generation and modeling with energy based models,” Advances in Neural Information Processing Systems , vol. 32, 2019

    work page 2019

  36. [36]

    Learning non-convergent non-persistent short-run mcmc toward energy-based model,

    E. Nijkamp, M. Hill, S.-C. Zhu, and Y . N. Wu, “Learning non-convergent non-persistent short-run mcmc toward energy-based model,” Advances in Neural Information Processing Systems , vol. 32, 2019

    work page 2019

  37. [37]

    Generative modeling by estimating gradients of the data distribution,

    Y . Song and S. Ermon, “Generative modeling by estimating gradients of the data distribution,” Advances in neural information processing systems, vol. 32, 2019

    2019

  38. [38]

    Estimation of non-normalized statistical models by score matching

    A. Hyv ¨arinen and P. Dayan, “Estimation of non-normalized statistical models by score matching.” Journal of Machine Learning Research , vol. 6, no. 4, 2005

    2005

  39. [39]

    Underactuated robotics: Learning, planning, and control for efficient and agile machines course notes for mit 6.832,

    R. Tedrake, “Underactuated robotics: Learning, planning, and control for efficient and agile machines course notes for mit 6.832,” Working draft edition, vol. 3, no. 4, p. 2, 2009

    work page 2009

  40. [40]

    Behavioral cloning from observa- tion,

    F. Torabi, G. Warnell, and P. Stone, “Behavioral cloning from observa- tion,” arXiv preprint arXiv:1805.01954 , 2018

    Pith/arXiv arXiv 2018

  41. [41]

    Morel: Model-based offline reinforcement learning,

    R. Kidambi, A. Rajeswaran, P. Netrapalli, and T. Joachims, “Morel: Model-based offline reinforcement learning,” Advances in neural infor- mation processing systems , vol. 33, pp. 21 810–21 823, 2020

    work page 2020

  42. [42]

    Adam: A method for stochastic optimization,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980 , 2014

    Pith/arXiv arXiv 2014

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.