Pith. sign in

REVIEW 3 major objections 5 minor 4 cited by

This paper shows that training diffusion and flow policies against an unnormalized Boltzmann target in online RL can be recast as posterior-mean estimation, and that the two established estimator families are special cases of one control-va

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 10:52 UTC pith:CE4MWMQ4

load-bearing objection Useful unification and a real flow-policy extension; the variance-optimality claim is overstated and the experiments are too thin to support it. the 3 major comments →

arxiv 2601.08136 v2 pith:CE4MWMQ4 submitted 2026-01-13 cs.LG cs.SYeess.SY

Reverse Flow Matching: A Unified Framework for Online Reinforcement Learning with Diffusion and Flow Policies

Zeyang Li , Sunbochen Tang , Navid Azizan This is my paper
classification cs.LG cs.SYeess.SY
keywords reverse flow matchingonline reinforcement learningdiffusion policiesflow policiesBoltzmann distributionposterior mean estimationcontrol variatesLangevin Stein operator
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Online reinforcement learning with diffusion or flow policies faces a chicken-and-egg problem: the improved policy is a Boltzmann distribution exp(Q/λ) that one cannot sample from directly, yet conditional flow matching training requires samples from the target. The paper's thesis is that this obstacle can be reversed into a posterior-mean estimation problem: given an interpolated noisy action, the missing target sample is a latent variable whose posterior is determined by the known source distribution and the exponentiated Q-function. It then argues that the reverse flow matching loss shares its global minimizers with conditional flow matching, and that a control-variate family built from Langevin Stein operators interpolates between the two existing estimator families as the special cases η=0 and η=1. If the argument is right, it yields a single recipe for training both diffusion and flow policies online from unnormalized Q-functions, with variance-reduced importance sampling, and explains why the two prior families were both viable.

Core claim

Writing the flow interpolant as X_t = α_t X_1 + β_t X_0 with p_1 ∝ exp(Q/λ), the target p_1 is unnormalized and no direct X_1 samples exist. The paper's central discovery is that one can nevertheless regress the velocity field onto the same optimum as conditional flow matching: replacing unavailable forward samples with samples from the posterior q*_{0|t}(x0|xt) ∝ p0(x0) exp(Q((xt−β_t x0)/α_t)/λ) — or symmetrically the data posterior q*_{1|t} — yields a loss whose global minimizers coincide with those of conditional flow matching. The posterior means E[X0|xt] and E[X1|xt] then become the only quantities to estimate. For Boltzmann targets, a generalized Langevin Stein operator produces vector

What carries the argument

The reverse inferential posterior: treat X_t as evidence and X_0 as latent; Bayes' theorem gives q*_{0|t}(x0|xt) ∝ p0(x0) p1((xt − β_t x0)/α_t). The linear interpolation constraint then forces X_1 = (xt − β_t x0)/α_t, so posterior samples of X0 determine posterior samples of the target X1, converting the flow-matching target into a posterior mean. The generalized Langevin Stein operator (T_{p,m}Φ)(x) = ∇·Φ(x) + Φ(x)^T ∇log p(x) is the device that manufactures zero-mean vector control variates under this posterior; with the Boltzmann form of p1, the posterior score splits into the source score and the Q-gradient, yielding the η-family estimator.

Load-bearing premise

The central equivalence assumes the velocity function class can attain the global regression minimum and that the proposal over intermediate noisy samples has exactly the same support as the true marginal; on top of that, the variance-minimizing coefficient formulas require inserting the unknown posterior mean being estimated, so the practical estimator does not directly inherit the paper's minimum-variance guarantee.

What would settle it

In a low-dimensional Boltzmann target with exactly known normalizer, compute the SNIS posterior-mean estimator with Λ set by formula (20) using the ground-truth posterior mean, then repeat with Λ set using the estimator's own output; if the plug-in version's empirical variance fails to approach the ground-truth-Λ variance as N grows, the practical variance-reduction guarantee is false. Separately, train with a proposal that puts zero mass on part of the target's support and check whether the learned velocity still matches the conditional flow matching minimizer on that region.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Flow policies can be trained online to target the Boltzmann distribution π ∝ exp(Q/λ) without ever sampling an action from it; previously only diffusion policies had this capability.
  • The noise-expectation and gradient-expectation families are not competing objectives but endpoints of one control-variate family, so any η (or coordinate-wise Λ) is a legitimate training target with the same global optimum.
  • The framework supplies a variance objective for choosing Λ or η, allowing Q-value and Q-gradient information to be combined in a principled way rather than selected by convention.
  • Because the loss equivalence holds under any proposal distribution mutually absolutely continuous with the true marginal, the framework accommodates different schedules, prediction parameterizations (velocity, score, noise, data), and non-Gaussian source distributions.
  • On eight continuous-control benchmark tasks, the resulting flow-policy algorithm reportedly kept consistent performance across all tasks, whereas each baseline struggled severely on some tasks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because η interpolates between Q-value and Q-gradient information, a natural testable extension is to treat η as a tuned hyperparameter per environment rather than fixing it to 0 or 1; the paper proposes variance-minimizing coefficients but does not study finite-sample tuning behavior.
  • The variance-optimal coefficient formulas depend on the unknown posterior mean being estimated, so a small-scale experiment comparing ground-truth-Λ SNIS variance with plug-in-Λ variance would clarify whether the practical estimator inherits the claimed gain.
  • The same posterior-mean view should transfer to any setting where an unnormalized Boltzmann density is the target, such as energy-based models or equilibrium sampling, by using a flow model as a learned sampler; the control variate construction is not specific to RL.
  • The proposal distribution over intermediate samples is constrained only by absolute continuity and affects optimization dynamics, so deliberately choosing it to emphasize high-noise states or to follow an exploration schedule could be an additional design axis left unexplored in the paper.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper proposes reverse flow matching (RFM), a framework for training diffusion and flow policies in online RL when the target is an unnormalized Boltzmann distribution exp(Q/λ). The key idea is to replace the intractable conditional flow matching loss, which needs samples from p1, with a reverse objective (Eqs. (8)-(10)) that regresses the velocity field onto posterior means E[X0|Xt] or E[X1|Xt] under the coupling induced by linear interpolation. The authors prove equivalence with conditional flow matching under mutual absolute continuity (Theorem 1), introduce Langevin-Stein control variates to estimate these posterior means, and show that the noise-expectation (η=0) and gradient-expectation (η=1) families correspond to two instances of a one-parameter family (Eq. (25)). Experiments on DeepMind Control Suite are reported for a flow-policy instantiation.

Significance. The unifying derivation is genuinely useful: it connects two previously separate estimator families and extends Boltzmann-target training to flow policies. The posterior-coupling derivation, Proposition 1, and Theorem 1 are mathematically sound (assuming the stated regularity), and the paper is careful to distinguish global-minimizer equivalence from gradient dynamics. The control-variate construction via Langevin Stein operators is principled. However, the paper's headline claim of an 'optimal, minimum-variance estimator' is not supported at the level of the implemented algorithm: the variance-optimal coefficients in Propositions 4-5 require the unknown posterior mean and are then estimated from the same samples, voiding the asymptotic guarantee. The experimental section is too thin to substantiate the claimed stability improvements. With revisions to the variance-reduction claim and experiments, this could be a useful contribution.

major comments (3)
  1. [§4.4, Eqs. (18)-(21)] The 'optimal, minimum-variance' claim is not established for the estimator actually used. Lemma 3 gives the asymptotic covariance only for a fixed coefficient Λ. In §4.4, Λ is 'estimated from the same samples' via (20), which itself contains the unknown posterior mean μ_{0|t}(xt). The resulting two-step estimator has dependent weights and first-stage estimate, so the covariance formula (19) does not apply and there is no proof that the plug-in estimator attains or approaches Var(Λ*). Since the abstract and Section 1 advertise a 'principled combination of Q-value and Q-gradient information to derive an optimal, minimum-variance estimator,' this is a load-bearing gap. Please either provide a two-stage asymptotic analysis (e.g., showing the effect of μ̂ is higher order) or explicitly re-label Λ as a heuristic tuning parameter and provide empirical variance diagnostics.
  2. [§4.4, Eq. (20)] There is a mismatch between the per-(t, x_t) nature of the optimal coefficient formula and the implementation. Equation (20) minimizes the asymptotic variance for a fixed t and x_t, while §4.4 states Λ∈R^d is a constant vector and does not specify how the per-sample optimal Λ is aggregated across the batch or across t. If a single global Λ is used, the estimator is not minimizing the variance at any particular (t, x_t); if Λ is recomputed per sample, the dependence on the same samples breaks Lemma 3. Please clarify the implementation and align it with the theoretical claim.
  3. [§5, Figure 1] The empirical support for the central stability and efficiency claims is incomplete. Only training curves are shown, with no numerical table of final performance, standard errors, hyperparameters, or training budget; the set of 'eight environments' is not enumerated. More importantly, no ablation isolates the contribution of the control-variate coefficient Λ: a comparison of RFM with η=0 and η=1, and ideally a direct measurement of SNIS variance (or policy-update target variance), would be needed to support the claim that the proposed estimator improves stability. Please add these details or soften the corresponding claims.
minor comments (5)
  1. [§4.1, Eqs. (6)-(7)] The proportionality is stated without the Jacobian factors 1/α_t and 1/β_t. These factors are independent of x_0 (or x_1) for fixed t and can be absorbed into the normalizing constant, but stating this explicitly would help readers unfamiliar with δ-calculus.
  2. [§4.4 / Algorithm 1] The symbol β is used both for the learning rate and for the schedule coefficient β_t; the text acknowledges this abuse, but using a different symbol (e.g., γ) for one of them would improve readability.
  3. [§5, Figure 1] The figure lacks axis labels/units and the caption does not state whether the curves are episodic returns or average returns over evaluation episodes. Please specify.
  4. [Appendix B] The recovery of iDEM uses α_t=1, β_t=σ_min(σ_max/σ_min)^{1-t} with forward-time convention; for readers using the original time convention, a short glossary of the time reversal would help.
  5. [Eq. (25)] The identity is a direct consequence of the zero-mean property of the control variate, so 'the posterior mean simplifies to a linear combination' is somewhat misleading—it holds for any η, not an optimal choice. Please rephrase.

Circularity Check

1 steps flagged

Main RFM equivalence is independent; the 'optimal, minimum-variance' control-variate claim is self-referential because Λ* in (20) depends on the posterior mean it estimates.

specific steps
  1. self definitional [§4.2, Proposition 4, Eq. (20); §4.4, policy update paragraph]
    "Λ∗j = − E¯q[ w(X0, xt)^2 ( X0,j − μ0|t,j(xt) ) s∗0|t,j(X0, xt) ] / E¯q[ w(X0, xt)^2 ( s∗0|t,j(X0, xt) )^2 ] , (20) ... 'The vector Λ can be estimated from the same samples using Proposition 4 and (20).'"

    The variance-minimizing coefficient Λ* is an explicit function of the posterior mean μ0|t(xt), which is exactly the quantity the estimator (18) is built to estimate. In the implementation, the paper estimates Λ from the same SNIS samples used for the posterior-mean estimate. Lemma 3 and Proposition 4 establish the minimum-variance property only for a fixed coefficient; a data-dependent Λ̂ that is correlated with the self-normalized weights is not covered. Thus the advertised 'optimal, minimum-variance estimator' property is not derived for the implemented estimator: the oracle-minimizing coefficient requires knowledge of the target it is supposed to produce, making the variance-reduction claim circular in the practical algorithm.

full rationale

The central derivation is not circular. Proposition 1 and Theorem 1 show that L_RFM, L_RFM-N, and L_RFM-D reduce to regression onto the same posterior-mean velocity target, and that this target coincides with the conditional flow matching target up to the weighting measure; the argument is a direct quadratic-loss decomposition plus mutual absolute continuity, with no fitted parameter or self-cited uniqueness theorem. The recovery of QNE/RSM (η=0) and iDEM/DQS (η=1) as special cases of Eq. (25) is also not circular: those endpoints are derived from the same control-variate identity rather than fitted to reproduce the prior methods. The Langevin-Stein zero-mean property is proved in Lemma 2, so it does not rest on self-citation. The only substantive circularity is in the control-variate coefficient selection: Eq. (20) defines the 'optimal' Λ in terms of the posterior mean being estimated, and §4.4 then plugs an estimate from the same samples into that formula. Because the variance guarantee applies to a fixed coefficient and not to the implemented adaptive plug-in estimator, the paper's headline minimum-variance benefit is not established by the derivation. This is a partial circularity in a prominently advertised but non-central claim; the main RFM equivalence remains independent and non-circular.

Axiom & Free-Parameter Ledger

4 free parameters · 6 axioms · 0 invented entities

No new physical entities. The framework introduces a new objective (RFM) and control-variate family but no new latent variables beyond standard X0,X1; the main free parameters are algorithmic. The deepest caveat is that the optimal Λ formula is self-referential.

free parameters (4)
  • λ (soft-max temperature) = not reported
    Controls sharpness of Boltzmann target exp(Q/λ); set by hand in experiments; not learned or justified.
  • N (SNIS particles) = not reported
    Number of noise samples for posterior-mean estimation; directly trades bias/variance against compute; not reported.
  • t_min (time cutoff) = not reported
    Lower bound on t to avoid numerical instability near t=0; arbitrary.
  • Λ (control-variate coefficients) = estimated from samples
    Per-(t,x_t) coefficients from Eq. (20)/(21); formula depends on unknown posterior mean µ.
axioms (6)
  • standard math Bayes posterior q* (Eq. 6-7) under independent coupling and linear interpolation
    The reverse inference step rests on Bayes' theorem and the chosen interpolation constraint.
  • standard math Boundary/regularity conditions for Stein operator (Lemma 2)
    The zero-mean property of the control variates requires integrability and a vanishing boundary term at infinity.
  • domain assumption Mutual absolute continuity of p_t and proposal p_hat_t and rich function class (Theorem 1)
    Needed for the global-minimizer equivalence; may be only approximately true in RL with replay-buffer proposals and finite networks.
  • domain assumption Boltzmann form of improved policy from maximum-entropy RL (Section 3.1)
    The target distribution is taken to be π_new ∝ exp(Q/λ); this is standard soft policy improvement.
  • domain assumption Q is differentiable and ∇Q is available
    The gradient-expectation estimators and the control variates in Theorem 2 require ∇Q.
  • ad hoc to paper Diagonal constant test function class Φ=diag(Λ)
    The tractable formulas (20)-(21) and the recovery of prior methods are obtained by restricting to this subclass; a general optimal test function is left open.

pith-pipeline@v1.3.0-alltime-deepseek · 20653 in / 12346 out tokens · 119464 ms · 2026-08-03T10:52:35.902477+00:00 · methodology

0 comments
read the original abstract

Diffusion and flow policies are gaining prominence in online reinforcement learning (RL) due to their expressive power, yet training them efficiently remains a critical challenge. A fundamental difficulty that distinguishes online RL from standard generative modeling is the lack of direct samples from the target Boltzmann distribution defined by the Q-function. To address this, two seemingly distinct families of methods have been proposed for diffusion policies: a noise-expectation family, which uses a weighted average of noise as the training target, and a gradient-expectation family, which employs a weighted average of Q-function gradients. However, it remains unclear how these objectives are formally related, or whether they can be synthesized into a more general formulation. In this paper, we propose a unified framework, reverse flow matching (RFM), which rigorously addresses the problem of training diffusion and flow models without direct target samples. By adopting a reverse inferential perspective, we formulate the training target as a posterior mean estimation problem given an intermediate noisy sample. Crucially, we introduce Langevin Stein operators to construct zero-mean control variates, deriving a general class of estimators that share the same expectation. We show that existing noise-expectation and gradient-expectation methods are simply two specific instances within this broader class. This unified view yields two key advancements: it extends the capability of targeting Boltzmann distributions from diffusion to flow policies, and it enables the principled combination of Q-value and Q-gradient information to form an effective estimator, thereby improving training efficiency and stability. We instantiate RFM to train a flow policy in online RL and demonstrate improved performance on continuous-control benchmarks compared to diffusion policy baselines.

Figures

Figures reproduced from arXiv: 2601.08136 by Navid Azizan, Sunbochen Tang, Zeyang Li.

Figure 1
Figure 1. Figure 1: Training curves on eight environments. The solid lines correspond to the mean and the shaded [PITH_FULL_IMAGE:figures/full_fig_p015_1.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Start Right, Arrive Right: Asynchronous Execution via Initial Noise Selection

    cs.RO 2026-06 unverdicted novelty 7.0

    PAINT reframes asynchronous flow-based action chunking as an initial noise selection problem solved via backward Euler inversion and a repainting rule.

  2. GeMPO: Generalized Measure Matching for Online Diffusion Reinforcement Learning

    cs.LG 2026-03 conditional novelty 6.5

    GeMPO unifies diffusion RL reweighting as measure matching to a regularized target, enabling flexible and negative weights that improve exploration and performance.

  3. Mean Flow Policy Optimization

    cs.LG 2026-04 conditional novelty 6.0

    Mean Flow Policy Optimization (MFPO) uses few-step flow-based models for RL policies and achieves performance on par with or better than diffusion-based methods while substantially lowering training and inference time...

  4. Mean Flow Policy Optimization

    cs.LG 2026-04 conditional novelty 6.0

    MeanFlow policies optimized by soft policy iteration with an average-divergence network and adaptive SNIS velocity estimation match diffusion RL performance at far lower sampling cost.

Reference graph

Works this paper leans on

29 extracted references · 1 linked inside Pith · cited by 3 Pith papers

  1. [1]

    Deep unsupervised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. InInternational conference on machine learning, pages 2256–2265. pmlr, 2015

    2015

  2. [2]

    Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    2020

  3. [3]

    Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. InThe Eleventh International Conference on Learning Representations, 2023

    2023

  4. [4]

    High- resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    2022

  5. [5]

    Scaling rectified flow transformers for high- resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high- resolution image synthesis. InForty-first international conference on machine learning, 2024

    2024

  6. [6]

    Video diffusion models.Advances in neural information processing systems, 35:8633–8646, 2022

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models.Advances in neural information processing systems, 35:8633–8646, 2022

    2022

  7. [7]

    Pyramidal flow matching for efficient video generative modeling

    Yang Jin, Zhicheng Sun, Ningyuan Li, Kun Xu, Kun Xu, Hao Jiang, Nan Zhuang, Quzhe Huang, Yang Song, Yadong MU, and Zhouchen Lin. Pyramidal flow matching for efficient video generative modeling. InThe Thirteenth International Conference on Learning Representations, 2025

    2025

  8. [8]

    Diffusion policy: Visuomotor policy learning via action diffusion.The International Journal of Robotics Research, page 02783649241273668, 2023

    Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion.The International Journal of Robotics Research, page 02783649241273668, 2023

    2023

  9. [9]

    Fast and robust visuomotor riemannian flow matching policy.IEEE Transactions on robotics, 2025

    Haoran Ding, Noémie Jaquier, Jan Peters, and Leonel Rozo. Fast and robust visuomotor riemannian flow matching policy.IEEE Transactions on robotics, 2025

    2025

  10. [10]

    Diffusion policies as an expressive policy class for offline reinforcement learning

    Zhendong Wang, Jonathan J Hunt, and Mingyuan Zhou. Diffusion policies as an expressive policy class for offline reinforcement learning. InThe Eleventh International Conference on Learning Representa- tions, 2023. 19

    2023

  11. [11]

    Flow q-learning

    Seohong Park, Qiyang Li, and Sergey Levine. Flow q-learning. InForty-second International Conference on Machine Learning, 2025

    2025

  12. [12]

    Soft actor-critic: Off-policy max- imum entropy deep reinforcement learning with a stochastic actor

    Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy max- imum entropy deep reinforcement learning with a stochastic actor. InInternational Conference on Machine Learning, pages 1861–1870. PMLR, 2018

    2018

  13. [13]

    Efficient online reinforcement learning for diffusion policy

    Haitong Ma, Tianyi Chen, Kai Wang, Na Li, and Bo Dai. Efficient online reinforcement learning for diffusion policy. InForty-second International Conference on Machine Learning, 2025

    2025

  14. [14]

    Maximum entropy reinforcement learning with diffusion policy

    Xiaoyi Dong, Jian Cheng, and Xi Sheryl Zhang. Maximum entropy reinforcement learning with diffusion policy. InForty-second International Conference on Machine Learning, 2025

    2025

  15. [15]

    Iterated denois- ing energy matching for sampling from boltzmann densities

    Tara Akhound-Sadegh, Jarrid Rector-Brooks, Avishek Joey Bose, Sarthak Mittal, Pablo Lemos, Cheng- Hao Liu, Marcin Sendera, Siamak Ravanbakhsh, Gauthier Gidel, Yoshua Bengio, et al. Iterated denois- ing energy matching for sampling from boltzmann densities. InProceedings of the Forty-first Interna- tional Conference on Machine Learning, pages 760–786, 2024

    2024

  16. [16]

    Sampling from energy-based policies using diffusion

    Vineet Jain, Tara Akhound-Sadegh, and Siamak Ravanbakhsh. Sampling from energy-based policies using diffusion. InReinforcement Learning Conference, 2025

    2025

  17. [17]

    Diffusion actor-critic with entropy regulator.Advances in Neural Information Processing Systems, 37:54183–54204, 2024

    Yinuo Wang, Likun Wang, Yuxuan Jiang, Wenjun Zou, Tong Liu, Xujie Song, Wenxuan Wang, Liming Xiao, Jiang Wu, Jingliang Duan, et al. Diffusion actor-critic with entropy regulator.Advances in Neural Information Processing Systems, 37:54183–54204, 2024

    2024

  18. [18]

    Flow-based policy for online reinforcement learning

    Lei Lv, Yunfei Li, Yu Luo, Fuchun Sun, Tao Kong, Jiafeng Xu, and Xiao Ma. Flow-based policy for online reinforcement learning. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025

    2025

  19. [19]

    Learning a diffusion model policy from rewards via q-score matching

    Michael Psenka, Alejandro Escontrela, Pieter Abbeel, and Yi Ma. Learning a diffusion model policy from rewards via q-score matching. InThe Forty-first International Conference on Machine Learning, 2024

    2024

  20. [20]

    Langevin soft actor-critic: Effi- cient exploration through uncertainty-driven critic learning

    Haque Ishfaq, Guangyuan Wang, Sami Nur Islam, and Doina Precup. Langevin soft actor-critic: Effi- cient exploration through uncertainty-driven critic learning. InThe Thirteenth International Conference on Learning Representations, 2025

    2025

  21. [21]

    Diffusion-based reinforcement learning via q-weighted variational policy optimization.Advances in Neural Information Processing Systems, 37:53945–53968, 2024

    Shutong Ding, Ke Hu, Zhenhao Zhang, Kan Ren, Weinan Zhang, Jingyi Yu, Jingya Wang, and Ye Shi. Diffusion-based reinforcement learning via q-weighted variational policy optimization.Advances in Neural Information Processing Systems, 37:53945–53968, 2024

    2024

  22. [22]

    Online reward- weighted fine-tuning of flow matching with wasserstein regularization

    Jiajun Fan, Shuaike Shen, Chaoran Cheng, Yuxin Chen, Chumeng Liang, and Ge Liu. Online reward- weighted fine-tuning of flow matching with wasserstein regularization. InThe Thirteenth International Conference on Learning Representations, 2025

    2025

  23. [23]

    Control functionals for monte carlo integration

    Chris J Oates, Mark Girolami, and Nicolas Chopin. Control functionals for monte carlo integration. Journal of the Royal Statistical Society Series B: Statistical Methodology, 79(3):695–718, 2017

    2017

  24. [24]

    Measuringsamplequalitywithkernels

    JacksonGorhamandLesterMackey. Measuringsamplequalitywithkernels. InInternational Conference on Machine Learning, pages 1292–1301. PMLR, 2017

    2017

  25. [25]

    Importance sampling: a review.Wiley Interdisciplinary Reviews: Computational Statistics, 2(1):54–60, 2010

    Surya T Tokdar and Robert E Kass. Importance sampling: a review.Wiley Interdisciplinary Reviews: Computational Statistics, 2(1):54–60, 2010

    2010

  26. [26]

    Model-based diffusion for trajectory optimization

    Chaoyi Pan, Zeji Yi, Guanya Shi, and Guannan Qu. Model-based diffusion for trajectory optimization. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  27. [27]

    Reverse diffusion monte carlo

    Xunpeng Huang, Hanze Dong, Yifan Hao, Yian Ma, and Tong Zhang. Reverse diffusion monte carlo. InThe Twelfth International Conference on Learning Representations, 2024. 20

    2024

  28. [28]

    Stochastic localization via iterative posterior sampling

    Louis Grenioux, Maxence Noble, Marylou Gabrié, and Alain Oliviero Durmus. Stochastic localization via iterative posterior sampling. InForty-first International Conference on Machine Learning, 2024

    2024

  29. [29]

    Deepmind control suite.arXiv preprint arXiv:1801.00690, 2018

    Yuval Tassa, Yotam Doron, Alistair Muldal, Tom Erez, Yazhe Li, Diego de Las Casas, David Budden, Abbas Abdolmaleki, Josh Merel, Andrew Lefrancq, et al. Deepmind control suite.arXiv preprint arXiv:1801.00690, 2018. 21

    Pith/arXiv arXiv 2018