Pith. sign in

REVIEW 2 major objections 4 minor 47 references

Efficient Controllable Diffusion via Optimal Classifier Guidance

T0 review · 2 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that SLCD, an iterative classifier-guidance algorithm built only on supervised learning, provably converges to the optimal KL-regularized reward-maximizing distribution for diffusion models.

desk verdict SLCD is a genuinely simple, empirically strong reduction of controllable diffusion to supervised learning, but the main theorem rests on an assumption that the paper's own histogram class appears to violate. read the letter →

arxiv 2505.21666 v1 pith:4FHLHG7I submitted 2025-05-27 cs.LG cs.AI

Owen Oertell , Shikun Sun , Yiding Chen , Jin Peng Zhou , Zhiyong Wang , Wen Sun This is my paper
classification cs.LGcs.AI
keywords controllablegenerationdiffusionmodelsclassifierguidanceKL-regularizedrewardmaximizationno-regretonlinelearningdataaggregationcovariateshiftdiscrete
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether reward-guided generation from a pretrained diffusion model can be solved without reinforcement learning, using only the classification oracle that powers standard classifier guidance. It proposes SLCD, which repeatedly uses a small classifier to guide sampling, collects intermediate states and final rewards, and retrains the classifier on the aggregated data. The paper proves that this loop is a no-regret online learning procedure, so the returned sampler's distribution approaches the optimal trade-off $p^\star(x) \propto q_0(x) \exp(\eta r(x))$ under KL divergence. Experiments on image and biological-sequence tasks show higher reward at nearly base-model inference speed, and the learned reward distribution can be reused across test-time values of $\eta$ without retraining.

What carries the argument

The load-bearing object is the distributional classifier identity combined with a rollin-rollout aggregation loop. The identity rewrites the guided posterior as an expectation over a one-dimensional reward distribution $R^{\mathrm{prior}}(r|x_t,t)$, so the algorithm learns this distribution (a histogram over reward bins) rather than learning a score directly. The roll-in step generates $x_t$ by guiding the prior with the latest score $f_n$; the roll-out step completes the trajectory with the prior and reads out the reward; the collected triples are aggregated to retrain the estimator via maximum likelihood. This closes the covariate shift between the classifier's training and deployment distributions, and the no-regret property of the online-learning reduction converts the learned distribution's error into guidance-score error through the Lipschitz condition.

What would settle it

Compute, on a held-out set of guided intermediate states, the true score $\nabla_{x_t} \ln \mathbb{E}_{r\sim R^{\mathrm{prior}}[\cdot|x_t,t]}[\exp(\eta r)]$ by Monte Carlo rollout from the prior and compare it to SLCD's estimated score at the same points; if the ratio of score error to value error grows without bound as $\eta$ increases, or if the empirical KL of the returned sampler stays far above the theorem's bound on a task where the true prior is available, Assumption 6 and the theorem's conclusion would be contradicted.

Watch

Extended reading notes

Core claim

For a pretrained diffusion score, define a binary label with $p(y=1|x)=\exp(\eta r(x))$; the paper shows the ideal guided classifier satisfies $p(y=1|x_t)=\mathbb{E}_{x_T\sim P^{\mathrm{prior}}_{t\to T}(\cdot|x_t)}[\exp(\eta r(x_T))]$, an expectation that depends only on the conditional reward distribution $R^{\mathrm{prior}}(\cdot|x_t,t)$. SLCD learns that one-dimensional reward distribution with a histogram classifier over reward bins, using maximum likelihood on data aggregated across iterations of guided rollout. Under realizability, a no-regret bound on the log loss, exponential convergence of the forward marginal to $\mathcal{N}(0,I)$, and a Lipschitz assumption linking classifier-value error to score error, Theorem 7 gives $\mathbb{E}[\mathrm{KL}(P^{f_{\hat n}}_{0\to T}(\cdot|\mathcal{N}(0,I))\,\|\,p^\star)] \le \epsilon_T + \tfrac12 T\|g\|_\infty^2 L^2 \gamma_N$. The proof bounds the KL between the guided and ideal reverse processes by the accumulated distribution-estimation error, so the sampler returned by SLCD is provably close to the optimal KL-regularized target.

Load-bearing premise

The load-bearing premise is Assumption 6: a pointwise Lipschitz bound saying that whenever two reward-distribution estimators agree on the expected value of $\exp(\eta r)$, their guidance scores agree up to a fixed constant factor.

Editorial extensions

If this is right

  • SLCD's returned sampler is within $\epsilon_T + \tfrac12 T\|g\|_\infty^2 L^2\gamma_N$ of $p^\star$ in expected KL divergence, where the first term decays like $e^{-T}$ for Ornstein-Uhlenbeck forward processes and the second decays like $1/\sqrt{N}$ for standard no-regret learners.
  • Because the learned reward distribution does not depend on $\eta$, the guidance strength can be changed at test time without retraining, giving a smooth reward-versus-fidelity trade-off with constant inference cost.
  • The method needs only a supervised classification oracle, so it applies to both continuous diffusion (images) and discrete diffusion (DNA and RNA sequences) without RL solvers or modifications to the noise scheduler.
  • Each aggregation iteration reduces the covariate shift between the offline training distribution and the online guided sampling distribution, which the experiments show as rising reward across iterations.
  • Inference time remains nearly identical to the base model because only a small classifier gradient is added at each denoising step.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The histogram classifier used in the experiments likely does not satisfy the pointwise Lipschitz condition in Assumption 6, and the appendix's sufficient conditions have constants growing like $\exp(4\eta R_{\max})$; for large $\eta$ the stated bound may be vacuous, so the practical test-time $\eta$ sweep is not covered by the theorem.
  • The same distributional-classifier reduction could be applied to flow matching or consistency models, where the forward/reverse conditional equivalence takes a different form and would need its own identity.
  • A testable extension is to check whether the convergence rate is improved by relevance weighting or variance-reduced reward estimators in the rollout, since the current theory only requires the prior rollout distribution.
  • If Assumption 6 fails but the value error is still small, the method may still work in practice; a weaker averaged Lipschitz condition evaluated under the guided distribution would make the theorem more realistic.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper proposes SLCD, an iterative algorithm for controllable diffusion that trains a lightweight classifier (a histogram over reward bins) to guide a pretrained diffusion model, with the goal of sampling from the KL-regularized reward-maximizing distribution p*(x) ∝ q0(x) exp(η r(x)). The algorithm alternates between generating data by rolling in with the current guidance and rolling out with the prior to collect rewards, then retraining the reward-distribution estimator on the aggregated data. The theoretical section claims, via a reduction to no-regret online learning, that the returned guidance yields a sampler whose KL divergence to p* is bounded by ε_T + (1/2)T||g||^2∞ L^2 γ_N under Assumptions 3-6. Empirically, SLCD is evaluated on image compression, image aesthetics, 5'UTR, and DNA enhancer tasks, reporting higher reward quantiles than Best-of-N, DPS, SMC, SVDD-MC, and SVDD-PM, with nearly the same inference time as the base model and a favorable reward-FID trade-off.

Significance. If the theoretical guarantee were valid, this would be an important contribution: it would show that KL-regularized controllable generation can be solved with supervised learning oracles (classification) rather than RL, with a DAgger-style no-regret analysis, and with test-time control over the KL penalty η. The empirical results are convincing and reproducible (code provided), and the method's inference cost is essentially that of the base model, which is a practical advantage over Monte-Carlo-based guidance. The paper also makes useful connections to distributional RL (Q#) and to classifier guidance. However, the main theorem as stated hinges on an assumption that is not satisfied by the implemented model class, and the proof contains an initial-distribution mismatch; these issues must be addressed before the theoretical claim can be accepted.

major comments (2)
  1. [Section 5, Assumption 6; Remark 1; Section E] Assumption 6 is not satisfied by the histogram reward-distribution class described in Remark 1 and Section E. For a histogram model \hat R(·|x,t) with bin probabilities p_i(x,t), the scalar v_R(x,t)=Σ_{i=1}^B p_i(x,t)e^{ηr_i} is determined by the B values p_i(x,t) at the point x, while the guidance score ∇_x ln v_R(x,t) depends on the spatial gradients ∇_x p_i(x,t). Two different histogram predictors can agree at a single point (same p_i(x_0,t_0), hence same v_R and v_{R'}) yet have different gradients, making the left side of Assumption 6 positive while the right side is zero. Therefore the pointwise Lipschitz condition cannot hold for the proposed class unless an unstated architectural restriction is imposed. Appendix B does not repair this: Lemmas 8-11 give averaged L2 bounds or gradient-estimator bounds under additional smoothness and distributional assumptions, not the pointwise inequality needed in the proof of Theorem 7, and the constants grow exponentially with ηR_max. Since the final KL bound in Theorem 7 scales as L^2 γ_N, an infinite or uncontrolled L voids the guarantee for the implemented algorithm.
  2. [Section 5, Theorem 7 proof (Appendix C)] In the proof of Theorem 7, the process p_t is defined as P^{f*}_{0→t}(·|q_T), where q_T is the marginal distribution of the forward process, yet Assumption 5 bounds KL(N(0,I)∥q_T(·|y=1)) and the theorem's conclusion concerns KL relative to p^*. The proof then applies Assumption 5 to bound KL(\hat p_0∥p_0) with p_0=q_T, which does not follow from Assumption 5. Moreover, even with that bound, the argument would only show that \hat p_T is close to the guided process initialized from the marginal q_T, not to the target p^*, which requires initialization from the conditional q_T(·|y=1). This definitional mismatch must be corrected (e.g., by defining p_t := P^{f*}_{0→t}(·|q_T(·|y=1))) and the proof adjusted accordingly; as written, Theorem 7 does not establish convergence to p^*.
minor comments (4)
  1. [Appendix B] The appendix heading 'We discuss Theorem 6' should read 'Assumption 6', since the object is an assumption, not a theorem.
  2. [Appendix D, Lemma 13] Lemma 13 states that both processes have the same deterministic initial condition a, but the proof of Theorem 7 applies it to processes with different initial distributions (e.g., N(0,I) and q_T). The lemma should be stated for general initial distributions, with the initial KL appearing explicitly in the bound.
  3. [Section E.2.2] The sentence 'Because the gradient is invariant to the reward scale, we set r_i to be a B equally spaced partition of [0,1]' is imprecise: the gradient of log Σ_i p_i e^{ηr_i} is invariant to an additive shift of all bin centers, not to scaling, and the relationship between the original negative rewards and the [0,1] partition should be clarified (the effective exponent changes to ηR_max after an affine rescaling).
  4. [References] The reference list contains minor typos, such as 'V olodymyr' in the Sahoo et al. entry and 'Christoph Schumman' in the LAION aesthetics entry.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: p* is defined by the fixed reward and base distribution, the learned classifier enters only through the conditional reward distribution, and the only same-group citation is a non-load-bearing motivation for an explicit assumption.

full rationale

SLCD's derivation is a conditional reduction, not a fitting of the target. The target p* is defined in Eq. (6) from the fixed q0 and reward r, with no learned parameter entering; the algorithm trains R-hat by MLE (Eq. 11) on rewards collected by rolling out with the prior, and the guidance f_n is then defined by Eq. (9). Theorem 7's proof converts the assumed no-regret guarantee (Assumption 4) into average KL between R-hat and R_prior under the roll-in distribution via Pinsker's inequality, then uses the pointwise regularity Assumption 6 and Lemma 13 to bound KL between the guided and optimal SDEs. The no-regret bound is an explicit assumption, not a fitted input disguised as a prediction. The only same-group citation that comes close to being load-bearing is Chen et al. (2025), invoked to motivate Assumption 5's eps_T term; since Assumption 5 is assumed directly and the exponential OU convergence is standard, the self-citation does not force the result. The serious caveat is Assumption 6, which is a strong pointwise Lipschitz condition and may fail for the histogram classifier used in experiments; Appendix B only supplies L2-type bounds with exp(4 eta R_max) constants. That is a correctness/vacuity risk, not circularity: the theorem is honestly conditional on the assumption rather than deriving its conclusion by assuming it. Score 2 reflects only the minor non-load-bearing self-citations; no circular step is present.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim rests on five explicit assumptions: realizability, no-regret, a pointwise score-error bound, forward-process convergence, and standard SDE theory. The first three are strong and unverified for the practical instantiation; the last two are standard.

free parameters (3)
  • eta (KL penalty) = test-time selected
    Controls the trade-off between reward and KL divergence; chosen at inference time without retraining, so it is a user-specified knob rather than a fitted constant.
  • reward histogram bins B = not reported
    The reward distribution is modeled as a histogram over B bins; B controls the expressivity of the classifier and is never specified in the paper.
  • training-time guidance scale = 150/75 (images), 10 (sequences)
    The scale used in the roll-in phase of data collection is chosen by hand and affects the training distribution; it is not derived from theory.
assumptions (5)
  • domain assumption Realizability: R_prior in R (Assumption 3)
    The true conditional reward distribution is assumed to belong to the histogram class; with a finite number of bins this is unrealistic for continuous rewards.
  • domain assumption No-regret learning (Assumption 4)
    The sequence of learned estimators is assumed to have average regret gamma_N = o(N)/N; this is assumed, not established for the neural classifier used in practice.
  • ad hoc to paper Score-estimation regularity (Assumption 6)
    A pointwise Lipschitz bound between the score of the log-partition and the value error; it is crucial for converting distribution error into guidance-score error and is not verified for the function class.
  • standard math Forward process convergence (Assumption 5)
    KL(N(0,I)||q_T(.|y=1)) <= epsilon_T with epsilon_T decaying exponentially for OU processes; standard diffusion theory.
  • standard math Anderson's reverse-time lemma and the SDE conditional-distribution equivalence
    Used to derive the closed-form classifier identity in Eq. (7); standard stochastic calculus result.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Efficient Controllable Diffusion via Optimal Classifier Guidance." pith.science (2026). https://pith.science/paper/4FHLHG7I

@misc{pith2026250521666,
  author       = {Pith},
  title        = {Pith review of: Efficient Controllable Diffusion via Optimal Classifier Guidance},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4FHLHG7I}},
  note         = {Machine review of arXiv:2505.21666}
}
read the original abstract

The controllable generation of diffusion models aims to steer the model to generate samples that optimize some given objective functions. It is desirable for a variety of applications including image generation, molecule generation, and DNA/sequence generation. Reinforcement Learning (RL) based fine-tuning of the base model is a popular approach but it can overfit the reward function while requiring significant resources. We frame controllable generation as a problem of finding a distribution that optimizes a KL-regularized objective function. We present SLCD -- Supervised Learning based Controllable Diffusion, which iteratively generates online data and trains a small classifier to guide the generation of the diffusion model. Similar to the standard classifier-guided diffusion, SLCD's key computation primitive is classification and does not involve any complex concepts from RL or control. Via a reduction to no-regret online learning analysis, we show that under KL divergence, the output from SLCD provably converges to the optimal solution of the KL-regularized objective. Further, we empirically demonstrate that SLCD can generate high quality samples with nearly the same inference time as the base model in both image generation with continuous diffusion and biological sequence generation with discrete diffusion. Our code is available at https://github.com/Owen-Oertell/slcd

Figures

Figures reproduced from arXiv: 2505.21666 by the authors.

Figure 1
Figure 1. An overview of the main experimental results. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Covariate shift (left) and data collection in our approach (right). The left figure illustrates [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Reward vs. number of iterations of [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Images generated by SLCD with varying η values and their rewards . The first column shows results from the base model, SD1.5, which corresponds to our method with η = 0. As η increases, the KL penalty is relaxed, allowing the generated images to be more strongly optimi…
Figure 5
Figure 5. Figure 5: Distribution of rewards for RNA sequences (5’UTR) across different methods. SLCD achieves higher median rewards and bet￾ter overall distribution compared to baseline ap￾proaches. 0 200 400 600 800 Time (s) 100 80 60 40 20 Reward ( ) 3 7 11 4 5 5 10 10 15 15 20 20 Rewar…
Figure 7
Figure 7. Figure 7: Additional images generated by SLCD with varying [PITH_FULL_IMAGE:figures/full_fig_p026_7.png]
Figure 8
Figure 8. Figure 8: More images generated by SLCD with varying [PITH_FULL_IMAGE:figures/full_fig_p027_8.png]
Figure 9
Figure 9. Figure 9: More images generated by SLCD with varying [PITH_FULL_IMAGE:figures/full_fig_p028_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

47 extracted references · 17 canonical work pages

  1. [1]

    Reinforcement learning: Theory and algorithms

    Alekh Agarwal, Nan Jiang, Sham M Kakade, and Wen Sun. Reinforcement learning: Theory and algorithms. CS Dept., UW Seattle, Seattle, WA, USA, Tech. Rep, pp.\ 10--4, 2019

    work page 2019

  2. [2]

    Reverse-time diffusion equation models

    Brian DO Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications, 12 0 (3): 0 313--326, 1982

    1982

  3. [3]

    Ledsam, Agnieszka Grabska-Barwinska, Kyle R

    Žiga Avsec, Vikram Agarwal, Daniel Visentin, Joseph R. Ledsam, Agnieszka Grabska-Barwinska, Kyle R. Taylor, Yannis Assael, John Jumper, Pushmeet Kohli, and David R. Kelley. Effective gene expression prediction from sequence by integrating long-range interactions. Nature Methods, 2021. doi:10.1038/s41592-021-01252-x

    doi:10.1038/s41592-021-01252-x 2021

  4. [4]

    Training diffusion models with reinforcement learning, 2024

    Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement learning, 2024

    2024

  5. [5]

    Prediction, learning, and games

    Nicolo Cesa-Bianchi and G \'a bor Lugosi. Prediction, learning, and games. Cambridge university press, 2006

    2006

  6. [6]

    Improved analysis of score-based generative modeling: User-friendly bounds under minimal smoothness assumptions

    Hongrui Chen, Holden Lee, and Jianfeng Lu. Improved analysis of score-based generative modeling: User-friendly bounds under minimal smoothness assumptions. In International Conference on Machine Learning, pp.\ 4735--4763. PMLR, 2023

    work page 2023

  7. [7]

    Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions

    Sitan Chen, Sinho Chewi, Jerry Li, Yuanzhi Li, Adil Salim, and Anru R Zhang. Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions. arXiv preprint arXiv:2209.11215, 2022

    arXiv 2022

  8. [8]

    Convergence of consistency model with multistep sampling under general data assumptions

    Yiding Chen, Yiyi Zhang, Owen Oertell, and Wen Sun. Convergence of consistency model with multistep sampling under general data assumptions. arXiv preprint arXiv:2505.03194, 2025

    arXiv 2025

Show all 47 references
  1. [9]

    Diffusion posterior sampling for general noisy inverse problems

    Hyungjin Chung, Jeongsol Kim, Michael Thompson McCann, Marc Louis Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. ICLR, 2023. URL https://dblp.org/rec/conf/iclr/ChungKMKY23

    2023

  2. [10]

    Kevin Clark, Paul Vicol, Kevin Swersky, and David J. Fleet. Directly fine-tuning diffusion models on differentiable rewards. ICLR, 2024. URL https://dblp.org/rec/conf/iclr/ClarkVSF24

    2024

  3. [11]

    Particle methods: An introduction with applications

    Piere Del Moral and Arnaud Doucet. Particle methods: An introduction with applications. In ESAIM: proceedings, volume 44, pp.\ 1--46. EDP Sciences, 2014

    2014

  4. [12]

    Rahul Dey and Fathi M. Salem. Gate-variants of gated recurrent unit (gru) neural networks. arXiv, 2017

    2017

  5. [13]

    Diffusion models beat gans on image synthesis, 2021

    Prafulla Dhariwal and Alex Nichol. Diffusion models beat gans on image synthesis, 2021. URL https://arxiv.org/abs/2105.05233

    2021 arXiv

  6. [14]

    Carles Domingo-Enrich, Michal Drozdzal, Brian Karrer, and Ricky T. Q. Chen. Adjoint matching: Fine-tuning flow and diffusion generative models with memoryless stochastic optimal control. ICLR, 2025. URL https://dblp.org/rec/conf/iclr/Domingo-EnrichD25

    2025

  7. [15]

    Dpok: Reinforcement learning for fine-tuning text-to-image diffusion models

    Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, Kangwook Lee, and Kimin Lee. Dpok: Reinforcement learning for fine-tuning text-to-image diffusion models. arXiv preprint arXiv:2305.16381, 2023

    2023 arXiv

  8. [16]

    Online convex optimization in the bandit setting: gradient descent without a gradient

    Abraham D Flaxman, Adam Tauman Kalai, and H Brendan McMahan. Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007, 2004

    2004 arXiv

  9. [17]

    The statistical complexity of interactive decision making

    Dylan J Foster, Sham M Kakade, Jian Qian, and Alexander Rakhlin. The statistical complexity of interactive decision making. arXiv preprint arXiv:2112.13487, 2021

    2021 arXiv

  10. [18]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017

    2017

  11. [19]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. CoRR, 2022. doi:10.48550/ARXIV.2207.12598

    2022 doi arXiv

  12. [20]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33: 0 6840--6851, 2020

    2020

  13. [21]

    Imagen video: High definition video generation with diffusion models

    Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.02303, 2022

    arXiv

  14. [22]

    Equivariant diffusion for molecule generation in 3d

    Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant diffusion for molecule generation in 3d. ICML, 2022. URL https://dblp.org/rec/conf/icml/HoogeboomSVW22

    2022

  15. [23]

    Derivative-free guidance in continuous and discrete diffusion models with soft value-based decoding, 2024

    Xiner Li, Yulai Zhao, Chenyu Wang, Gabriele Scalia, Gokcen Eraslan, Surag Nair, Tommaso Biancalani, Shuiwang Ji, Aviv Regev, Sergey Levine, and Masatoshi Uehara. Derivative-free guidance in continuous and discrete diffusion models with soft value-based decoding, 2024. URL http...

    2024 arXiv

  16. [24]

    Controlled decoding from language models

    Sidharth Mudgal, Jong Lee, Harish Ganapathy, YaGuang Li, Tao Wang, Yanping Huang, Zhifeng Chen, Heng-Tze Cheng, Michael Collins, Trevor Strohman, et al. Controlled decoding from language models. arXiv preprint arXiv:2310.17022, 2023

    arXiv

  17. [25]

    Unlocking guidance for discrete state-space diffusion and flow models

    Hunter Nisonoff, Junhao Xiong, Stephan Allenspach, and Jennifer Listgarten. Unlocking guidance for discrete state-space diffusion and flow models. ICLR, 2025. URL https://dblp.org/rec/conf/iclr/NisonoffXAL25

    2025

  18. [26]

    Chang, Yiyi Zhang, Kianté Brantley, and Wen Sun

    Owen Oertell, Jonathan D. Chang, Yiyi Zhang, Kianté Brantley, and Wen Sun. Rl for consistency models: Faster reward guided text-to-image generation, 2024. URL https://arxiv.org/abs/2404.03673

    2024 arXiv

  19. [27]

    Aligning text-to-image diffusion models with reward backpropagation, 2023

    Mihir Prabhudesai, Anirudh Goyal, Deepak Pathak, and Katerina Fragkiadaki. Aligning text-to-image diffusion models with reward backpropagation, 2023

    2023

  20. [28]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj\"orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 10684--10695, June 2022

    2022

  21. [29]

    Reinforcement and imitation learning via interactive no-regret learning

    Stephane Ross and J Andrew Bagnell. Reinforcement and imitation learning via interactive no-regret learning. arXiv preprint arXiv:1406.5979, 2014

    2014 arXiv

  22. [30]

    A reduction of imitation learning and structured prediction to no-regret online learning

    St \'e phane Ross, Geoffrey Gordon, and Drew Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pp.\ 627--635. JMLR Workshop and...

    2011

  23. [31]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Information...

    2022

  24. [32]

    Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin T

    Subham S. Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin T. Chiu, Alexander Rush, and Volodymyr Kuleshov. Simple and effective masked diffusion language models. NeurIPS, 2024. URL https://dblp.org/rec/conf/nips/SahooASGMCRK24

    2024

  25. [33]

    Sample, Ban Wang, David W

    Paul J. Sample, Ban Wang, David W. Reid, Vlad Presnyak, Iain J. McFadyen, David R. Morris, and Georg Seelig. Human 5' utr design and variant effect prediction from a massively parallel translation assay. Nature Biotechnology, 2019. doi:10.1038/s41587-019-0164-5

    2019 doi

  26. [34]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  27. [35]

    Laion aesthetics

    Chrisoph Schumman. Laion aesthetics. https://laion.ai/blog/ laion- aesthetics/, 2022

    2022

  28. [36]

    Online learning and online convex optimization

    Shai Shalev-Shwartz et al. Online learning and online convex optimization. Foundations and Trends in Machine Learning , 4 0 (2): 0 107--194, 2012

    2012

  29. [37]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2021 a . URL https://openreview.net/forum?id=St1giarCHLP

    2021

  30. [38]

    Score-based generative modeling through stochastic differential equations

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In Proceedings of International Conference on Learning Representations, 2021 b

    2021

  31. [39]

    Online non-convex learning: Following the perturbed leader is optimal

    Arun Sai Suggala and Praneeth Netrapalli. Online non-convex learning: Following the perturbed leader is optimal. In Algorithmic Learning Theory, pp.\ 845--861. PMLR, 2020

    2020

  32. [40]

    Deeply aggrevated: Differentiable imitation learning for sequential prediction

    Wen Sun, Arun Venkatraman, Geoffrey J Gordon, Byron Boots, and J Andrew Bagnell. Deeply aggrevated: Differentiable imitation learning for sequential prediction. In International conference on machine learning, pp.\ 3309--3318. PMLR, 2017

    2017

  33. [41]

    Diffusion probabilistic modeling of protein backbones in 3d for the motif-scaffolding problem

    Brian L Trippe, Jason Yim, Doug Tischer, David Baker, Tamara Broderick, Regina Barzilay, and Tommi Jaakkola. Diffusion probabilistic modeling of protein backbones in 3d for the motif-scaffolding problem. arXiv preprint arXiv:2206.04119, 2022

    2022 arXiv

  34. [42]

    Fine-tuning of continuous-time diffusion models as entropy-regularized control, 2024 a

    Masatoshi Uehara, Yulai Zhao, Kevin Black, Ehsan Hajiramezanali, Gabriele Scalia, Nathaniel Lee Diamant, Alex M Tseng, Tommaso Biancalani, and Sergey Levine. Fine-tuning of continuous-time diffusion models as entropy-regularized control, 2024 a

    2024

  35. [43]

    Feedback efficient online fine-tuning of diffusion models

    Masatoshi Uehara, Yulai Zhao, Kevin Black, Ehsan Hajiramezanali, Gabriele Scalia, Nathaniel Lee Diamant, Alex M Tseng, Sergey Levine, and Tommaso Biancalani. Feedback efficient online fine-tuning of diffusion models. arXiv preprint arXiv:2402.16359, 2024 b

    2024 arXiv

  36. [44]

    The central role of the loss function in reinforcement learning

    Kaiwen Wang, Nathan Kallus, and Wen Sun. The central role of the loss function in reinforcement learning. arXiv preprint arXiv:2409.12799, 2024

    2024 arXiv

  37. [45]

    Practical and asymptotically exact conditional sampling in diffusion models

    Luhuan Wu, Brian Trippe, Christian Naesseth, David Blei, and John P Cunningham. Practical and asymptotically exact conditional sampling in diffusion models. Advances in Neural Information Processing Systems, 36: 0 31372--31403, 2023

    2023

  38. [46]

    Q : Provably optimal distributional rl for llm post-training

    Jin Peng Zhou, Kaiwen Wang, Jonathan Chang, Zhaolin Gao, Nathan Kallus, Kilian Q Weinberger, Kiant \'e Brantley, and Wen Sun. Q : Provably optimal distributional rl for llm post-training. arXiv preprint arXiv:2502.20548, 2025

    2025

  39. [47]

    Maximum entropy inverse reinforcement learning

    Brian D Ziebart, Andrew L Maas, J Andrew Bagnell, Anind K Dey, et al. Maximum entropy inverse reinforcement learning. In Aaai, volume 8, pp.\ 1433--1438. Chicago, IL, USA, 2008

    2008

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.