REVIEW 8 cited by
Feedback Efficient Online Fine-Tuning of Diffusion Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Diffusion models excel at modeling complex data distributions, including those of images, proteins, and small molecules. However, in many cases, our goal is to model parts of the distribution that maximize certain properties: for example, we may want to generate images with high aesthetic quality, or molecules with high bioactivity. It is natural to frame this as a reinforcement learning (RL) problem, in which the objective is to fine-tune a diffusion model to maximize a reward function that corresponds to some property. Even with access to online queries of the ground-truth reward function, efficiently discovering high-reward samples can be challenging: they might have a low probability in the initial distribution, and there might be many infeasible samples that do not even have a well-defined reward (e.g., unnatural images or physically impossible molecules). In this work, we propose a novel reinforcement learning procedure that efficiently explores on the manifold of feasible samples. We present a theoretical analysis providing a regret guarantee, as well as empirical validation across three domains: images, biological sequences, and molecules.
Forward citations
Cited by 8 Pith papers
-
Bootstrap Flow-Map Tree Sampling Enables Online Feedback Driven Search
Bootstrap Flow-Map Trees construct complete DDPM-like trajectories with a single NFE and dynamic steps, enabling efficient online feedback-driven search and alignment that beats prior tree and SMC samplers.
-
Efficient Controllable Diffusion via Optimal Classifier Guidance
SLCD provably converges, under no-regret learning and a strong score-estimation assumption, to the KL-regularized optimal distribution using only supervised classification oracles.
-
Sample and Computationally Efficient Continuous-Time Reinforcement Learning with General Function Approximation
PURE achieves an Õ(√(d_R+d_F)/√N) suboptimality gap, up to horizon factors, in continuous-time RL with general function approximation, and adds low-switching and low-rollout variants.
-
One-for-All Adaptive Radiotherapy Planning Agent: A Foundation Framework for Daily CBCT-guided Radiotherapy
A unified agentic model performs online adaptive radiotherapy planning from daily CBCT in under two minutes with target dose errors generally within 2.0 Gy of clinical reference plans.
-
ChatVLA-2: Vision-Language-Action Model with Open-World Embodied Reasoning from Pretrained Knowledge
ChatVLA-2 uses dynamic mixture-of-experts and a two-stage training recipe to let a vision-language-action model retain pretrained reasoning while following robot instructions.
-
On the Design Space of Discrete Diffusion Online Adaptation for Molecular Optimization
Online fine-tuning of discrete diffusion models with complementary acquisition, CVaR shaping, density-entropy debiasing, replay, and validity control finds better molecules under fixed oracle budgets than offline fine...
-
MolFORM: Multi-modal Flow Matching for Structure-Based Drug Design
A flow-matching model with direct preference optimization fine-tuning generates protein-binding molecules faster than diffusion baselines, with improved docking scores on the CrossDocked2020 benchmark.
-
Direct Preference Optimization-Enhanced Multi-Guided Diffusion Model for Traffic Scenario Generation
MuDi-Pro fine-tunes a multi-guided diffusion transformer with DPO using guidance-score preferences to improve controllability of traffic scenario generation on nuScenes.
Discussion (0). Continue with ORCID to comment.