Pith. sign in

REVIEW 3 major objections 5 minor 54 references

Sample-Adaptive Latent Rewards for Uncertainty-Guided Diffusion Post-Training

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper shows that per-sample reward uncertainty, learned from ordinary pairwise preferences, lets diffusion post-training down-weight ambiguous feedback and beat strong baselines on image and video alignment.

desk verdict Useful idea but the sample-adaptive uncertainty claim is not supported by the preference loss, which only identifies the sum of the two candidate variances. read the letter →

arxiv 2608.06125 v1 pith:I5PHEQH6 submitted 2026-08-06 cs.CV

Rui Li , Yuanzhi Liang , Ke Hao , Ziqiao Weng , Haibin Huang , Chi Zhang , XueLong Li This is my paper · ORCID
classification cs.CV
keywords diffusionmodelslatentrewarduncertaintyestimationheteroscedasticpreferencelearninghackingpost-trainingalignmenttext-to-imagetext-to-video
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that when a latent reward model scores noisy diffusion latents, the reliability of that score depends not only on the noise level of the timestep but also on the specific content of the sample, and that this sample-level reliability can be learned from ordinary pairwise human preferences without extra confidence labels. The authors build a reward model (SURE-LRM) that predicts a Gaussian utility whose variance is the sum of a fixed noise prior and a learned, sample-specific residual, and then use the frozen model's variance to weight dense reward feedback along the denoising trajectory (SURE-REFL). If correct, diffusion post-training can down-weight ambiguous local feedback automatically, which the paper shows improves preference-prediction accuracy, raises image and video alignment scores, and reduces reward hacking compared with fixed-noise latent-reward baselines. The empirical payoff is concrete: the lowest-variance half of training pairs reaches 90.10% accuracy versus 79.40% overall, and SURE-REFL ranks first in eight of nine image backbone–metric comparisons and on VBench aggregates.

What carries the argument

The load-bearing object is the heteroscedastic latent reward model: a DiT-based encoder that maps a noisy latent, prompt, and timestep to a mean reward $\mu$ and an unconstrained residual-variance parameter $a$, converted via softplus to $v_{\eta,\mathrm{res}}$, which is added to the noise prior $v_0(\sigma)=2\sigma^2+\sigma_0^2$ to give total utility variance $v_\eta(\sigma)$. Preference learning follows the probit random-utility model with utility difference variance $S$ as above; backpropagation of the negative log-likelihood through $S$ gives the variance head a learning signal, while the log-variance regularizer $L_{\mathrm{var}}$ prevents variance inflation as an escape hatch. The companion mechanism is SURE-REFL, which converts the frozen model's variance into a detached reliability weight $\rho_{j,b}=V^{-1/2}_{j,b}$, normalizes it by the matched-transition mean, caps the upside with $\beta_{\mathrm{base}}+\lambda_p\min([\rho_{j,b}/\bar{\rho}_j-1]_+, B_p)$, and backpropagates each weighted mean reward only through its local denoising transition, so the uncertainty modulates how much each local sample contributes without ever entering the gradient path.

What would settle it

A direct falsifier: train SURE-LRM on one prompt distribution, then evaluate the predicted residual variance on held-out preference pairs from a different distribution (e.g., different style or subject), and measure whether error AUROC (correct vs. incorrect rankings) stays above chance and whether the low-variance half still concentrates correct predictions. If the variance–accuracy correlation collapses under this shift, the sample-adaptive claim is an in-distribution artifact rather than a learned reliability signal.

Watch

Extended reading notes

Core claim

The central claim is that candidate-specific reward uncertainty is identifiable from pairwise preference data, and that treating it as a first-class signal improves both reward modeling and generator post-training. Concretely, SURE-LRM augments a timestep-conditioned latent reward with a residual variance head so that the total variance of the utility difference is $S = 2v_0(\sigma) + v^+_{\eta,\mathrm{res}} + v^-_{\eta,\mathrm{res}}$, and training uses a probit likelihood with variance, margin, and anchor regularizers. The paper shows the learned residual variance correlates with ranking errors (24.26% higher on errors, AUROC 0.6941) and with human confidence. Then SURE-REFL freezes that reward model, evaluates selected denoising transitions, and weights each local transition's reward by the detached inverse standard deviation $\rho_{j,b}=V^{-1/2}_{j,b}$, normalized within each matched transition, with a bounded positive offset; each reward backpropagates only through its local transition. The paper reports that this uncertainty weighting outperforms the compared baselines on HPSv2.1, ImageReward, and PickScore across SD3.5-Medium, Z-Image-Turbo, and FLUX.1-dev, achieves the best VBench quality, semantic, and total scores for Wan2.1 video, and keeps the optimized reward aligned with an independent HPSv3 score far longer than the DiNa-LRM-REFL baseline.

Load-bearing premise

The paper's central claim requires that the learned residual variance reflects sample-level reliability rather than an artifact of the training objective; as the authors note, one preference pair fixes only the sum of the two candidates' variances, so any candidate-level signal must come from the shared variance head and could in principle encode dataset-level difficulty or timestep covariates instead.

Editorial extensions

If this is right

  • SURE-LRM improves preference prediction over matched DiNa-LRM backbones, with the average across backbones rising from 70.51 to 72.14, and generalizes to video.
  • SURE-REFL achieves leading image post-training, ranking first in eight of nine backbone–metric comparisons across SD3.5-Medium, Z-Image-Turbo, and FLUX.1-dev.
  • SURE-REFL achieves the highest VBench quality, semantic, and total scores for Wan2.1-480P video post-training, improving total from 0.8092 to 0.8357.
  • Uncertainty weighting improves optimization stability: the optimized reward and independent HPSv3 score stay aligned far longer than with DiNa-LRM-REFL, which exhibits reward hacking.
  • The learned residual variance is informative: incorrect rankings have higher variance, and the lowest-variance half of pairs reaches 90.10% accuracy versus 79.40% overall.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because one preference pair constrains only the variance sum, the method's success hinges on the variance head learning a dataset-wide mapping from latent content to reliability; an independent test would be to train SURE-LRM on one prompt distribution and evaluate the variance–accuracy AUROC on a different distribution.
  • Since the reliability weighting compares samples only within the same transition, the same rule could in principle be applied to any dense latent reward signal, such as step-level rewards from other diffusion-native reward models; this is an extension the paper does not test.
  • The variance head could serve as a free error detector at inference time, rejecting or reranking low-reliability generations without any human labels; this follows directly from the AUROC result but is not claimed by the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes SURE, a latent-space framework for uncertainty-aware reward modeling and diffusion model post-training. SURE-LRM augments a latent reward model with a heteroscedastic residual-variance head, so that each noisy latent receives a Gaussian utility with a mean reward and a variance that combines a timestep-dependent prior v0(σ) and a learned residual. SURE-REFL freezes this reward model and uses the detached inverse standard deviation to weight dense local reward feedback at a subset of denoising transitions, normalizing weights across samples at the same transition. The paper reports improved pairwise preference accuracy over DiNa-LRM across three image backbones, competitive video preference accuracy, and post-training gains on HPSv2.1, ImageReward, PickScore, and VBench.

Significance. The contribution is timely and, if the sample-adaptive variance is genuinely informative, practically valuable: it extends latent reward models with an explicit reliability signal and unifies image and video post-training under one weighting rule. The derivation in Eq. (9) is transparent, the authors correctly identify the variance-sum degeneracy in a single pair, and they are candid that Table 2 is an in-distribution diagnostic rather than held-out calibration. The main weakness is that the load-bearing claim — that the learned residual variance captures per-sample reliability beyond covariates and dataset statistics — is not yet backed by held-out or covariate-controlled evidence, and the main post-training tables do not report uncertainty intervals. Those gaps are fixable, and the conceptual framework is sound enough to warrant revision.

major comments (3)
  1. [3.2, Eqs. (6)-(10)] The preference likelihood depends on the two candidate residual variances only through their sum S, and Eq. (9) gives identical signed gradients to vi_η,res for i ∈ {+,−}; therefore the pairwise label alone cannot distinguish a reliable from an unreliable candidate. The only symmetry-breaking term, Lvar in Eq. (10), is label-independent and symmetric, so any candidate-level split must come from the shared variance head's inductive bias rather than from the preference signal. The paper acknowledges this in §3.2 ("A single pair constrains only the variance sum S..."), but the diagnostic offered in Table 2 is in-sample, does not stratify by timestep or prompt difficulty, and cannot rule out the alternative that residual variance encodes dataset-level or covariate-level difficulty. Because SURE-REFL's weighting in Eqs. (19)-(21) is exactly this variance, this identifiability gap is load-bearing. Please provide held-out calibration (e.g., AUROC on unseen pairs, stratified by timestep and prompt) and a same-prompt, same-timestep analysis showing that variance differentiates outcomes within a matched covariate cell.
  2. [4.3, Tables 4 and 5] The headline post-training results are point estimates from three random seeds, as stated in §4.1, but the tables report neither standard deviations nor significance tests. Several margins in Table 4 are small (e.g., FLUX.1-dev PickScore: SURE-REFL 22.49 vs ViPO 22.51), and the VBench total gain of 0.0109 over HY-PRFL in Table 5 is within the plausible seed-to-seed variation for this suite. Please report per-seed values, standard deviations, and paired tests (or bootstrap confidence intervals) for the main claim of improved post-training performance.
  3. [4.4, Table 5] The VBench aggregate claim is made on official aggregates in which several dimension scores are near saturation (e.g., Base already achieves 0.9647 Subject Consistency and 0.9731 Background Consistency), and some selected dimensions in the SURE-REFL row are not the best (Subject Consistency 0.9665 vs DanceGRPO 0.9769; Aesthetics 0.6388 vs ViPO 0.6527). The aggregate gain may be driven by a few dimensions (e.g., Multi. Object 0.9280 vs 0.8922 for Video-DPO). A fuller treatment would report dimension-wise confidence intervals and discuss whether the gains are concentrated rather than uniform.
minor comments (5)
  1. [Abstract and throughout] There are missing spaces and typographical issues throughout the text (e.g., "Thismakesalignment...", "SURE-LRMpredicts", "state of art", "various matrics"); a careful proofread is needed.
  2. [Table 1 and §4.1] The benchmark names are inconsistent: Table 1 uses HPD v2 and HPD v3 while §4.1 and elsewhere use HPSv2 and HPSv3; please adopt one consistent nomenclature.
  3. [Table 4] In the FLUX.1-dev block, the two LPO rows appear to be swapped or duplicated: the second LPO row lists ImageReward 0.8783, which is identical to the Z-Image-Turbo LPO row; please verify the entries.
  4. [Eq. (21)] The truncation bracket in the reliability weight formula is introduced without a definition in the main text; please define (x)_+ and the cap B_p explicitly where the equation appears.
  5. [4.1 and Table 3] The tie-excluded VisionRewardDB protocol and the held-out split of T2V Ranking are described only briefly; please specify the exact size, construction, and overlap of these evaluation sets in the main text or clearly reference the supplementary material.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the uncertainty signal is learned from preference labels, the in-distribution diagnostics are explicitly labeled as such, and SURE-REFL post-training gains are measured on external metrics.

full rationale

The paper's central derivation is self-contained. SURE-LRM's residual variance is trained with the heteroscedastic probit objective (Eqs. 6-8); the fact that a single pair's likelihood depends only on the variance sum S (Eq. 6) and sends identical gradients to both candidate variances (Eq. 9) is an identifiability property, not a circular reduction. The paper explicitly acknowledges this: "A single pair constrains only the variance sum S, so candidate-level variance is not independently identified by one label. Candidate-level structure can arise only from the state-conditioned variance head shared across many training comparisons." This is a stated modeling assumption, and relative differences can in principle be constrained by overlapping sum observations across pairs; no quantity is defined in terms of the target claim. Table 2's uncertainty diagnostics are explicitly labeled as an in-distribution diagnostic rather than held-out calibration: "Because these pairs appeared during training, we treat this evidence as an in-distribution diagnostic rather than held-out calibration." Thus the paper does not disguise fitted values as predictions. SURE-REFL post-training is assessed on external metrics (HPSv2.1, ImageReward, PickScore, VBench) with the reward model frozen, so the main empirical claims have independent support. The only overlapping-author citation (ViPO, Ni et al. 2026) is used as a baseline and in related work, not as load-bearing evidence for the paper's own claims. No circular step meeting the quoted-reduction standard is present.

Assumptions & free parameters 6 free parameters · 7 assumptions · 0 invented entities

The paper introduces no new physical object or external entity. Its new variables, the residual variance and the reliability weight, are neural-network outputs and derived scalars, not independently falsifiable entities. The cost of the method is carried by the listed hyperparameters and by the distributional assumptions inherited from preference learning and latent reward modeling.

free parameters (6)
  • sigma^2_0 (variance floor) = not reported
    Small variance floor in Eq. 2, chosen by hand; sets the minimum total variance and affects all reliability weights.
  • epsilon_v (numerical floor) = not reported
    Positive floor added to softplus residual variance in Eqs. 4 and 17; an ad hoc constant.
  • lambda_var, lambda_margin, lambda_anchor (loss weights) = not reported
    Weights in Eq. 13; chosen by hand and ablated in Table 6. They balance likelihood, variance control, margin, and anchor.
  • beta_base, lambda_p, B_p (reliability weight hyperparameters) = not reported
    In Eq. 21; control the minimum reward weight, strength of reliability modulation, and cap. Directly determine how uncertainty affects SURE-REFL.
  • eta_R (reward scale) = not reported
    Scale in Eq. 22 multiplying the uncertainty-weighted reward objective.
  • transition weights w_j = not reported
    Weights over queried denoising transitions in Eq. 22; chosen by hand and not given in the main text.
assumptions (7)
  • domain assumption Thurstone random utility model with Gaussian utility residuals (Eq. 7)
    Assumes pairwise preference probability is Phi(m / sqrt(S)); standard in preference learning but an unverified model of human choice.
  • domain assumption Noise-dependent variance prior v0(sigma) = 2 sigma^2 + sigma^2_0 (Eq. 2)
    Adopted from DiNa-LRM; assumes uncertainty grows linearly with noise variance and is shared by all samples at the same noise level.
  • domain assumption Conditional independence of the two utility residuals (Eq. 6)
    Needed to write the variance of the utility difference as the sum of the two variances.
  • ad hoc to paper Candidate-level variance can be identified from pairwise labels through a shared variance head
    The paper says a single pair constrains only the variance sum S; the claim that sample-specific variance is learnable relies on the shared head across many comparisons. This is the load-bearing identifiability assumption behind 'sample-adaptive' uncertainty.
  • domain assumption Latent states at intermediate noise levels are scoreable and correlate with human preference
    Core to latent reward methods; inherited from LRM and DiNa-LRM and not separately validated here for video.
  • domain assumption Backpropagating rewards only through the local transition approximates full-chain credit assignment
    SURE-REFL uses sg(z_j) and one-step solver advancement; this assumes the one-step gradient is a sufficient training signal.
  • domain assumption Automatic metrics (HPSv2.1, ImageReward, PickScore, VBench) proxy human preference
    All post-training conclusions rest on these metrics; no human study results are included in the main text.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Sample-Adaptive Latent Rewards for Uncertainty-Guided Diffusion Post-Training." pith.science (2026). https://pith.science/paper/I5PHEQH6

@misc{pith2026260806125,
  author       = {Pith},
  title        = {Pith review of: Sample-Adaptive Latent Rewards for Uncertainty-Guided Diffusion Post-Training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I5PHEQH6}},
  note         = {Machine review of arXiv:2608.06125}
}
read the original abstract

Latent reward models can supervise visual diffusion models without decoding intermediate states into pixel space. This makes alignment with human preferences more efficient. However, existing latent reward models output only scalar scores. They do not estimate the uncertainty of each prediction. The generator therefore cannot determine which feedback is reliable. This can drive optimization in the wrong direction and lead to reward hacking. We propose \textsc{SURE}, a unified latent-space framework for image and video diffusion models. It learns reward distributions and directly uses their reliability to guide dense post-training. First, we propose sample-adaptive latent reward model (\textsc{SURE-LRM}). It predicts a Gaussian utility for each noisy latent. Its mean predicts the reward score. Its variance reflect the uncertainty of prediction without human annotation. The learned distribution then guides post-training through uncertainty-guided reward feedback learning (\textsc{SURE-REFL}). This method provides uncertainty-guided dense feedback along the denoising trajectory. At selected transitions, \textsc{SURE-REFL} queries the frozen \textsc{SURE-LRM}. It converts detached variance into reliability weights for samples at the same transition. Each weighted reward is backpropagated only through its local transition. The entire process remains in latent space and requires neither pixel-space decoding nor the full denoising graph. Experiments show that \textsc{SURE-LRM} improves preference prediction over strong baselines. \textsc{SURE-REFL} achieves the sota performance among various metrics and further improves optimization stability. It also achieves the highest VBench quality, semantic, and total scores among the evaluated methods.

Figures

Figures reproduced from arXiv: 2608.06125 by the authors.

Figure 1
Figure 1. Comparison of reward feedback paradigms. The [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of SURE-LRM. The latent reward back￾bone predicts a mean reward and sample-adaptive residual uncertainty from the noisy latent, prompt, and timestep. The residual is combined with the noise-dependent prior to form the total predictive variance. random-utility model (Thurstone 1927), we write U i θ = µ i θ + η i θ , with η i θ ∼ N (0, vi η (σ)). Here, µ i θ is the mean reward and η i θ is an unobserved utili… view at source ↗
Figure 3
Figure 3. Overview of SURE-REFL. The frozen SURE￾LRM evaluates selected denoising transitions and con￾verts detached uncertainty into matched-transition reliability weights. Each reward is backpropagated only through its lo￾cal transition. learned queries aggregate multi-layer visual–text features; for videos, a query-attention pooler summarizes interme￾diate spatiotemporal features from the Wan DiT. A shared two-output proje… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison on video results of [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Optimization dynamics of DiNa-LRM-REFL (left) [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Qualitative comparison on image results of [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

54 extracted references · 24 canonical work pages

  1. [1]

    Advances in Neural Information Processing Systems , volume =

    Denoising Diffusion Probabilistic Models , author =. Advances in Neural Information Processing Systems , volume =. 2020 , eprint =

    work page 2020

  2. [2]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =

    High-Resolution Image Synthesis with Latent Diffusion Models , author =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =. 2022 , month = jun, doi =. 2112.10752 , archivePrefix =

    arXiv 2022

  3. [3]

    The Eleventh International Conference on Learning Representations , year =

    Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow , author =. The Eleventh International Conference on Learning Representations , year =. 2209.03003 , archivePrefix =

    arXiv

  4. [4]

    Psychological Review , volume =

    A Law of Comparative Judgment , author =. Psychological Review , volume =. 1927 , doi =

    work page 1927

  5. [5]

    Proceedings of the 22nd International Conference on Machine Learning , pages =

    Preference Learning with Gaussian Processes , author =. Proceedings of the 22nd International Conference on Machine Learning , pages =. 2005 , doi =

    work page 2005

  6. [6]

    2023 , doi =

    Xu, Jiazheng and Liu, Xiao and Wu, Yuchen and Tong, Yuxuan and Li, Qinkai and Ding, Ming and Tang, Jie and Dong, Yuxiao , booktitle =. 2023 , doi =. 2304.05977 , archivePrefix =

    arXiv 2023

  7. [7]

    2023 , doi =

    Kirstain, Yuval and Polyak, Adam and Singer, Uriel and Matiana, Shahbuland and Penna, Joe and Levy, Omer , booktitle =. 2023 , doi =. 2305.01569 , archivePrefix =

    arXiv 2023

  8. [8]

    2023 , eprint =

    Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis , author =. 2023 , eprint =. doi:10.48550/arXiv.2306.09341 , url =

    arXiv doi:10.48550/arxiv.2306.09341 2023

Show all 54 references
  1. [9]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =

    Learning Multi-Dimensional Human Preference for Text-to-Image Generation , author =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =. 2024 , month = jun, doi =. 2405.14705 , archivePrefix =

    2024 arXiv

  2. [10]

    2025 , eprint =

    Unified Reward Model for Multimodal Understanding and Generation , author =. 2025 , eprint =. doi:10.48550/arXiv.2503.05236 , url =

    2025 doi arXiv

  3. [11]

    Advances in Neural Information Processing Systems , volume =

    Unified Multimodal Chain-of-Thought Reward Model through Reinforcement Fine-Tuning , author =. Advances in Neural Information Processing Systems , volume =. 2025 , eprint =

    2025

  4. [12]

    2025 , month = oct, eprint =

    Ma, Yuhang and Wu, Xiaoshi and Sun, Keqiang and Li, Hongsheng , booktitle =. 2025 , month = oct, eprint =

    2025

  5. [13]

    Liu, Jie and Liu, Gongye and Liang, Jiajun and Li, Yangguang and Liu, Jiaheng and Wang, Xintao and Wan, Pengfei and Zhang, Di and Ouyang, Wanli , booktitle =. Flow-. 2025 , eprint =

    2025

  6. [14]

    2023 , eprint =

    Aligning Text-to-Image Diffusion Models with Reward Backpropagation , author =. 2023 , eprint =. doi:10.48550/arXiv.2310.03739 , url =

    2023 doi arXiv

  7. [15]

    The Twelfth International Conference on Learning Representations , year =

    Directly Fine-Tuning Diffusion Models on Differentiable Rewards , author =. The Twelfth International Conference on Learning Representations , year =. 2309.17400 , archivePrefix =

    arXiv

  8. [16]

    The Twelfth International Conference on Learning Representations , year =

    Training Diffusion Models with Reinforcement Learning , author =. The Twelfth International Conference on Learning Representations , year =. 2305.13301 , archivePrefix =

    arXiv

  9. [17]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =

    Diffusion Model Alignment Using Direct Preference Optimization , author =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =. 2024 , month = jun, doi =. 2311.12908 , archivePrefix =

    2024 arXiv

  10. [18]

    doi:10.48550/arXiv.2511.22699 , url =

    2025 , eprint =. doi:10.48550/arXiv.2511.22699 , url =

    2025 doi arXiv

  11. [19]

    Liu, Gongye and Yang, Bo and Zhi, Yida and Zhong, Zhizhou and Ke, Lei and Deng, Didan and Gao, Han and Huang, Yongxiang and Zhang, Kaihao and Fu, Hongbo and Luo, Wenhan , booktitle =. Beyond. 2026 , eprint =

    2026

  12. [20]

    Advances in Neural Information Processing Systems , volume =

    Diffusion Model as a Noise-Aware Latent Reward Model for Step-Level Preference Optimization , author =. Advances in Neural Information Processing Systems , volume =. 2025 , eprint =

    2025

  13. [21]

    2026 , eprint =

    Zhang, Jaxon and Yang, Binxin and Yin, Hubery and Li, Chen and Lyu, Jing , booktitle =. 2026 , eprint =

    2026

  14. [22]

    2026 , eprint =

    Reward Models Are Secretly Value Functions: Temporally Coherent Reward Modeling , author =. 2026 , eprint =. doi:10.48550/arXiv.2604.22981 , url =

    2026 doi arXiv

  15. [23]

    doi:10.48550/arXiv.2509.15110 , url =

    Zhang, Dan and Cai, Min and Light, Jonathan and Hu, Ziniu and Yue, Yisong and Tang, Jie , year =. doi:10.48550/arXiv.2509.15110 , url =. 2509.15110 , archivePrefix =

    doi

  16. [24]

    ICLR 2025 Workshop on Deep Generative Model in Machine Learning: Theory, Principle and Efficacy , year =

    Stable Consistency Tuning: Understanding and Improving Consistency Models , author =. ICLR 2025 Workshop on Deep Generative Model in Machine Learning: Theory, Principle and Efficacy , year =. 2410.18958 , archivePrefix =

    2025 arXiv

  17. [25]

    Proceedings of the 43rd International Conference on Machine Learning , series =

    Temporal Difference Learning for Diffusion Models , author =. Proceedings of the 43rd International Conference on Machine Learning , series =. 2026 , eprint =

    2026

  18. [26]

    The Fourteenth International Conference on Learning Representations , year =

    Probabilistic Uncertain Reward Model , author =. The Fourteenth International Conference on Learning Representations , year =. 2503.22480 , archivePrefix =

    arXiv

  19. [27]

    The Twelfth International Conference on Learning Representations , year =

    Confidence-aware Reward Optimization for Fine-tuning Text-to-Image Models , author =. The Twelfth International Conference on Learning Representations , year =. 2404.01863 , archivePrefix =

    arXiv

  20. [28]

    doi:10.48550/arXiv.2505.07818 , url =

    Xue, Zeyue and Wu, Jie and Gao, Yu and Kong, Fangyuan and Zhu, Lingting and Chen, Mengzhao and Liu, Zhiheng and Liu, Wei and Guo, Qiushan and Huang, Weilin and Luo, Ping , year =. doi:10.48550/arXiv.2505.07818 , url =. 2505.07818 , archivePrefix =

    doi arXiv

  21. [29]

    2025 , eprint =

    Video Generation Models Are Good Latent Reward Models , author =. 2025 , eprint =. doi:10.48550/arXiv.2511.21541 , url =

    2025 doi

  22. [30]

    doi:10.48550/arXiv.2508.20751 , url =

    Wang, Yibin and Li, Zhimin and Zang, Yuhang and Zhou, Yujie and Bu, Jiazi and Wang, Chunyu and Lu, Qinglin and Jin, Cheng and Wang, Jiaqi , year =. doi:10.48550/arXiv.2508.20751 , url =. 2508.20751 , archivePrefix =

    doi arXiv

  23. [31]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =

    Seeing What Matters: Visual Preference Policy Optimization for Visual Generation , author =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =. 2026 , eprint =

    2026

  24. [32]

    2024 , month = jun, eprint =

    Huang, Ziqi and He, Yinan and Yu, Jiashuo and Zhang, Fan and Si, Chenyang and Jiang, Yuming and Zhang, Yuanhan and Wu, Tianxing and Jin, Qingyang and Chanpaisit, Nattapol and Wang, Yaohui and Chen, Xinyuan and Wang, Limin and Lin, Dahua and Qiao, Yu and Liu, Ziwei , booktitle ...

    2024

  25. [33]

    2024 , doi =

    Jiang, Dongfu and Ku, Max and Li, Tianle and Ni, Yuansheng and Sun, Shizhuo and Fan, Rongqi and Chen, Wenhu , booktitle =. 2024 , doi =. 2406.04485 , archivePrefix =

    2024 arXiv

  26. [34]

    Proceedings of the 41st International Conference on Machine Learning , editor =

    Scaling Rectified Flow Transformers for High-Resolution Image Synthesis , author =. Proceedings of the 41st International Conference on Machine Learning , editor =. 2024 , eprint =

    2024

  27. [35]

    doi:10.48550/arXiv.2503.20314 , url =

    2025 , eprint =. doi:10.48550/arXiv.2503.20314 , url =

    2025 doi arXiv

  28. [36]

    2024 , howpublished =

    2024

  29. [37]

    Proceedings of the 38th International Conference on Machine Learning , editor =

    Learning Transferable Visual Models from Natural Language Supervision , author =. Proceedings of the 38th International Conference on Machine Learning , editor =. 2021 , url =

    2021

  30. [38]

    2025 , issn =

    Li, Bo and Zhang, Yuanhan and Guo, Dong and Zhang, Renrui and Li, Feng and Zhang, Hao and Zhang, Kaichen and Zhang, Peiyuan and Li, Yanwei and Liu, Ziwei and Li, Chunyuan , journal =. 2025 , issn =. 2408.03326 , archivePrefix =

    2025 arXiv

  31. [39]

    doi:10.48550/arXiv.2409.12191 , url =

    Wang, Peng and Bai, Shuai and Tan, Sinan and Wang, Shijie and Fan, Zhihao and Bai, Jinze and Chen, Keqin and Liu, Xuejing and Wang, Jialin and Ge, Wenbin and Fan, Yang and Dang, Kai and Du, Mengfei and Ren, Xuancheng and Men, Rui and Liu, Dayiheng and Zhou, Chang and Zhou, Jin...

    doi arXiv

  32. [40]

    The Twelfth International Conference on Learning Representations , year =

    Podell, Dustin and English, Zion and Lacey, Kyle and Blattmann, Andreas and Dockhorn, Tim and M. The Twelfth International Conference on Learning Representations , year =. 2307.01952 , archivePrefix =

    arXiv

  33. [41]

    and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu , booktitle =

    Hu, Edward J. and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu , booktitle =. 2022 , eprint =

    2022

  34. [42]

    International Conference on Learning Representations , year =

    Decoupled Weight Decay Regularization , author =. International Conference on Learning Representations , year =. 1711.05101 , archivePrefix =

    arXiv

  35. [43]

    Advances in Neural Information Processing Systems , volume =

    Attention Is All You Need , author =. Advances in Neural Information Processing Systems , volume =. 2017 , url =

    2017

  36. [44]

    2022 , eprint =

    Classifier-Free Diffusion Guidance , author =. 2022 , eprint =. doi:10.48550/arXiv.2207.12598 , url =

    2022 doi arXiv

  37. [45]

    Shao, Zhihong and Wang, Peiyi and Zhu, Qihao and Xu, Runxin and Song, Junxiao and Bi, Xiao and Zhang, Haowei and Zhang, Mingchuan and Li, Y. K. and Wu, Y. and Guo, Daya , year =. doi:10.48550/arXiv.2402.03300 , url =. 2402.03300 , archivePrefix =

    doi arXiv

  38. [46]

    Proceedings of the European Conference on Computer Vision (ECCV) , pages =

    Group Normalization , author =. Proceedings of the European Conference on Computer Vision (ECCV) , pages =. 2018 , doi =

    2018

  39. [47]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =

    Cross-Iteration Batch Normalization , author =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =. 2021 , doi =. 2002.05712 , archivePrefix =

    2021 arXiv

  40. [48]

    2026 , doi =

    Xu, Jiazheng and Huang, Yu and Cheng, Jiale and Yang, Yuanming and Xu, Jiajun and Wang, Yuan and Duan, Wenbo and Yang, Shen and Jin, Qunlin and Li, Shurun and Teng, Jiayan and Yang, Zhuoyi and Zheng, Wendi and Liu, Xiao and Zhang, Dan and Ding, Ming and Zhang, Xiaohan and Huan...

    2026

  41. [49]

    doi:10.48550/arXiv.2509.22799 , url =

    He, Xuan and Jiang, Dongfu and Nie, Ping and Liu, Minghao and Jiang, Zhengxuan and Su, Mingyi and Ma, Wentao and Lin, Junru and Ye, Chun and Lu, Yi and Wu, Keming and Schneider, Benjamin and Do, Quy Duc and Li, Zhuofeng and Jia, Yiming and Zhang, Yuxuan and Cheng, Guo and Wang...

    doi

  42. [50]

    Advances in Neural Information Processing Systems , editor =

    Improving Video Generation with Human Feedback , author =. Advances in Neural Information Processing Systems , editor =. 2025 , url =

    2025

  43. [51]

    2024 , address =

    He, Xuan and Jiang, Dongfu and Zhang, Ge and Ku, Max and Soni, Achint and Siu, Sherman and Chen, Haonan and Chandra, Abhranil and Jiang, Ziyan and Arulraj, Aaran and Wang, Kai and Do, Quy Duc and Ni, Yuansheng and Lyu, Bohan and Narsupalli, Yaswanth and Fan, Rongqi and Lyu, Zh...

    2024 doi

  44. [52]

    2025 , url =

    Li, Weiqi and Zhang, Xuanyu and Zhao, Shijie and Zhang, Yabin and Li, Junlin and Zhang, Li and Zhang, Jian , booktitle =. 2025 , url =

    2025

  45. [53]

    2026 , doi =

    Zhang, Xuanyu and Li, Weiqi and Zhao, Shijie and Li, Junlin and Zhang, Li and Zhang, Jian , journal =. 2026 , doi =

    2026

  46. [54]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    VideoDPO: Omni-Preference Alignment for Video Diffusion Generation , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.