Pith. sign in

REVIEW 3 major objections 6 minor 31 cited by

RewardDance claims that recasting reward as a yes-token probability in a VLM unlocks scaling to 26B parameters and yields resistance to reward hacking.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 20:03 UTC pith:VSKFQMK2

load-bearing objection Useful scaling study; the 'proving' reward-hacking claim rests on an unvalidated variance proxy and should be softened. the 3 major comments →

arxiv 2509.08826 v1 pith:VSKFQMK2 submitted 2025-09-10 cs.CV

RewardDance: Reward Scaling in Visual Generation

Jie Wu , Yu Gao , Zilyu Ye , Ming Li , Liang Li , Hanzhong Guo , Jie Liu , Zeyue Xue
show 4 more authors
Xiaoxia Hou Wei Liu Yan Zeng Weilin Huang
This is my paper
classification cs.CV
keywords reward modelsvisual generationreinforcement learning from human feedbackreward hackingscaling lawsvision-language modelstext-to-imagetext-to-video
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

RewardDance argues that the usual way to build visual reward models—a scalar regression head trained with Bradley-Terry loss—is what keeps them from scaling well. Instead, it reformulates reward as the probability that a vision-language model outputs the token 'yes' when asked whether one image beats another, which natively fits how VLMs generate text. This reformulation unlocks scaling along two axes: model parameters (from 1B to 26B) and input context (task instructions, reference images, and chain-of-thought reasoning). Across text-to-image, text-to-video, and image-to-video, the paper reports consistent quality gains from both kinds of scaling. It also reports that large reward models maintain high reward variance during RL fine-tuning, which it reads as evidence that the policy avoids reward hacking and mode collapse.

Core claim

RewardDance replaces the regression head of a VLM reward model with a generative question: 'Is image 2 better than image 1?' The reward is simply the predicted probability of the token 'yes'. This aligns the reward objective with the VLM's next-token prediction, and with that alignment the paper demonstrates that scaling model parameters from 1B to 26B and scaling context (instructions, reference images, chain-of-thought) yields consistent quality improvements in text-to-image, text-to-video, and image-to-video generation. The paper also observes that large reward models keep high reward variance during RL fine-tuning, interpreting this as evidence that the policy avoids reward hacking and m

What carries the argument

The central object is the 'yes'-token probability: r(x1,x2,y,i) = P('yes' | x1,x2,y,i), where x1,x2 are images being compared, y is the prompt, and i is a task instruction. This turns reward prediction into the VLM's native autoregressive task and removes the regression head. Two scaling axes then become effective: model scaling (InternVL variants from 1B to 26B) and context scaling (task instructions, reference images, and chain-of-thought reasoning). The reward curves during RL fine-tuning, with their variance bands, serve as the diagnostic that larger models keep exploring rather than collapsing.

Load-bearing premise

The paper treats high reward variance during RL fine-tuning as proof that the policy is not reward-hacking, but variance alone could also reflect a noisy or miscalibrated reward signal rather than genuinely broad exploration.

What would settle it

Run an RL fine-tuning loop with a large generative reward model and measure actual output diversity (e.g., perceptual feature coverage or pairwise image distance) alongside the reward variance. If reward variance stays high while output diversity drops sharply, or if held-out human preference scores flatten or worsen, the anti-hacking claim would be refuted. Comparing a 26B generative RM against a matching 26B regression RM on identical reference images would isolate whether the benefit comes from the generative formulation or simply from parameter count.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If scaling RMs is the right principle, then visual generation systems should invest in larger VLM-based reward models rather than only larger generators.
  • Context scaling with reference examples and chain-of-thought provides a practical path to better reward signals without changing the base generator.
  • The reported variance signature gives a cheap, training-time early warning for reward hacking: a shrinking variance band during RL fine-tuning indicates the policy is collapsing.
  • The same generative reward formulation should carry over to other preference-based multimodal tasks, such as editing or audio-to-video, with minimal changes.
  • OOD accuracy of the reward model, not in-domain accuracy, is the metric that predicts downstream RL gains, pointing toward new benchmark design for RMs.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's anti-hacking evidence rests on reward variance as a proxy; a stronger test would measure output diversity directly (e.g., feature coverage or perceptual diversity) independently of the reward model used.
  • Scaling laws for reward models may interact with generator scale: the paper already notes that larger diffusion models benefit more from larger RMs, implying a joint-scaling recipe rather than an isolated RM-scaling law.
  • If high variance is indeed the key signal, then reward-model designers might deliberately tune calibration and output entropy, not just accuracy, when training RMs for RL.
  • The pairwise reference-image formulation introduces an N-way search cost at inference; a testable extension would be to amortize or distill the pairwise comparisons into a pointwise model that retains the scaling benefits.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces RewardDance, a generative reward modeling framework for visual generation. Instead of a regression head, the reward score is defined as a VLM's probability of predicting a 'yes' token for 'image 2 is better than image 1' under task-specific instructions, reference examples, and chain-of-thought reasoning. The framework is scaled from 1B to 26B parameters and evaluated on text-to-image, text-to-video, and image-to-video tasks under RL fine-tuning and test-time scaling. The main reported results are consistent scaling gains in alignment score, GenEval, Bench-240, and SeedVideoBench-1.0. The paper further claims that large reward models exhibit high reward variance during RL fine-tuning, which it interprets as proof of resistance to reward hacking and sustained output diversity.

Significance. If the scaling results hold, the paper makes a practically important contribution: it is the first systematic study of scaling a generative VLM-based reward model for visual generation, and it demonstrates gains across multiple base generators (FLUX.1-dev, Seedream-3.0, Seedance-1.0) and two optimization regimes (RL and test-time scaling). The OOD-accuracy discussion is a useful suggestion for reward-model evaluation. However, the headline 'Crucially, we resolve reward hacking' claim is not currently established: it rests entirely on an unvalidated proxy, reward variance, and the body text itself only says the evidence 'suggests' resistance. Because this is the central novelty emphasized in the abstract and introduction, the paper needs either direct validation of the anti-hacking mechanism or a substantial tempering of the claim.

major comments (3)
  1. [Abstract; §4.4; Figs. 2, 5, 6; Eq. (2)] The anti-hacking conclusion is based solely on reward-variance dynamics. The abstract says high reward variance 'proves' resistance to hacking, but §4.4 only says the relationship 'strongly suggests' it, and Fig. 1 calls variance an 'indicator.' No independent validation is provided: no output-diversity metric (e.g., LPIPS, Vendi), no reward-overoptimization curve on held-out human preference, and no comparison with a policy known to have been hacked. A noisy or miscalibrated reward model can produce high temporal variance even for a degenerate policy, while a well-calibrated model can assign stable rewards to diverse high-quality outputs. As the central 'Crucially' contribution, this needs direct evidence, not a proxy.
  2. [Table 2; §4.2] The in-domain RM accuracy is non-monotonic with scale (64.70, 69.36, 65.37, 74.92, 78.44), and the paper introduces OOD accuracy post hoc as 'more critical' after observing that it correlates with the desired scaling trend. With only five RM sizes and no independent, pre-registered metric selection, this is an ad-hoc reinterpretation. OOD accuracy could indeed be the better predictor, but that claim needs support beyond a single correlation on the authors' own data, e.g., cross-validation across RM families or benchmarks, or an a priori argument for why OOD accuracy should be decisive.
  3. [Tables 5 and 6; §4.1.2] Two of the three headline benchmarks, Bench-240 and SeedVideoBench-1.0, are developed by the same organization that produces the evaluated models (Seedream and Seedance). The paper reports no independent human evaluation, no confidence intervals, and no third-party replication for these benchmarks. The SOTA claims in Tables 5 and 6 therefore rest largely on internally constructed evaluation sets. Independent evaluation or release of the full prompt/rating protocol is needed before the SOTA claims can be accepted at face value.
minor comments (6)
  1. [Abstract] Typo: 'It primarily due' should be 'It is primarily due.'
  2. [Figs. 2, 5, 6] The numbers next to the curves (e.g., '=7.2e-3') are not labeled. Please state explicitly that these are standard deviations of raw/smoothed reward scores, and report the sliding-window length (stated in §4.4 as 1,000) in the captions.
  3. [Table 3] The cell entries such as '+28%+32% +4%' are visually confusing. Use separate columns for GSB improvement and its uncertainty, or explain the notation in the caption.
  4. [§3.3.2] Typo: 'point-wisee' should be 'point-wise.'
  5. [§3.2.3] The weighted CE loss coefficient for the pointwise generative variant is described only as 'small.' Please give the exact value for reproducibility.
  6. [§4.4 and Fig. 7] The claim that larger DiT architectures benefit more from reward scaling rests on a single comparison (Seedream-Lite vs. Seedream). Add at least one more model pair or error bars before drawing a scaling-law conclusion.

Circularity Check

0 steps flagged

No significant circularity; central scaling results are empirically grounded, though the anti-hacking claim relies on an unvalidated proxy rather than a circular derivation.

full rationale

RewardDance's core reward score is defined as P(yes) (Eq. 2), and the scaling experiments compare RM sizes on a mix of external (GenEval, ImageReward, HPS) and internal (Bench-240, SeedVideoBench) benchmarks. The GenEval results and OOD preference sets provide independent grounding for the scaling claims, so the central scaling conclusion is not circular. Self-citations to Seedream/Seedance technical reports and benchmarks are present, but they are not used to derive the scaling law; they are evaluation instruments, and external GenEval mitigates the risk. The abstract's statement that high reward variance 'proves' resistance to reward hacking is stronger than Section 4.4's 'strongly suggests'; however, this is a validity threat—variance is an unvalidated proxy for diversity and anti-hacking—rather than a circularity, because the paper does not define resistance to hacking as reward variance by equation. No fitted parameter is renamed as a prediction, and no load-bearing argument reduces to a self-citation. The anti-hacking claim deserves scrutiny on evidential grounds, but it does not meet the standard for derivation-level circularity.

Axiom & Free-Parameter Ledger

3 free parameters · 6 axioms · 0 invented entities

The paper introduces no new physical or mathematical entities. Its conceptual invention is a metric, reward variance as an anti-hacking indicator, which is an unvalidated proxy rather than an entity. The free parameters and assumptions above are the load-bearing choices the central claims depend on.

free parameters (3)
  • CE loss coefficient for pointwise variant = not reported (described as 'a small coefficient')
    Introduced in Section 3.2.3 to stabilize pointwise generative RM training; the value is unspecified, so it is a hand-chosen hyperparameter the reader cannot audit.
  • Best-of-N reference count and selection = N=2, 6, or 16, with Top-2 selection in main experiments
    Section 3.3.1 and Table 8; reference image quality is load-bearing for pairwise reward scoring, and the choice is experimental, not derived.
  • Reward variance window length = 1,000 RL iterations
    Used in Figures 5 and 6 to quantify exploration; the window length is chosen by hand and determines the reported variance values that support the anti-hacking claim.
axioms (6)
  • ad hoc to paper High reward variance during RL training indicates resistance to reward hacking and sustained exploration.
    Figures 2, 5, 6 and Section 4.4 use variance as the evidence for the paper's central anti-hacking claim; no external measure of hacking or mode collapse is provided.
  • domain assumption In-house Bench-240 and SeedVideoBench-1.0 human scores are reliable measures of generation quality.
    Section 4.1.2; benchmarks and scoring rubrics are defined by the authors' organization, and inter-annotator agreement and error bars are not reported.
  • ad hoc to paper OOD RM accuracy, rather than ID accuracy, is the predictive metric for downstream RL quality.
    Section 4.2 adopts this after observing non-monotonic ID accuracies, making it a post hoc modeling choice rather than a pre-specified hypothesis.
  • domain assumption Uniform scaling of the InternVL backbone from 1B to 26B preserves and improves preference-judgment capability.
    Section 3.2.2; no analysis of data scaling, compute, or possible negative scaling effects is given beyond the reported trend.
  • domain assumption CoT reasoning distilled from SEED-VL 1.5 is accurate and improves reward judgments.
    Section 3.2.1 and Figure 4; teacher quality is assumed, not independently evaluated.
  • standard math ReFL with a frozen generative reward model provides a valid RLHF signal for diffusion and flow generators.
    Section 3.3.1; this is a standard assumption from prior work such as ReFL and DDPO.

pith-pipeline@v1.3.0-alltime-deepseek · 17712 in / 12489 out tokens · 133540 ms · 2026-08-04T20:03:58.485216+00:00 · methodology

0 comments
read the original abstract

Reward Models (RMs) are critical for improving generation models via Reinforcement Learning (RL), yet the RM scaling paradigm in visual generation remains largely unexplored. It primarily due to fundamental limitations in existing approaches: CLIP-based RMs suffer from architectural and input modality constraints, while prevalent Bradley-Terry losses are fundamentally misaligned with the next-token prediction mechanism of Vision-Language Models (VLMs), hindering effective scaling. More critically, the RLHF optimization process is plagued by Reward Hacking issue, where models exploit flaws in the reward signal without improving true quality. To address these challenges, we introduce RewardDance, a scalable reward modeling framework that overcomes these barriers through a novel generative reward paradigm. By reformulating the reward score as the model's probability of predicting a "yes" token, indicating that the generated image outperforms a reference image according to specific criteria, RewardDance intrinsically aligns reward objectives with VLM architectures. This alignment unlocks scaling across two dimensions: (1) Model Scaling: Systematic scaling of RMs up to 26 billion parameters; (2) Context Scaling: Integration of task-specific instructions, reference examples, and chain-of-thought (CoT) reasoning. Extensive experiments demonstrate that RewardDance significantly surpasses state-of-the-art methods in text-to-image, text-to-video, and image-to-video generation. Crucially, we resolve the persistent challenge of "reward hacking": Our large-scale RMs exhibit and maintain high reward variance during RL fine-tuning, proving their resistance to hacking and ability to produce diverse, high-quality outputs. It greatly relieves the mode collapse problem that plagues smaller models.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 31 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. DiT-Reward: Generative Representations for Text-to-Image Reward Modeling

    cs.LG 2026-06 unverdicted novelty 7.0

    DiT-Reward converts pretrained DiT models into reward predictors that outperform HPSv3 on four benchmarks while providing 1.65x inference speedup.

  2. Each Judge Its Own Yardstick: Discovering Per-VLM Taxonomies for Physical Video Evaluation

    cs.CV 2026-06 unverdicted novelty 7.0

    JudgeFit produces per-VLM physical video evaluation taxonomies that improve held-out accuracy by a mean 32% relative to a single global schema across 16 models from eight families.

  3. DiffusionOPD: A Unified Perspective of On-Policy Distillation in Diffusion Models

    cs.LG 2026-05 unverdicted novelty 7.0

    DiffusionOPD applies online policy distillation from per-task teachers to a unified diffusion student, with a derived closed-form per-step KL objective that unifies SDE and ODE sampling via mean matching.

  4. CaC: Advancing Video Reward Models via Hierarchical Spatiotemporal Concentrating

    cs.CV 2026-05 unverdicted novelty 7.0

    CaC is a hierarchical spatiotemporal concentrating reward model for video anomalies that reports 25.7% accuracy gains on fine-grained benchmarks and 11.7% anomaly reduction in generated videos via a new dataset and GR...

  5. CaC: Advancing Video Reward Models via Hierarchical Spatiotemporal Concentrating

    cs.CV 2026-05 unverdicted novelty 7.0

    CaC presents a new spatiotemporal concentrating reward model for video anomalies, built on a novel large-scale dataset and three-stage training with RL and IoU rewards, claiming 25.7% accuracy gains and 11.7% anomaly ...

  6. TMPO: Trajectory Matching Policy Optimization for Diverse and Efficient Diffusion Alignment

    cs.LG 2026-05 unverdicted novelty 7.0

    TMPO uses Softmax Trajectory Balance to match policy probabilities over multiple trajectories to a Boltzmann reward distribution, improving diversity by 9.1% in diffusion alignment tasks.

  7. TMPO: Trajectory Matching Policy Optimization for Diverse and Efficient Diffusion Alignment

    cs.LG 2026-05 unverdicted novelty 7.0

    TMPO replaces scalar reward maximization with trajectory-level matching to a Boltzmann distribution via Softmax-TB, improving generative diversity by 9.1% while keeping competitive reward performance.

  8. Beyond VLM-Based Rewards: Diffusion-Native Latent Reward Modeling

    cs.CV 2026-02 unverdicted novelty 7.0

    DiNa-LRM introduces a diffusion-native latent reward model using a noise-calibrated Thurstone likelihood on noisy states, matching VLM performance at lower compute in image alignment and preference optimization.

  9. MixGRPO: Unlocking Flow-based GRPO Efficiency with Mixed ODE-SDE

    cs.AI 2025-07 unverdicted novelty 7.0

    MixGRPO speeds up GRPO for flow-based image generators by restricting SDE sampling and optimization to a sliding window while using ODE elsewhere, cutting training time by up to 71% with better alignment performance.

  10. SciForma: Structure-Faithful Generation of Scientific Diagrams

    cs.CV 2026-07 conditional novelty 6.0

    A 9B open-weights model trained with axis-decomposed conjunctive preference optimization (M-DPO) and a structural inventory beats GPT-Image-1.5 on scientific-diagram structural-fidelity benchmarks.

  11. NormGuard: Reward-Preserving Norm Constraints in Flow-Matching Reinforcement Learning

    cs.LG 2026-06 conditional novelty 6.0

    A hinge regularizer that penalizes velocity-norm growth beyond the reference model improves perceptual quality and realism in RL-finetuned image flow models without sacrificing reward.

  12. NormGuard: Reward-Preserving Norm Constraints in Flow-Matching Reinforcement Learning

    cs.LG 2026-06 unverdicted novelty 6.0

    NormGuard adds a training-time hinge penalty on velocity norm inflation in flow-matching RL to improve MLLM-judged image quality and forensic realism while preserving reward across multiple setups.

  13. NormGuard: Reward-Preserving Norm Constraints in Flow-Matching Reinforcement Learning

    cs.LG 2026-06 conditional novelty 6.0

    NormGuard, a hinge penalty on excess velocity norm during RL post-training of flow models, improves perceptual quality and realism without sacrificing reward.

  14. Z-Reward: Beyond Scalar Rewards by Internalizing Reasoning into Score Distributions

    cs.CV 2026-06 unverdicted novelty 6.0

    Z-Reward trains a 27B reasoning teacher VLM on score distributions via GDSO and distills it via RISD into a 9B student, reaching 89.6% and 88.6% human preference accuracy with 41.3% optimization gain over SFT baseline.

  15. Z-Reward: Beyond Scalar Rewards by Internalizing Reasoning into Score Distributions

    cs.CV 2026-06 conditional novelty 6.0

    A teacher-student reward model learns reasoning-conditioned score distributions for text-to-image images, yielding ~89% preference accuracy and a 41% net human-preference gain when used for generator optimization.

  16. Are we really tilting? The mechanics of reward guidance in flow and diffusion models

    cs.LG 2026-06 unverdicted novelty 6.0

    Finite-particle approximation of the Doob h-function causes reward hacking via two failure modes in reward-guided diffusion; a damping schedule corrects within-mode bias in Gaussian settings.

  17. StressDream: Steering Video World Models for Robust Policy Evaluation and Improvement

    cs.CV 2026-05 unverdicted novelty 6.0

    StressDream optimizes initial noise in diffusion video world models using VLM semantic and plausibility objectives to steer generations toward specified high-impact outcomes for improved policy evaluation.

  18. Think, then Score: Decoupled Reasoning and Scoring for Video Reward Modeling

    cs.CV 2026-05 unverdicted novelty 6.0

    DeScore decouples CoT reasoning from reward scoring in video reward models using a two-stage training process to improve generalization and avoid optimization bottlenecks of coupled generative RMs.

  19. Leveraging Verifier-Based Reinforcement Learning in Image Editing

    cs.CV 2026-04 unverdicted novelty 6.0

    Edit-R1 trains a CoT-based reasoning reward model with GCPO and uses it to boost image editing performance over VLMs and models like FLUX.1-kontext via GRPO.

  20. Leveraging Verifier-Based Reinforcement Learning in Image Editing

    cs.CV 2026-04 unverdicted novelty 6.0

    Edit-R1 builds a CoT-based reasoning reward model (RRM) via SFT and GCPO, then applies it with GRPO to improve image editing models such as FLUX.1-kontext.

  21. Seedance 1.5 pro: A Native Audio-Visual Joint Generation Foundation Model

    cs.CV 2025-12 unverdicted novelty 6.0

    Seedance 1.5 pro is a joint audio-visual generation model achieving high synchronization via dual-branch diffusion transformer and post-training optimizations.

  22. Reward Forcing: Efficient Streaming Video Generation with Rewarded Distribution Matching Distillation

    cs.CV 2025-12 conditional novelty 6.0

    Reward Forcing combines EMA-Sink tokens and Rewarded Distribution Matching Distillation to deliver state-of-the-art streaming video generation at 23.1 FPS without copying initial frames.

  23. Distribution Matching Distillation Meets Reinforcement Learning

    cs.CV 2025-11 conditional novelty 6.0

    Combining DMD distillation with RL during training produces few-step text-to-image models that outperform their multi-step teacher on several benchmarks.

  24. Uniworld-V2: Reinforce Image Editing with Diffusion Negative-aware Finetuning and MLLM Implicit Feedback

    cs.CV 2025-10 unverdicted novelty 6.0

    UniWorld-V2 applies policy optimization via DiffusionNFT and MLLM logit feedback with group filtering to reach state-of-the-art scores of 4.49 on ImgEdit and 7.83 on GEdit-Bench while remaining model-agnostic.

  25. Multi-Axis Max@K Reinforcement Learning for Representative Diversity in Text-to-Image Generation

    cs.LG 2026-07 conditional novelty 5.0

    A per-category best-of-K RL reward, multi-axis max@K, shifts SD3.5-M perceived-appearance distributions toward uniform coverage (Fairness Score +0.23 to +0.36) without quality loss.

  26. Improving Visual Representation Alignment Generation with GRPO

    cs.CV 2026-05 unverdicted novelty 5.0

    VRPO applies generative representation policy optimization to dynamically align diffusion features with pretrained visual encoders, claiming +1.8 FID gains and 2.3x faster training versus REPA.

  27. Edit-GRPO: A Locality-Preserving Policy Optimization Framework for Image Editing

    cs.CV 2026-05 unverdicted novelty 5.0

    Edit-GRPO decouples editing and preservation objectives via region-specific signals in a policy optimization framework to improve locality in image editing tasks.

  28. Think, then Score: Decoupled Reasoning and Scoring for Video Reward Modeling

    cs.CV 2026-05 unverdicted novelty 5.0

    DeScore decouples explicit CoT reasoning from reward regression in video reward models via a two-stage cold-start plus dual-objective RL training pipeline.

  29. A Systematic Post-Train Framework for Video Generation

    cs.CV 2026-04 unverdicted novelty 5.0

    A post-training pipeline for video generation models combines SFT, RLHF with novel GRPO, prompt enhancement, and inference optimization to improve visual quality, temporal coherence, and instruction following.

  30. Seedance 2.0: Advancing Video Generation for World Complexity

    cs.CV 2026-04 unverdicted novelty 3.0

    Seedance 2.0 is an updated multi-modal model for generating 4-15 second audio-video content at 480p/720p with support for up to 3 video, 9 image, and 3 audio references.

  31. Seedream 4.0: Toward Next-generation Multimodal Image Generation

    cs.CV 2025-09 unverdicted novelty 3.0

    Seedream 4.0 unifies text-to-image synthesis, image editing, and multi-image composition in an efficient diffusion transformer pretrained on billions of pairs and accelerated to 1.8 seconds for 2K output.

Reference graph

Works this paper leans on

72 extracted references · 37 linked inside Pith · cited by 24 Pith papers

  1. [1]

    Vidu: a highly consistent, dynamic and skilled text-to-video generator with diffusion models

    Fan Bao, Chendong Xiang, Gang Yue, Guande He, Hongzhou Zhu, Kaiwen Zheng, Min Zhao, Shilong Liu, Yaole Wang, and Jun Zhu. Vidu: a highly consistent, dynamic and skilled text-to-video generator with diffusion models. arXiv preprint arXiv:2405.04233, 2024

    Pith/arXiv arXiv 2024

  2. [2]

    Improving image generation with better captions.Computer Science

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions.Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2(3):8, 2023

    2023

  3. [3]

    Training diffusion models with reinforcement learning

    Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement learning. arXiv preprint arXiv:2305.13301, 2023

    Pith/arXiv arXiv 2023

  4. [4]

    Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023

    Pith/arXiv arXiv 2023

  5. [5]

    Rank analysis of incomplete block designs: I

    Ralph Allan Bradley and Milton E Terry. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(3/4):324–345, 1952

    1952

  6. [6]

    Video generation models as world simulators.OpenAI Blog, 1(8):1, 2024

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, et al. Video generation models as world simulators.OpenAI Blog, 1(8):1, 2024

    2024

  7. [7]

    Pixart-α: Fast training of diffusion transformer for photorealistic text-to-image synthesis

    Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, et al. Pixart-α: Fast training of diffusion transformer for photorealistic text-to-image synthesis. arXiv preprint arXiv:2310.00426, 2023

    Pith/arXiv arXiv 2023

  8. [8]

    Control-a-video: Controllable text-to-video generation with diffusion models.CoRR, 2023

    Weifeng Chen, Jie Wu, Pan Xie, Hefeng Wu, Jiashi Li, Xin Xia, Xuefeng Xiao, and Liang Lin. Control-a-video: Controllable text-to-video generation with diffusion models.CoRR, 2023

    2023

  9. [9]

    The accuracy paradox in rlhf: When better reward models don’t yield better language models.arXiv preprint arXiv:2410.06554, 2024

    Yanjun Chen, Dawei Zhu, Yirong Sun, Xinghao Chen, Wei Zhang, and Xiaoyu Shen. The accuracy paradox in rlhf: When better reward models don’t yield better language models.arXiv preprint arXiv:2410.06554, 2024

    Pith/arXiv arXiv 2024

  10. [10]

    Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling.arXiv preprint arXiv:2412.05271, 2024

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling.arXiv preprint arXiv:2412.05271, 2024

    Pith/arXiv arXiv 2024

  11. [11]

    Raft: Reward ranked finetuning for generative foundation model alignment

    Hanze Dong, Wei Xiong, Deepanshu Goyal, Yihan Zhang, Winnie Chow, Rui Pan, Shizhe Diao, Jipeng Zhang, Kashun Shum, and Tong Zhang. Raft: Reward ranked finetuning for generative foundation model alignment. arXiv preprint arXiv:2304.06767, 2023

    Pith/arXiv arXiv 2023

  12. [12]

    Scaling rectified flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-firstinternational conference on machine learning, 2024

    2024

  13. [13]

    Reinforcement learning for fine-tuning text-to-image diffusion mod- els

    Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, Kangwook Lee, and Kimin Lee. Reinforcement learning for fine-tuning text-to-image diffusion mod- els. InThirty-seventhConference on Neural Information Processing Systems (NeurIPS) 2023. Neural Information Processing Systems Foundation, 2023

    2023

  14. [14]

    Seedream 3.0 technical report.arXiv preprint arXiv:2504.11346, 2025

    Yu Gao, Lixue Gong, Qiushan Guo, Xiaoxia Hou, Zhichao Lai, Fanshi Li, Liang Li, Xiaochen Lian, Chao Liao, Liyang Liu, et al. Seedream 3.0 technical report.arXiv preprint arXiv:2504.11346, 2025

    Pith/arXiv arXiv 2025

  15. [15]

    Seedance 1.0: Exploring the boundaries of video generation models

    Yu Gao, Haoyuan Guo, Tuyen Hoang, Weilin Huang, Lu Jiang, Fangyuan Kong, Huixia Li, Jiashi Li, Liang Li, Xiaojie Li, et al. Seedance 1.0: Exploring the boundaries of video generation models. arXiv preprint arXiv:2506.09113, 2025

    Pith/arXiv arXiv 2025

  16. [16]

    Seedream 2.0: A native chinese-english bilingual image generation foundation model.arXiv preprint arXiv:2503.07703, 2025

    Lixue Gong, Xiaoxia Hou, Fanshi Li, Liang Li, Xiaochen Lian, Fei Liu, Liyang Liu, Wei Liu, Wei Lu, Yichun Shi, et al. Seedream 2.0: A native chinese-english bilingual image generation foundation model.arXiv preprint arXiv:2503.07703, 2025

    Pith/arXiv arXiv 2025

  17. [17]

    Veo.https://deepmind.google/models/veo/, 2025

    Google. Veo.https://deepmind.google/models/veo/, 2025

    2025

  18. [18]

    Multi-reward as condition for instruction-based image editing.arXiv preprint arXiv:2411.04713, 2024

    Xin Gu, Ming Li, Libo Zhang, Fan Chen, Longyin Wen, Tiejian Luo, and Sijie Zhu. Multi-reward as condition for instruction-based image editing.arXiv preprint arXiv:2411.04713, 2024. 16

    Pith/arXiv arXiv 2024

  19. [19]

    Animatediff: Animate your personalized text-to-image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to-image diffusion models without specific tuning. arXiv preprint arXiv:2307.04725, 2023

    Pith/arXiv arXiv 2023

  20. [20]

    A simple and effective reinforcement learning method for text-to-image diffusion fine-tuning

    Shashank Gupta, Chaitanya Ahuja, Tsung-Yu Lin, Sreya Dutta Roy, Harrie Oosterhuis, Maarten de Rijke, and Satya Narayan Shukla. A simple and effective reinforcement learning method for text-to-image diffusion fine-tuning. arXiv preprint arXiv:2503.00897, 2025

    arXiv 2025

  21. [21]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020

    2020

  22. [22]

    Ideogram.https://about.ideogram.ai/1.0., 2024

    Ideogram. Ideogram.https://about.ideogram.ai/1.0., 2024

    2024

  23. [23]

    Pick-a-pic: An open dataset of user preferences for text-to-image generation.Advancesin neural information processing systems, 36:36652–36663, 2023

    Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. Pick-a-pic: An open dataset of user preferences for text-to-image generation.Advancesin neural information processing systems, 36:36652–36663, 2023

    2023

  24. [24]

    klingai.https://app.klingai.com/cn/, 2025

    klingai. klingai.https://app.klingai.com/cn/, 2025

    2025

  25. [25]

    Hunyuanvideo: A systematic framework for large video generative models.arXiv preprint arXiv:2412.03603, 2024

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models.arXiv preprint arXiv:2412.03603, 2024

    Pith/arXiv arXiv 2024

  26. [26]

    Flux: Official inference repository for flux.1 models, 2024

    Black Forest Labs. Flux: Official inference repository for flux.1 models, 2024. URL https://github.com/ black-forest-labs/flux. Accessed: 2024-11-12

    2024

  27. [27]

    Flux.https://github.com/black-forest-labs/flux, 2024

    Black Forest Labs. Flux.https://github.com/black-forest-labs/flux, 2024

    2024

  28. [28]

    Controlnet++: Improving conditional controls with efficient consistency feedback: Project page: liming-ai

    Ming Li, Taojiannan Yang, Huafeng Kuang, Jie Wu, Zhaoning Wang, Xuefeng Xiao, and Chen Chen. Controlnet++: Improving conditional controls with efficient consistency feedback: Project page: liming-ai. github. io/controlnet_- plus_plus. In European Conference on Computer Vision, pages 129–147. Springer, 2024

    2024

  29. [29]

    Superedit: Rectifying and facilitating supervision for instruction-based image editing.arXiv preprint arXiv:2505.02370, 2025

    Ming Li, Xin Gu, Fan Chen, Xiaoying Xing, Longyin Wen, Chen Chen, and Sijie Zhu. Superedit: Rectifying and facilitating supervision for instruction-based image editing.arXiv preprint arXiv:2505.02370, 2025

    Pith/arXiv arXiv 2025

  30. [30]

    Exploring how generative mllms perceive more than clip with the same vision encoder.arXiv preprint arXiv:2411.05195, 2024

    Siting Li, Pang Wei Koh, and Simon Shaolei Du. Exploring how generative mllms perceive more than clip with the same vision encoder.arXiv preprint arXiv:2411.05195, 2024

    Pith/arXiv arXiv 2024

  31. [31]

    An inverse scaling law for clip training.Advancesin Neural Information Processing Systems, 36:49068–49087, 2023

    Xianhang Li, Zeyu Wang, and Cihang Xie. An inverse scaling law for clip training.Advancesin Neural Information Processing Systems, 36:49068–49087, 2023

    2023

  32. [32]

    Mogao: An omni foundation model for interleaved multi-modal generation.arXiv preprint arXiv:2505.05472, 2025

    Chao Liao, Liyang Liu, Xun Wang, Zhengxiong Luo, Xinyu Zhang, Wenliang Zhao, Jie Wu, Liang Li, Zhi Tian, and Weilin Huang. Mogao: An omni foundation model for interleaved multi-modal generation.arXiv preprint arXiv:2505.05472, 2025

    Pith/arXiv arXiv 2025

  33. [33]

    Flow matching for generative modeling

    Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747, 2022

    Pith/arXiv arXiv 2022

  34. [34]

    Flow-grpo: Training flow matching models via online rl.arXiv preprint arXiv:2505.05470, 2025

    Jie Liu, Gongye Liu, Jiajun Liang, Yangguang Li, Jiaheng Liu, Xintao Wang, Pengfei Wan, Di Zhang, and Wanli Ouyang. Flow-grpo: Training flow matching models via online rl.arXiv preprint arXiv:2505.05470, 2025

    Pith/arXiv arXiv 2025

  35. [35]

    Improving video generation with human feedback.arXiv preprint arXiv:2501.13918, 2025

    Jie Liu, Gongye Liu, Jiajun Liang, Ziyang Yuan, Xiaokun Liu, Mingwu Zheng, Xiele Wu, Qiulin Wang, Wenyu Qin, Menghan Xia, et al. Improving video generation with human feedback.arXiv preprint arXiv:2501.13918, 2025

    Pith/arXiv arXiv 2025

  36. [36]

    Inference-time scaling for generalist reward modeling.arXiv preprint arXiv:2504.02495, 2025

    Zijun Liu, Peiyi Wang, Runxin Xu, Shirong Ma, Chong Ruan, Peng Li, Yang Liu, and Yu Wu. Inference-time scaling for generalist reward modeling.arXiv preprint arXiv:2504.02495, 2025

    arXiv 2025

  37. [37]

    lumalabs.https://lumalabs.ai/, 2024

    lumalabs. lumalabs.https://lumalabs.ai/, 2024

    2024

  38. [38]

    Step-video-t2v technical report: The practice, challenges, and future of video foundation model.arXiv preprint arXiv:2502.10248, 2025

    Guoqing Ma, Haoyang Huang, Kun Yan, Liangyu Chen, Nan Duan, Shengming Yin, Changyi Wan, Ranchen Ming, Xiaoniu Song, Xing Chen, et al. Step-video-t2v technical report: The practice, challenges, and future of video foundation model.arXiv preprint arXiv:2502.10248, 2025

    Pith/arXiv arXiv 2025

  39. [39]

    Inference-time scaling for diffusion models beyond scaling denoising steps

    Nanye Ma, Shangyuan Tong, Haolin Jia, Hexiang Hu, Yu-Chuan Su, Mingda Zhang, Xuan Yang, Yandong Li, Tommi Jaakkola, Xuhui Jia, et al. Inference-time scaling for diffusion models beyond scaling denoising steps. arXiv preprint arXiv:2501.09732, 2025. 17

    Pith/arXiv arXiv 2025

  40. [40]

    Hpsv3: Towards wide-spectrum human preference score

    Yuhang Ma, Xiaoshi Wu, Keqiang Sun, and Hongsheng Li. Hpsv3: Towards wide-spectrum human preference score. arXiv preprint arXiv:2508.03789, 2025

    Pith/arXiv arXiv 2025

  41. [41]

    midjourney.https://www.midjourney.com/home, 2024

    midjourney. midjourney.https://www.midjourney.com/home, 2024

    2024

  42. [42]

    Inference-time text-to-video alignment with diffusion latent beam search.arXiv preprint arXiv:2501.19252, 2025

    Yuta Oshima, Masahiro Suzuki, Yutaka Matsuo, and Hiroki Furuta. Inference-time text-to-video alignment with diffusion latent beam search.arXiv preprint arXiv:2501.19252, 2025

    arXiv 2025

  43. [43]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744, 2022

    2022

  44. [44]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023

    Pith/arXiv arXiv 2023

  45. [45]

    Hierarchical text-conditional image generation with clip latents.arXiv preprint arXiv:2204.06125, 1(2):3, 2022

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents.arXiv preprint arXiv:2204.06125, 1(2):3, 2022

    Pith/arXiv arXiv 2022

  46. [46]

    What makes a reward model a good teacher? an optimization perspective.arXiv preprint arXiv:2503.15477, 2025

    Noam Razin, Zixuan Wang, Hubert Strauss, Stanley Wei, Jason D Lee, and Sanjeev Arora. What makes a reward model a good teacher? an optimization perspective.arXiv preprint arXiv:2503.15477, 2025

    arXiv 2025

  47. [47]

    recraft.https://www.recraft.ai/, 2024

    recraft. recraft.https://www.recraft.ai/, 2024

    2024

  48. [48]

    Byteedit: Boost, comply and accelerate generative image editing

    Yuxi Ren, Jie Wu, Yanzuo Lu, Huafeng Kuang, Xin Xia, Xionghui Wang, Qianqian Wang, Yixing Zhu, Pan Xie, Shiyin Wang, et al. Byteedit: Boost, comply and accelerate generative image editing. InEuropean Conference on Computer Vision, pages 184–200. Springer, 2024

    2024

  49. [49]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    2022

  50. [50]

    Runway.https://runwayml.com/research/introducing-runway-gen-4, 2025

    Runway. Runway.https://runwayml.com/research/introducing-runway-gen-4, 2025

    2025

  51. [51]

    Seaweed-7b: Cost-effective training of video generation foundation model.arXiv preprint arXiv:2504.08685, 2025

    Team Seawead, Ceyuan Yang, Zhijie Lin, Yang Zhao, Shanchuan Lin, Zhibei Ma, Haoyuan Guo, Hao Chen, Lu Qi, Sen Wang, et al. Seaweed-7b: Cost-effective training of video generation foundation model.arXiv preprint arXiv:2504.08685, 2025

    Pith/arXiv arXiv 2025

  52. [52]

    Deep unsupervised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. InInternational conference on machine learning, pages 2256–2265. pmlr, 2015

    2015

  53. [53]

    Score-based generative modeling through stochastic differential equations.arXiv preprint arXiv:2011.13456, 2020

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations.arXiv preprint arXiv:2011.13456, 2020

    Pith/arXiv arXiv 2011

  54. [54]

    Autoregressive model beats diffusion: Llama for scalable image generation.arXiv preprint arXiv:2406.06525, 2024

    Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation.arXiv preprint arXiv:2406.06525, 2024

    Pith/arXiv arXiv 2024

  55. [55]

    Diffusion model alignment using direct preference optimization

    Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8228–8238, 2024

    2024

  56. [56]

    Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianx- iao Yang, et al. Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025

    Pith/arXiv arXiv 2025

  57. [57]

    Worldpm: Scaling human preference modeling.arXiv preprint arXiv:2505.10527, 2025

    Binghai Wang, Runji Lin, Keming Lu, Le Yu, Zhenru Zhang, Fei Huang, Chujie Zheng, Kai Dang, Yang Fan, Xingzhang Ren, et al. Worldpm: Scaling human preference modeling.arXiv preprint arXiv:2505.10527, 2025

    Pith/arXiv arXiv 2025

  58. [58]

    Emu3: Next-token prediction is all you need.arXiv preprint arXiv:2409.18869, 2024

    Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, et al. Emu3: Next-token prediction is all you need.arXiv preprint arXiv:2409.18869, 2024

    Pith/arXiv arXiv 2024

  59. [59]

    Unified multimodal chain-of-thought reward model through reinforcement fine-tuning.arXiv preprint arXiv:2505.03318, 2025

    Yibin Wang, Zhimin Li, Yuhang Zang, Chunyu Wang, Qinglin Lu, Cheng Jin, and Jiaqi Wang. Unified multimodal chain-of-thought reward model through reinforcement fine-tuning.arXiv preprint arXiv:2505.03318, 2025

    arXiv 2025

  60. [60]

    Rethinking reward model evaluation: Are we barking up the wrong tree?arXiv preprint arXiv:2410.05584, 2024

    Xueru Wen, Jie Lou, Yaojie Lu, Hongyu Lin, Xing Yu, Xinyu Lu, Ben He, Xianpei Han, Debing Zhang, and Le Sun. Rethinking reward model evaluation: Are we barking up the wrong tree?arXiv preprint arXiv:2410.05584, 2024. 18

    Pith/arXiv arXiv 2024

  61. [61]

    Qwen-image technical report.arXiv preprint arXiv:2508.02324, 2025

    Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng-ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, et al. Qwen-image technical report.arXiv preprint arXiv:2508.02324, 2025

    Pith/arXiv arXiv 2025

  62. [62]

    Better aligning text-to-image models with human preference

    Xiaoshi Wu, Keqiang Sun, Feng Zhu, Rui Zhao, and Hongsheng Li. Better aligning text-to-image models with human preference. arXiv preprint arXiv:2303.14420, 1(3), 2023

    Pith/arXiv arXiv 2023

  63. [63]

    Human preference score: Better aligning text-to-image models with human preference

    Xiaoshi Wu, Keqiang Sun, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score: Better aligning text-to-image models with human preference. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 2096–2105, 2023

    2096

  64. [64]

    Imagereward: Learning and evaluating human preferences for text-to-image generation

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: Learning and evaluating human preferences for text-to-image generation. Advances in Neural Information Processing Systems, 36:15903–15935, 2023

    2023

  65. [65]

    Visionreward: Fine-grained multi-dimensional human preference learning for image and video generation

    Jiazheng Xu, Yu Huang, Jiale Cheng, Yuanming Yang, Jiajun Xu, Yuan Wang, Wenbo Duan, Shen Yang, Qunlin Jin, Shurun Li, et al. Visionreward: Fine-grained multi-dimensional human preference learning for image and video generation. arXiv preprint arXiv:2412.21059, 2024

    Pith/arXiv arXiv 2024

  66. [66]

    A unified pairwise framework for rlhf: Bridging generative reward modeling and policy optimization.arXiv preprint arXiv:2504.04950, 2025

    Wenyuan Xu, Xiaochen Zuo, Chao Xin, Yu Yue, Lin Yan, and Yonghui Wu. A unified pairwise framework for rlhf: Bridging generative reward modeling and policy optimization.arXiv preprint arXiv:2504.04950, 2025

    Pith/arXiv arXiv 2025

  67. [67]

    Dancegrpo: Unleashing grpo on visual generation.arXiv preprint arXiv:2505.07818, 2025

    Zeyue Xue, Jie Wu, Yu Gao, Fangyuan Kong, Lingting Zhu, Mengzhao Chen, Zhiheng Liu, Wei Liu, Qiushan Guo, Weilin Huang, et al. Dancegrpo: Unleashing grpo on visual generation.arXiv preprint arXiv:2505.07818, 2025

    Pith/arXiv arXiv 2025

  68. [68]

    Schedule on the fly: Diffusion time prediction for faster and better image generation

    Zilyu Ye, Zhiyang Chen, Tiancheng Li, Zemin Huang, Weijian Luo, and Guo-Jun Qi. Schedule on the fly: Diffusion time prediction for faster and better image generation. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 23412–23422, 2025

    2025

  69. [69]

    Make pixels dance: High-dynamic video generation

    Yan Zeng, Guoqiang Wei, Jiani Zheng, Jiaxin Zou, Yang Wei, Yuchen Zhang, and Hang Li. Make pixels dance: High-dynamic video generation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8850–8860, 2024

    2024

  70. [70]

    Onlinevpo: Align video diffusion model with online video-centric preference optimization.arXiv preprint arXiv:2412.15159, 2024

    Jiacheng Zhang, Jie Wu, Weifeng Chen, Yatai Ji, Xuefeng Xiao, Weilin Huang, and Kai Han. Onlinevpo: Align video diffusion model with online video-centric preference optimization.arXiv preprint arXiv:2412.15159, 2024

    arXiv 2024

  71. [71]

    Unifl: Improve latent diffusion model via unified feedback learning.Advances in Neural Information Processing Systems, 37:67355–67382, 2024

    Jiacheng Zhang, Jie Wu, Yuxi Ren, Xin Xia, Huafeng Kuang, Pan Xie, Jiashi Li, Xuefeng Xiao, Weilin Huang, Shilei Wen, et al. Unifl: Improve latent diffusion model via unified feedback learning.Advances in Neural Information Processing Systems, 37:67355–67382, 2024

    2024

  72. [72]

    Fine-tuning language models from human preferences.arXiv preprint arXiv:1909.08593, 2019

    Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences.arXiv preprint arXiv:1909.08593, 2019. 19

    Pith/arXiv arXiv 1909