Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

The paper establishes that aligning a flow-matching model to a target distribution is mathematically an imitation-learning problem, and that an adversarial discriminator can do the alignment from 13,000 single-example demonstrations.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 23:53 UTC pith:RFWNDDP3

load-bearing objection Frames flow post-training as adversarial imitation learning with promising results, but FAIL-PD's one-step denoising gradient is a biased surrogate that needs error analysis, and the theory is overclaimed. the 4 major comments →

arxiv 2602.12155 v2 pith:RFWNDDP3 submitted 2026-02-12 cs.CV

FAIL: Flow Matching Adversarial Imitation Learning for Image Generation

Yeyao Ma , Chen Li , Xiaosong Zhang , Han Hu , Weidi Xie This is my paper
classification cs.CV
keywords flow matchingadversarial imitation learningimage generationpost-training alignmentpathwise derivativepolicy gradientreward hackingdiscriminator
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Post-training a flow-matching generative model is, at root, an imitation-learning problem: the goal is to move the model's output distribution toward an expert distribution. The paper introduces an adversarial imitation-learning framework, FAIL, that does this with a discriminator instead of reward models or preference pairs. Two update rules are derived: FAIL-PD backpropagates discriminator gradients through the differentiable ODE solver, and FAIL-PG uses a black-box policy-gradient update for discrete or compute-limited settings. With only 13,000 expert images, one per prompt, fine-tuning a large text-to-image model lifts prompt-following and aesthetic benchmarks substantially. The same recipe extends to discrete image generation and video, and stabilizes reward-based fine-tuning against reward hacking.

Core claim

Aligning a flow-matching model to a target distribution is, the paper argues, the same problem as imitation learning, so adversarial imitation can substitute for supervised fine-tuning and preference optimization. FAIL treats the flow model's vector field as a policy and trains a discriminator to tell generated samples from expert samples. FAIL-PD backpropagates discriminator gradients through the differentiable ODE solver via a single-step denoising approximation, yielding low-variance pathwise gradients; FAIL-PG treats the discriminator as a scalar reward and uses a clipped policy-gradient objective whose likelihood ratio is estimated from conditional-flow-matching losses, with a KL constr

What carries the argument

The load-bearing object is the minimax objective in Eq. (1), where a policy πθ (the flow model's vector field) tries to make generated samples indistinguishable from expert samples in the eyes of a discriminator Dω. For FAIL-PD, the central identity is the single-step denoising update in Eq. (2), which reconstructs the clean sample x'_0 from a noisy interpolation x_t under the assumption that the vector field is locally linear; this lets discriminator gradients flow back to the policy without unrolling the full ODE. For FAIL-PG, the central identity is the likelihood-ratio estimate r(θ)=exp(L_CFM(θ_old,x) − L_CFM(θ,x)), which converts conditional-flow-matching losses into a policy weight for

Load-bearing premise

The method assumes the generation path is locally straight enough that one denoising step can reconstruct the clean image and give an unbiased learning signal; where the path curves more sharply, FAIL-PD's gradients are biased.

What would settle it

Run FAIL-PD on a flow with deliberately curved trajectories—for example, a synthetic Gaussian-mixture flow—and compare the single-step approximation against full ODE-unrolled backpropagation: if sample fidelity or gradient alignment degrades as curvature increases, the local-linearity premise is falsified.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Supervised fine-tuning's distribution-shift problem can be corrected with dense adversarial feedback, so small single-example expert datasets become viable for alignment.
  • No preference pairs or trained reward models are needed; a dynamically updated discriminator supplies the training signal, removing a major data bottleneck for post-training.
  • The pathwise variant, FAIL-PD, remains stable over thousands of update steps, suggesting that differentiable flow dynamics admit long-horizon alignment without mode collapse.
  • The identical framework transfers to discrete autoregressive image generation and to video, indicating the imitation-learning formulation is modality-agnostic.
  • Combining FAIL with a reward model prevents reward hacking: the adversarial distribution-matching term keeps the policy from overfitting to reward artifacts.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the single-step linearity assumption in Eq. (2) degrades on curved vector fields, FAIL-PD's advantage over FAIL-PG would shrink; the bias could be measured by comparing single-step gradients against full-trajectory backpropagation on the same samples.
  • Because the discriminator is learned online, the method's ceiling is tied to discriminator capacity; with a weaker discriminator, the regularization effect against reward hacking would likely weaken, which is testable by ablating the backbone.
  • The stated equivalence suggests that any generative model with differentiable dynamics—not just flow matching—could inherit the same post-training recipe, which would make adversarial imitation a general alignment primitive.
  • A scaling puzzle is left implicit: 13,000 demonstrations may not be enough to transfer deep semantic skills like text rendering, and it is unclear whether the UniGen gain comes from distribution matching or from the discriminator's own inductive biases.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes FAIL, an adversarial imitation learning framework for post-training flow matching text-to-image models. It introduces two algorithms: FAIL-PD, which backpropagates discriminator gradients through a single-step denoising approximation of the ODE trajectory, and FAIL-PG, a black-box policy-gradient variant using FPO-style likelihood-ratio estimates with KL regularization. The method is evaluated by fine-tuning FLUX.1-dev on 13,000 single-image expert demonstrations from a Gemini Pro 3 model, reporting improvements on UniGen-Bench, DPG-Bench, HPSv3, and UnifiedReward over SFT and online-DPO baselines, plus extensions to discrete autoregressive image generation and video generation. The paper also positions SFT as behavioral cloning and post-training as imitation learning, and claims that FAIL acts as a regularizer against reward hacking.

Significance. If the empirical claims hold, FAIL offers a preference-free, reward-free alignment method that is data-efficient and easy to combine with existing RL fine-tuning, with broad modality coverage. The release of code and data is a strength. The central novelty, however, rests on two under-verified pillars: the single-step denoising pathwise gradient in FAIL-PD and the CFM-loss-based likelihood-ratio/KL surrogate in FAIL-PG. The theoretical framing as an unbiased DDPG limit with GAN convergence properties is asserted rather than proved. The empirical evaluation also mixes evaluation protocols across tables and uses HPSv3, a metric authored by one of the paper's authors, as a headline aesthetic metric. These concerns are substantial but addressable within the manuscript's scope.

major comments (4)
  1. [§2.1, Eq. (2)] The text claims that unrolling the ODE solver lets one 'compute the exact gradient of the discriminator's output with respect to the flow parameters via the chain rule,' but the actual algorithm replaces full backpropagation with the single-step denoising approximation in Eq. (2). This surrogate is exact only when vθ coincides with the conditional vector field (x_t − ε)/(1 − t) along the segment; in general it is a first-order biased estimate of the clean sample and ∇θ log σ(D(x'_0)) is not the gradient of the adversarial objective in Eq. (1). No error bound, bias analysis, or empirical validation (e.g., comparison to full ODE unrolling, sensitivity to Δt) is provided. Because FAIL-PD is one of the two headline algorithms, the claimed low-variance white-box advantage is unverified. Please add a bias analysis or an experiment that measures the discrepancy between Eq. (2) updates and full
  2. [§2.1 (Theoretical Connection)] The assertion that FAIL-PD is 'a low-variance, unbiased limit of DDPG' and that it 'inherits the local convergence properties of GANs' is not supported by any theorem, proposition, or empirical variance measurement. The DDPG analogy is informal: the policy is not learned through a learned critic, and the single-step denoising approximation is not shown to be unbiased. Either provide a formal statement with explicit assumptions and a proof, or soften these claims to qualitative motivation. The current wording overstates what is established.
  3. [§3.1, §4, Tables 1 and 7] The evaluation protocol is inconsistent across the main tables. Section 3.1 states that ablation samples are generated at 512×512 with CFG disabled, while Section 4 says system-level samples use 'optimal CFG' without giving a value. Table 1 reports UniGen scores for FLUX/FAIL-PD of 51.84/63.38, while Table 7 reports 61.30/73.70 for the same names; Appendix C reveals different UniGen splits and different evaluators (Qwen-72B vs Gemini 2.5 Pro API). The abstract's headline improvement 'from 61.61 to 73.70' is therefore not directly comparable to the Table 1 ablation numbers. Please report the exact split, CFG value, number of inference samples, and ideally multiple seeds/confidence intervals for each table, and avoid mixing protocols without explicit labeling.
  4. [§2.2 and Appendix A] The FAIL-PG likelihood ratio r(θ)=exp(L_CFM(θ_old,x)−L_CFM(θ,x)) and the KL penalty in Eq. (4) rely on treating the CFM loss as equal to negative log-likelihood (Eqs. 6–7). Eq. (6) is an ELBO inequality; replacing it with an equality and then using differences of stochastic CFM losses to define advantages and KL terms is an approximation. No evidence is provided that this surrogate is reliable enough for policy-gradient updates, yet the stability claims for FAIL-PG in §3.3 depend on the KL constraint. Please add a formal justification or an empirical validation of this approximation.
minor comments (5)
  1. [References] DPG-Bench is cited as (Silver et al., 2014), but Silver et al. 2014 is the deterministic policy gradient paper, not the DPG-Bench benchmark. The benchmark citation should be corrected to the appropriate source.
  2. [Abstract / §3.1] The abstract uses 'Nano Banana pro' while the text uses 'Gemini Pro 3' and the footnote gives 'Gemini 3 Pro image (nano banana pro)'. Please standardize the expert-model terminology.
  3. [Algorithm 1, line 4] The line 'sample G image noise pairs (x_j0, ε_j)' does not specify whether x_j0 are fresh policy rollouts or expert samples. This is important for the discriminator update and for interpreting the algorithm. Please clarify.
  4. [Table 7 / References] There are typos: 'OpneAI' should be 'OpenAI' in Table 6 and the reference list, and 'exxcessively' appears in Appendix A.1.
  5. [§3.2] The claim that Online DPO can be viewed as 'a special case of FAIL-PG where the discriminator is assumed perfect' is not precise: the binarized reward of DPO has a different functional form and does not use group-normalized advantages. Please rephrase or justify.

Circularity Check

0 steps flagged

No circular derivation: FAIL's objective and gradient estimators are externally grounded; the only self-citation (HPSv3) is an evaluation metric, not a fitted input.

full rationale

The central claim—post-training as adversarial imitation learning—is implemented through Eq. (1) (standard GAN/GAIL objective), Eq. (2) (one-step denoising estimate of x0), and Eqs. (3)–(5) (FPO-style likelihood-ratio and KL constraint). None of these defines the method in terms of the reported benchmarks or of the target expert distribution by construction; the expert images are produced by Gemini, and no parameter is fitted to HPSv3, UniGen, DPG, or UnifiedReward. The single-step approximation in Eq. (2) is a first-order surrogate whose bias is unquantified; that is a correctness/verification limitation (the paper itself says it 'assumes local linearity' and, in Section 6, acknowledges sensitivity and boundedness by pretraining), but it is not a circular reduction. The only self-overlap is HPSv3/HPDv3 (Ma et al., 2025), written by the first author and used for prompt curation and aesthetic evaluation; since HPSv3 is a fixed, externally applicable scorer and FAIL's training signal does not come from it, this is a minor self-citation, not load-bearing. No circular step meets the evidentiary bar.

Axiom & Free-Parameter Ledger

4 free parameters · 3 axioms · 0 invented entities

The method relies on approximations from prior work (ELBO-CFM equivalence) and a new local-linearity assumption for efficient gradients. Most free parameters are standard training choices (learning rates, batch composition), but the lack of sensitivity analyses makes their influence on the reported results uncertain.

free parameters (4)
  • KL coefficient β for FAIL-PG = 0.05
    Chosen to constrain the policy to the reference; no sensitivity analysis provided.
  • Policy and discriminator learning rates (lrθ, lrω) = Varies per method/discriminator (Table 10)
    Independently tuned for each algorithm and discriminator type to ensure stability; could overfit to the chosen benchmarks.
  • Number of policy rollouts per prompt = 3
    Design choice in the batch composition (3 policy rollouts + 1 expert per prompt); no ablation.
  • Discriminator warmup steps = 25
    Chosen to let the discriminator stabilize before policy updates; no sensitivity analysis.
axioms (3)
  • domain assumption log πθ(x) ≈ −L_CFM(θ,x) + C with C independent of θ
    Used in Appendix A to derive KL(πθ||πref) for FAIL-PG. This ELBO-CFM equality is an approximation from Kingma & Gao and FPO; if the constant depends on θ or the ELBO is not tight, the KL penalty is inaccurate.
  • ad hoc to paper Local linearity of the vector field for single-step denoising
    Equation (2) assumes the vector field is locally linear to estimate x'_0 without full ODE integration; no error bound or validation is given.
  • standard math Discriminator provides a dense reward that makes minimization of JSD between policy and expert align the policy
    Basis of GAIL and GAN training; standard assumption in adversarial imitation learning.

pith-pipeline@v1.3.0-alltime-deepseek · 19835 in / 13467 out tokens · 120385 ms · 2026-08-02T23:53:06.038870+00:00 · methodology

0 comments
read the original abstract

Post-training of flow matching models-aligning the output distribution with a high-quality target-is mathematically equivalent to imitation learning. While Supervised Fine-Tuning mimics expert demonstrations effectively, it cannot correct policy drift in unseen states. Preference optimization methods address this but require costly preference pairs or reward modeling. We propose Flow Matching Adversarial Imitation Learning (FAIL), which minimizes policy-expert divergence through adversarial training without explicit rewards or pairwise comparisons. We derive two algorithms: FAIL-PD exploits differentiable ODE solvers for low-variance pathwise gradients, while FAIL-PG provides a black-box alternative for discrete or computationally constrained settings. Fine-tuning FLUX with only 13,000 demonstrations from Nano Banana pro, FAIL achieves competitive performance on prompt following and aesthetic benchmarks. Furthermore, the framework generalizes effectively to discrete image and video generation, and functions as a robust regularizer to mitigate reward hacking in reward-based optimization. Code and data are available at https://github.com/HansPolo113/FAIL.

Figures

Figures reproduced from arXiv: 2602.12155 by Chen Li, Han Hu, Weidi Xie, Xiaosong Zhang, Yeyao Ma.

Figure 1
Figure 1. Figure 1: (a) We propose FAIL, an adversarial imitation learning framework for flow matching model. (b) With 13K limited data, FAIL significantly improved performance of FLUX baseline. ensuring the generation of aesthetically pleasing and prompt￾faithful images. Currently, two primary paradigms govern this alignment phase: Supervised Fine-Tuning (SFT) and Preference Optimization (e.g., RLHF (Ouyang et al., 2022), DP… view at source ↗
Figure 2
Figure 2. Figure 2: Convergence dynamics of FAIL variants. PG converges rapidly but suffers from collapse, PD shows long-term stability. highlights a critical bottleneck: with only 13,000 samples, the learned reward model is too weak to provide a robust direction toward the true high-quality distribution. In contrast, FAIL achieves superior performance. Compar￾ing Online DPO with FAIL-PG, we attribute FAIL’s lead to two prima… view at source ↗
Figure 3
Figure 3. Figure 3: The visualization results of FAIL-PD in different training steps. FAIL-PD show consistence improvement and distribution alignment to the expert demonstrations. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: FAIL-PG convergence rapidly, it improve the overall quality first, then optimize the fine-grained detail. 17 [PITH_FULL_IMAGE:figures/full_fig_p017_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Integrate FAIL with reward model could alleviate the reward hacking phenomenon. 18 [PITH_FULL_IMAGE:figures/full_fig_p018_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Reward Hacking in the Era of Large Models: Mechanisms, Emergent Misalignment, Challenges

    cs.LG 2026-04 unverdicted novelty 5.0

    The paper introduces the Proxy Compression Hypothesis as a unifying framework explaining reward hacking in RLHF as an emergent result of compressing high-dimensional human objectives into proxy reward signals under op...

  2. Proxy Reward Internalization and Mechanistic Exploitation: A Learned Precursor to Reward Hacking and Its Generalization

    cs.AI 2026-06 unverdicted novelty 4.0

    Proxy RL produces a staged proxy-internalization capability that emerges before and predicts reward hacking in coding environments.

Reference graph

Works this paper leans on

37 extracted references · 30 linked inside Pith · cited by 2 Pith papers

  1. [4]

    Janus-pro: Unified multimodal understand- ing and generation with data and model scaling.arXiv preprint arXiv:2501.17811,

    Chen, X., Wu, Z., Liu, X., Pan, Z., Liu, W., Xie, Z., Yu, X., and Ruan, C. Janus-pro: Unified multimodal understand- ing and generation with data and model scaling.arXiv preprint arXiv:2501.17811,

    Pith/arXiv arXiv

  2. [6]

    Emu: Enhancing image generation models us- ing photogenic needles in a haystack.arXiv preprint arXiv:2309.15807,

    Dai, X., Hou, J., Ma, C.-Y ., Tsai, S., Wang, J., Wang, R., Zhang, P., Vandenhende, S., Wang, X., Dubey, A., et al. Emu: Enhancing image generation models us- ing photogenic needles in a haystack.arXiv preprint arXiv:2309.15807,

    Pith/arXiv arXiv

  3. [7]

    Learning robust rewards with adversarial inverse reinforcement learning.arXiv preprint arXiv:1710.11248, 2017a

    Fu, J., Luo, K., and Levine, S. Learning robust rewards with adversarial inverse reinforcement learning.arXiv preprint arXiv:1710.11248, 2017a. 9 FAIL: Flow Matching Adversarial Imitation Learning for Image Generation Fu, J., Luo, K., and Levine, S. Learning robust rewards with adversarial inverse reinforcement learning.arXiv preprint arXiv:1710.11248, 20...

    Pith/arXiv arXiv

  4. [8]

    X-omni: Reinforcement learning makes discrete autoregressive image generative models great again.arXiv preprint arXiv:2507.22058,

    Geng, Z., Wang, Y ., Ma, Y ., Li, C., Rao, Y ., Gu, S., Zhong, Z., Lu, Q., Hu, H., Zhang, X., et al. X-omni: Reinforcement learning makes discrete autoregressive image generative models great again.arXiv preprint arXiv:2507.22058,

    Pith/arXiv arXiv

  5. [9]

    We utilize the AdamW optimizer for both the policy and the discriminator 13 FAIL: Flow Matching Adversarial Imitation Learning for Image Generation Table 9.Global Hyperparameters.These settings are shared across all tasks (Image, Video, Discrete) unless specified otherwise. Hyperparameter Value Optimizer AdamW Optimizer Momentum (β1, β2) (0.9,0.999) Weigh...

    2014

  6. [10]

    K., Dwibedi, D., Levine, S., and Tompson, J

    Kostrikov, I., Agrawal, K. K., Dwibedi, D., Levine, S., and Tompson, J. Discriminator-actor-critic: Addressing sam- ple inefficiency and reward bias in adversarial imitation learning.arXiv preprint arXiv:1809.02925,

    Pith/arXiv arXiv

  7. [12]

    F., Batifol, S., Blattmann, A., Boesel, F., Con- sul, S., Diagne, C., Dockhorn, T., English, J., English, Z., Esser, P., et al

    Labs, B. F., Batifol, S., Blattmann, A., Boesel, F., Con- sul, S., Diagne, C., Dockhorn, T., English, J., English, Z., Esser, P., et al. Flux. 1 kontext: Flow matching for in-context image generation and editing in latent space. arXiv preprint arXiv:2506.15742,

    Pith/arXiv arXiv

  8. [13]

    Playground v2

    Li, D., Kamko, A., Akhgari, E., Sabet, A., Xu, L., and Doshi, S. Playground v2. 5: Three insights towards enhancing aesthetic quality in text-to-image generation. arXiv preprint arXiv:2402.17245, 2024a. Li, Z., Zhang, J., Lin, Q., Xiong, J., Long, Y ., Deng, X., Zhang, Y ., Liu, X., Huang, M., Xiao, Z., et al. Hunyuan- dit: A powerful multi-resolution dif...

    Pith/arXiv arXiv

  9. [15]

    Flow-grpo: Training flow matching models via online rl.arXiv preprint arXiv:2505.05470,

    Liu, J., Liu, G., Liang, J., Li, Y ., Liu, J., Wang, X., Wan, P., Zhang, D., and Ouyang, W. Flow-grpo: Training flow matching models via online rl.arXiv preprint arXiv:2505.05470,

    Pith/arXiv arXiv

  10. [16]

    Flow straight and fast: Learning to generate and transfer data with rectified flow

    Liu, X., Gong, C., and Liu, Q. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03003,

    Pith/arXiv arXiv

  11. [17]

    and Hutter, F

    Loshchilov, I. and Hutter, F. Decoupled weight decay regu- larization.arXiv preprint arXiv:1711.05101,

    Pith/arXiv arXiv

  12. [18]

    10 FAIL: Flow Matching Adversarial Imitation Learning for Image Generation Mao, W., Chen, H., Yang, Z., and Shou, M

    URL https: //arxiv.org/abs/2508.03789. 10 FAIL: Flow Matching Adversarial Imitation Learning for Image Generation Mao, W., Chen, H., Yang, Z., and Shou, M. Z. The im- age as its own reward: Reinforcement learning with ad- versarial reward for image generation.arXiv preprint arXiv:2511.20256,

    Pith/arXiv arXiv

  13. [19]

    M., Weber, E., Choi, H., Feng, H., and Kanazawa, A

    McAllister, D., Ge, S., Yi, B., Kim, C. M., Weber, E., Choi, H., Feng, H., and Kanazawa, A. Flow matching policy gradients.arXiv preprint arXiv:2507.21053,

    Pith/arXiv arXiv

  14. [20]

    Oquab, M., Darcet, T., Moutakanni, T., V o, H., Szafraniec, M., Khalidov, V ., Fernandez, P., Haziza, D., Massa, F., El- Nouby, A., et al

    URL https://openai.com/index/ introducing-4o-image-generation/. Oquab, M., Darcet, T., Moutakanni, T., V o, H., Szafraniec, M., Khalidov, V ., Fernandez, P., Haziza, D., Massa, F., El- Nouby, A., et al. Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193,

    Pith/arXiv arXiv

  15. [21]

    S., Bagnell, J

    Ren, J., Swamy, G., Wu, Z. S., Bagnell, J. A., and Choud- hury, S. Hybrid inverse reinforcement learning.arXiv preprint arXiv:2402.08848,

    Pith/arXiv arXiv

  16. [23]

    Proximal policy optimization algorithms

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347,

    Pith/arXiv arXiv

  17. [25]

    Directly aligning the full diffusion trajectory with fine-grained human preference

    Shen, X., Li, Z., Yang, Z., Zhang, S., Zhang, Y ., Li, D., Wang, C., Lu, Q., and Tang, Y . Directly aligning the full diffusion trajectory with fine-grained human preference. arXiv preprint arXiv:2509.06942,

    Pith/arXiv arXiv

  18. [26]

    Song, J., Meng, C., and Ermon, S

    URLhttps://arxiv.org/abs/2508.10104. Song, J., Meng, C., and Ermon, S. Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502,

    Pith/arXiv arXiv 2010

  19. [28]

    Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314,

    11 FAIL: Flow Matching Adversarial Imitation Learning for Image Generation Wan, T., Wang, A., Ai, B., Wen, B., Mao, C., Xie, C.-W., Chen, D., Yu, F., Zhao, H., Yang, J., Zeng, J., Wang, J., Zhang, J., Zhou, J., Wang, J., Chen, J., Zhu, K., Zhao, K., Yan, K., Huang, L., Feng, M., Zhang, N., Li, P., Wu, P., Chu, R., Feng, R., Zhang, S., Sun, S., Fang, T., W...

    Pith/arXiv arXiv

  20. [29]

    Emu3: Next-token prediction is all you need.arXiv preprint arXiv:2409.18869,

    Wang, X., Zhang, X., Luo, Z., Sun, Q., Cui, Y ., Wang, J., Zhang, F., Wang, Y ., Li, Z., Yu, Q., et al. Emu3: Next-token prediction is all you need.arXiv preprint arXiv:2409.18869,

    Pith/arXiv arXiv

  21. [30]

    Pref-grpo: Pairwise preference reward-based grpo for stable text-to-image reinforcement learning.arXiv preprint arXiv:2508.20751, 2025a

    Wang, Y ., Li, Z., Zang, Y ., Zhou, Y ., Bu, J., Wang, C., Lu, Q., Jin, C., and Wang, J. Pref-grpo: Pairwise preference reward-based grpo for stable text-to-image reinforcement learning.arXiv preprint arXiv:2508.20751, 2025a. Wang, Y ., Zang, Y ., Li, H., Jin, C., and Wang, J. Unified re- ward model for multimodal understanding and generation. arXiv prepr...

    Pith/arXiv arXiv

  22. [31]

    Janus: Decoupling visual encoding for unified multimodal understanding and generation

    Wu, C., Chen, X., Wu, Z., Ma, Y ., Liu, X., Pan, Z., Liu, W., Xie, Z., Yu, X., Ruan, C., et al. Janus: Decoupling visual encoding for unified multimodal understanding and generation. InProceedings of the Computer Vision and Pattern Recognition Conference, pp. 12966–12977, 2025a. Wu, C., Li, J., Zhou, J., Lin, J., Gao, K., Yan, K., Yin, S.-m., Bai, S., Xu,...

    Pith/arXiv arXiv

  23. [32]

    Tackling the generative learning trilemma with denoising diffusion gans.arXiv preprint arXiv:2112.07804,

    Xiao, Z., Kreis, K., and Vahdat, A. Tackling the generative learning trilemma with denoising diffusion gans.arXiv preprint arXiv:2112.07804,

    Pith/arXiv arXiv

  24. [33]

    Ad- vantage weighted matching: Aligning rl with pretraining in diffusion models.arXiv preprint arXiv:2509.25050, 2025a

    Xue, S., Ge, C., Zhang, S., Li, Y ., and Ma, Z.-M. Ad- vantage weighted matching: Aligning rl with pretraining in diffusion models.arXiv preprint arXiv:2509.25050, 2025a. Xue, Z., Wu, J., Gao, Y ., Kong, F., Zhu, L., Chen, M., Liu, Z., Liu, W., Guo, Q., Huang, W., et al. Dancegrpo: Unleashing grpo on visual generation.arXiv preprint arXiv:2505.07818, 2025...

    arXiv

  25. [34]

    Therefore, we approximate the log-probability of a samplexunder policyπ θ as: logπθ(x)≈ −LCF M(θ, x) +C.(7) A.2

    and utilized in recent reinforcement learning frameworks (McAl- lister et al., 2025; Xue et al., 2025a) for diffusion, the weighted flow matching lossLCF M(θ, x)is mathematically equivalent to the negative ELBO of the data, up to a constant C that only depends on the noise schedule and data distribution, but not on the learnable parametersθ: logπθ(x)≥ELBO...

    2025

  26. [36]

    For inference, we generate 4 images per prompt for UniGen and DPG, and 1 image per prompt for Alchemist

    to measure aesthetic quality and UnifiedReward (Wang et al., 2025b) (specifically theUnifiedReward-2.0-qwen3vl-8bvariant) to assess semantic alignment and coherence. For inference, we generate 4 images per prompt for UniGen and DPG, and 1 image per prompt for Alchemist. All samples are generated with 28 steps at512×512resolution with Classifier-Free Guida...

    2025

  27. [37]

    Subsequently, from step 100 to 400, the optimization shifts focus to refining low-level details, resulting in significant improvements in text rendering and fine-grained textures

    We observe distinct phases of improvement: between steps 0 and 100, the model achieves a rapid boost in overall visual quality and semantic alignment. Subsequently, from step 100 to 400, the optimization shifts focus to refining low-level details, resulting in significant improvements in text rendering and fine-grained textures. D.2. Visualization of Rewa...

    2025

  28. [2004]

    Qwen3-vl technical report

    Bai, S., Cai, Y ., Chen, R., Chen, K., Chen, X., Cheng, Z., Deng, L., Ding, W., Gao, C., Ge, C., Ge, W., Guo, Z., Huang, Q., Huang, J., Huang, F., Hui, B., Jiang, S., Li, Z., Li, M., Li, M., Li, K., Lin, Z., Lin, J., Liu, X., Liu, J., Liu, C., Liu, Y ., Liu, D., Liu, S., Lu, D., Luo, R., Lv, C., Men, R., Meng, L., Ren, X., Ren, X., Song, S., Sun, Y ., Tan...

    Pith/arXiv arXiv

  29. [2011]

    Fast high-resolution image synthe- sis with latent adversarial diffusion distillation

    Sauer, A., Boesel, F., Dockhorn, T., Blattmann, A., Esser, P., and Rombach, R. Fast high-resolution image synthe- sis with latent adversarial diffusion distillation. InSIG- GRAPH Asia 2024 Conference Papers, pp. 1–11, 2024a. Sauer, A., Lorenz, D., Blattmann, A., and Rombach, R. Ad- versarial diffusion distillation. InEuropean Conference on Computer Vision...

    2024

  30. [2015]

    T., Ben-Hamu, H., Nickel, M., and Le, M

    Lipman, Y ., Chen, R. T., Ben-Hamu, H., Nickel, M., and Le, M. Flow matching for generative modeling.arXiv preprint arXiv:2210.02747,

    Pith/arXiv arXiv

  31. [2016]

    and Salimans, T

    Ho, J. and Salimans, T. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598,

    Pith/arXiv arXiv

  32. [2017]

    Deepseekmath: Push- ing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300,

    Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y ., Wu, Y ., et al. Deepseekmath: Push- ing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300,

    Pith/arXiv arXiv

  33. [2020]

    Alchemist: Turning public text-to-image data into generative gold.arXiv preprint arXiv:2505.19297,

    Startsev, V ., Ustyuzhanin, A., Kirillov, A., Baranchuk, D., and Kastryulin, S. Alchemist: Turning public text-to-image data into generative gold.arXiv preprint arXiv:2505.19297,

    arXiv

  34. [2022]

    The gan landscape: Losses, architectures, regularization, and normalization.arXiv preprint arXiv:1807.04720,

    Kurach, K., Lucic, M., Zhai, X., Michalski, M., and Gelly, S. The gan landscape: Losses, architectures, regularization, and normalization.arXiv preprint arXiv:1807.04720,

    Pith/arXiv arXiv

  35. [2023]

    Z-image: An efficient image generation foundation model with single-stream diffusion transformer.arXiv preprint arXiv:2511.22699,

    Cai, H., Cao, S., Du, R., Gao, P., Hoi, S., Hou, Z., Huang, S., Jiang, D., Jin, X., Li, L., et al. Z-image: An efficient image generation foundation model with single-stream diffusion transformer.arXiv preprint arXiv:2511.22699,

    Pith/arXiv arXiv

  36. [2024]

    Training diffusion models with reinforcement learning

    Black, K., Janner, M., Du, Y ., Kostrikov, I., and Levine, S. Training diffusion models with reinforcement learning. arXiv preprint arXiv:2305.13301,

    Pith/arXiv arXiv

  37. [2025]

    Clark, K., Vicol, P., Swersky, K., and Fleet, D. J. Directly fine-tuning diffusion models on differentiable rewards. arXiv preprint arXiv:2309.17400,

    Pith/arXiv arXiv