Pith. sign in

REVIEW 4 major objections 4 minor 97 references

This paper claims that GRPO training for diffusion image models can replace most per-step backward passes with a two-endpoint Jacobian interpolation, roughly halving backward cost while preserving training quality.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 17:34 UTC pith:D33CMHNP

load-bearing objection Useful, mathematically clean acceleration for diffusion GRPO, but the empirical claim of 'negligible quality degradation' is not yet supported by the experiments as run. the 4 major comments →

arxiv 2607.17572 v3 pith:D33CMHNP submitted 2026-07-20 cs.LG cs.CVcs.SYeess.SY

JAGG: Jacobian-Aggregated Group Gradient for Efficient GRPO Training of Diffusion Models

Ruiyi Ding , Jie Li , He Kang , Ziyan Liu , Chengru Song , Yuan cheng This is my paper
classification cs.LG cs.CVcs.SYeess.SY
keywords GRPOreinforcement learningdiffusion modelsflow matchingbackpropagation efficiencyJacobian interpolationtext-to-image generationtrajectory linearity
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Group Relative Policy Optimization (GRPO) is a reinforcement-learning recipe for aligning generative models, but applying it to diffusion and flow-matching image generators is expensive: the policy gradient must be back-propagated through the full transformer at every denoising step. This paper claims that this cost can be cut dramatically because, during the high-noise part of denoising, the Jacobian of the velocity field with respect to model parameters is nearly linear in the state and time. That lets JAGG replace every intermediate-step Jacobian with a t-weighted interpolation of the two endpoint Jacobians, shrinking a group of W transformer backward passes to a single two-endpoint backward. The paper proves the interpolation is exact when the velocity is exactly linear, and measures on two text-to-image backbones that the direction agreement between JAGG and exact gradients stays above 0.5 for the first 60% of denoising groups. The reported payoff is a 17-18% reduction in end-to-end training-step time (about 30% on the update stage) with reward curves that track the full-cost baseline.

Core claim

The central claim, stated on the paper's own terms, is that the expensive per-step backward passes of diffusion GRPO can be faithfully compressed by exploiting trajectory smoothness. For a window of W consecutive denoising steps, JAGG runs all W forward passes but tracks gradients only at the two endpoints, computes the upstream gradient signals for each step through the lightweight SDE/PPO objective, and aggregates them into two composite signals via the timestep-normalized weights alpha_j=(t0-t_j)/(t0-t_{W-1}). A single joint backward pass through the two endpoint outputs then yields the parameter gradient. Theorem 1 proves that if the velocity field v_theta(z,t) is linear in (z,t) and the

What carries the argument

The load-bearing object is the velocity-Jacobian interpolation identity J_j = (1-alpha_j)J_0 + alpha_j J_{W-1}, where J_j = partial v_theta(z_j,t_j)/partial theta and alpha_j=(t0-t_j)/(t0-t_{W-1}). It turns W upstream gradient vectors s_j into two aggregates s_base = sum (1-alpha_j)s_j and s_corr = sum alpha_j s_j, so the full group gradient equals J_0^T s_base + J_{W-1}^T s_corr. This reduces the computation to two endpoint backward passes, and because each transformer backward is roughly twice a forward and triggers gradient synchronization under data parallelism, the W:2 reduction translates into wall-clock savings larger than raw FLOP counts suggest.

Load-bearing premise

The load-bearing premise is that the network's output Jacobian changes almost linearly as denoising progresses in the high-noise groups; the paper's own measurements show this fails badly in the final low-noise groups, where the approximate gradient points at only 0.22 direction agreement with the true gradient and is over 4x too large.

What would settle it

Run the paper's gradient-quality probe on a new backbone: compute JAGG's aggregated gradient and the exact W-step gradient for the first, highest-noise group; if cosine similarity is below 0.5 or the norm ratio departs far from 1, the linearity premise fails and the speedup trades away training fidelity. A cleaner test is to train the same model with jagg_frac=1.0 (JAGG everywhere) and compare final reward to exact GRPO; divergence on a fixed prompt suite would falsify the claim that the interpolation faithfully reproduces the gradient.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If correct, standard GRPO training on high-resolution text-to-image models becomes roughly 17-18% cheaper per training step, with the policy-update stage itself about 30% cheaper.
  • Because the backward cost no longer grows with W in the JAGG region, the same compute budget can support larger gradient windows W; the paper notes that larger W yields proportionally greater gains.
  • The PPO clipping behavior is inherited automatically: clipped steps produce zero upstream signal, so they contribute nothing to the aggregated backward, exactly matching the effect of clipping per step.
  • The cosine-similarity measurement used to set jagg_frac gives a model-agnostic recipe for deciding where JAGG is safe on any new backbone.
  • In the second RL recipe tested, JAGG kept improving after the exact baseline's metrics began to decline, suggesting the aggregation can act as a gradient smoother rather than merely a cheaper substitute.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • An editor's reading: the paper leaves jagg_frac fixed at 0.6, but its own per-group cosine measurements suggest a confidence-based router could push the same budget further by adapting group by group rather than by a fixed fraction.
  • A natural testable extension is to apply the two-endpoint Jacobian aggregation to any objective that back-propagates through a denoising trajectory, such as distillation or preference optimization, provided the same linearity check passes.
  • The paper does not analyze how approximation error scales with W beyond the reported W=4 experiments; an editor would expect quality degradation to grow with window width, with the low-noise cosine collapse hinting at where the ceiling lies.
  • The consistently weaker endpoint-only ablation suggests part of JAGG's benefit may be variance reduction from interpolating per-step signals, which could be isolated by comparing JAGG against a Monte-Carlo estimate of the group gradient using the same two-backprop budget.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes JAGG, a method to accelerate GRPO training of diffusion/flow-matching models by approximating intermediate-timestep velocity Jacobians via t-weighted interpolation of the two endpoint Jacobians. This reduces the number of full transformer backward passes from W to 2 per group of W consecutive denoising steps. The authors prove that the interpolation is exact when the velocity is linear in (z,t), and they introduce a cosine-similarity routing rule (jagg_frac) to deploy JAGG only in the high-noise groups where the approximation is empirically accurate. Experiments on QwenImage and Flux under DanceGRPO and Flow-GRPO-Fast report wall-clock speedups of 14–18% end-to-end and ~30% on the update stage, with quality that is claimed to be comparable to the exact per-step GRPO baseline.

Significance. If the empirical claims hold, JAGG is a practically useful plug-in for diffusion GRPO training: it attacks a real bottleneck (per-step transformer backward passes), the interpolation idea is well-motivated by prior inference-acceleration work, and the theoretical result is a clean exactness statement under a clearly stated assumption. The paper also ships code, includes an ablation isolating the interpolation component, and provides unusually detailed hyperparameter and hardware reporting. The main gap is that the central quality claim rests on single-seed comparisons and on a routing hyperparameter calibrated from the same gradient-quality measurements used to validate the method. With additional statistical evidence and a less circular routing analysis, this could be a solid systems/RL contribution; in its current form the evidence for 'negligible quality degradation' is not yet conclusive.

major comments (4)
  1. [Experiments; Tables 1–3, Figures 6–9; Appendix: Experimental Setup] All training runs use a single seed: the appendix states 'All runs use seed 42...'. The central headline 'negligible quality degradation' therefore has no statistical support. Checkpoint-level differences such as Table 3 step 80 (JAGG HPSv2.1 0.3448 vs baseline 0.3241, CLIPScore 0.2729 vs 0.2619) could be partly noise, and step 50 shows the baseline ahead. Because the two methods share initial noise, the authors should report at least three seeds with mean±std, or matched-pair differences and a confidence interval, for all metrics and checkpoints. This is load-bearing for the quality claim.
  2. [Empirical Gradient Quality and Choice of jagg_frac (Table 5); Analysis] The routing threshold cos≥0.5 is applied to the same Table 5 measurements that are used to support the linearity assumption: jagg_frac=0.6 is selected because groups 0–2 have mean cosine 0.85/0.77/0.74 (Flux) and 0.79/0.64/0.58 (QwenImage). This makes the validation set the calibration set. Moreover, group 2 QwenImage has mean cos 0.58±0.26, so a substantial fraction of individual rollouts fall below the 0.5 threshold, and group 3 Flux has mean 0.63±0.13. The paper should report the per-group distribution (e.g., fraction of rollouts satisfying cos≥0.5), use held-out rollouts/checkpoints for calibration, and include a jagg_frac sweep against final quality. Without this, the routing rule and the resulting quality are not independent evidence.
  3. [Theoretical Analysis, Eq. (8) and Corollary; Table 5] Theorem 1 and its corollary give exact equality of the group gradient only under global linearity of the velocity in (z,t). For real models, the only evidence is the empirical cosine and norm-ratio table; there is no quantitative perturbation bound of the form ‖∇θL_JAGG − ∇θL_exact‖ ≤ C·ε, where ε measures nonlinearity or trajectory curvature. Consequently, the step from 'approximate linearity' to 'negligible quality degradation' is not established. Please add a perturbation bound (e.g., in terms of second derivatives of vθ or Lipschitz constants), or explicitly reframe the claim as a heuristic validated by controlled experiments and give a safety analysis across training checkpoints.
  4. [Abstract; Method; Efficiency Analysis] With jagg_frac=0.6 and W=4, the global reduction in transformer backward passes is not 2×. For a 20-step trajectory split into 5 groups, JAGG uses 3·2 + 2·4 = 14 backward passes versus 20 for the baseline, i.e. a 1.43× reduction overall. The '~2× backward speedup' is the per-JAGG-group W:2 rate, not the achieved end-to-end rate once routing is applied. State the global backward-pass count and the per-group qualification; as written, the abstract's '~2× backward speedup' overstates the deployed efficiency.
minor comments (4)
  1. [Flow-GRPO-Fast experiments] Typo: 'accross' should be 'across' in the sentence 'To validate the robustness of our method accross different RL Algorithms...'.
  2. [Table 4 and Appendix: Experimental Setup] The appendix text says 'gradient clipping at 1×10−3', but Table 4 lists max_grad_norm=1.0 for QwenImage and 1×10−3 for Flux. Please clarify which value applies to each model and whether this affects the comparison.
  3. [Eq. (4) and Eq. (5)] The GRPO loss in Eq. (4) is written as an average over W steps with an advantage A_i that does not depend on j, but the per-step quantity L_j used later is not defined explicitly. Please define L_j = max(...) so the relation between s_j, L_j, and Eq. (4) is unambiguous.
  4. [Eq. (5) and SDE formulation] Eq. (5) uses a Gaussian transition with variance σ_t^2, but the SDEs in Eqs. (1)–(2) include stochasticity controlled by ε_t. The mapping between ε_t, the SDE coefficients, and the variance used in the log-probability should be stated explicitly, especially for mixed ODE/SDE schedules.

Circularity Check

0 steps flagged

No significant circularity: the central derivation is a conditional theorem plus external benchmarks; only minor within-method calibration and one non-load-bearing self-citation.

full rationale

JAGG's claimed derivation chain is self-contained. Theorem 1 assumes the velocity field is linear in (z,t) and proves, by taking the θ-derivative of the first-order Taylor expansion, that the Jacobian at an intermediate step is the α-weighted interpolation of the endpoint Jacobians (Eq. 8); the Corollary regroups this identity to show the two-endpoint backward returns the exact group gradient. This is a conditional algebraic equivalence, not a prediction derived from its own conclusion. The empirical support for the linearity premise is Table 5, which directly measures cos(ĝ_JAGG, g_exact) and norm ratios against the exact per-step gradient on real rollouts. The paper then sets jagg_frac by an explicit rule (cos ≥ 0.5) using those same measurements; this is a within-method calibration that shares data with the validation of the assumption, but it does not force the headline result, because final quality is evaluated on independent HPSv2.1/PickScore/CLIPScore benchmarks and the training-quality numbers are not used to fit any JAGG parameter. The only self-citation is MixGRPO (Li et al. 2026), cited as an analogy for why interpolation might smooth updates; it is not load-bearing for any theorem or result. Single-seed training comparisons are a statistical robustness concern, not evidence of circularity. Overall, no load-bearing step reduces to its own inputs.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

The central claim rests on the approximate linearity of velocity Jacobians along the denoising path. The paper sets a single routing parameter (jagg_frac) from its own grad-quality measurements, and relies on a detached-proxy objective inherited from GRPO practice. No new physical entities are introduced.

free parameters (3)
  • jagg_frac = 0.6 (T2I), 0.4 (Flow-GRPO-Fast)
    Fraction of denoising groups routed to JAGG; chosen from Table 5 cosine-similarity measurements on the same models, with threshold cos>=0.5.
  • cosine threshold = 0.5
    Threshold used to decide which groups can use JAGG; a design choice from Bottou et al. sufficient-descent argument, not independently validated.
  • group size W (sum_sample) = 4
    Window length; affects speedup magnitude (larger W gives larger speedup but coarser approximation). Chosen as default; not swept.
axioms (4)
  • domain assumption Velocity field v_theta(z,t) is approximately linear in (z,t) in the high-noise denoising regime
    Assumed in the Method section and empirically checked in Table 5; degrades at low noise.
  • domain assumption The rollout path (z_j,t_j) lies close to an affine interpolation between endpoints in high-noise groups
    Needed to satisfy the equidistant-path condition of Theorem 1; argued via the rectified-flow affine forward marginal but not proven.
  • domain assumption The per-step REINFORCE loss with detached z_j is a valid surrogate for the GRPO objective
    The method computes upstream gradients s_j through a proxy v_tilde_j, ignoring cross-step dependencies; the paper says this matches the exact baseline in their ablation, so it is inherited from prior GRPO practice rather than introduced here.
  • standard math Standard calculus/Taylor expansion apply (v is differentiable and second-order terms vanish)
    Used in the proof of Theorem 1.

pith-pipeline@v1.3.0-alltime-deepseek · 19691 in / 11585 out tokens · 88387 ms · 2026-08-01T17:34:57.709417+00:00 · methodology

0 comments
read the original abstract

Group Relative Policy Optimization (GRPO) is a powerful reinforcement learning algorithm for aligning generative models with human preferences. While successful in large language models~\cite{shao2024deepseekmathpushinglimitsmathematical}, its extension to diffusion and flow matching models introduces a severe computational bottleneck: gradients must be back-propagated through the high-capacity DiT backbone at \emph{every} timestep of the sampling trajectory, making high-resolution text-to-image (T2I) training prohibitively expensive. Training-free DiT inference acceleration methods (e.g., $\Delta$-DiT, ScalingCache) exploit the fact that DiT hidden states and velocity predictions vary \emph{smoothly and nearly linearly} along the trajectory. We ask whether the same linearity can reduce the backward-pass cost of DiT RL training, and answer affirmatively with \textbf{JAGG} (\textbf{J}acobian-\textbf{A}ggregated \textbf{G}roup \textbf{G}radient), which reduces full transformer backward passes from $W$ to $2$ per group of $W$ consecutive steps. JAGG approximates intermediate-step Jacobians via $t$-weighted interpolation of the endpoint Jacobians, then aggregates per-step upstream signals into two composite gradients applied through a single joint backward pass. We prove this interpolation is \emph{exact} when the velocity is linear in $(z,t)$, and a cosine-similarity routing rule (\texttt{jagg\_frac}) deploys JAGG only where the assumption holds. Experiments on T2I benchmarks show JAGG delivers $\sim$2$\times$ backward speedup with negligible quality degradation. The code for this work can be accessed through https://github.com/SchumiDing/JAGG.

Figures

Figures reproduced from arXiv: 2607.17572 by Chengru Song, He Kang, Jie Li, Ruiyi Ding, Yuan Cheng, Ziyan Liu.

Figure 1
Figure 1. Figure 1: Overview of JAGG. A group of W = 4 consec￾utive denoising steps are forward-passed through the DiT. Only the two endpoint steps (colored) maintain gradient graphs; intermediate steps use no_grad. Upstream gradi￾ents sj = ∂Lj/∂vˆj are computed cheaply via the lightweight SDE/PPO objective and then aggregated into ss and se us￾ing timestep-normalized weights α. Finally, a single joint backward pass from the … view at source ↗
Figure 2
Figure 2. Figure 2: QwenImage trained on mixed HPSv2.1+CLIPScore rewards. Mean of HPSv2.1, PickScore, and CLIPScore on the evaluation set vs. training steps for Baseline and JAGG. Per-metric numerical results at each checkpoint are reported in [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: QwenImage validate reward curve trained us [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Flow-GRPO-Fast + JAGG: Left: end-to-end mean training step time (seconds). Right: per-step update time (forward on stored experience + backward), which is the stage JAGG directly targets. Percentages above the bars indicate the wall-clock speedup of JAGG over the baseline. The end-to-end wall-clock numbers for Flow-GRPO-Fast in [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: QwenImage HPSv2.1-only smoke test. HPSv2.1 [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: QwenImage training with mixed HPSv2.1+CLIPScore rewards. Average reward score vs. training steps for Baseline (blue), JAGG (orange), and the ablation variant when available [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Flux DanceGRPO training with mixed HPSv2.1+CLIPScore rewards. Reward curve for Baseline, JAGG, and the ablation setting where available [PITH_FULL_IMAGE:figures/full_fig_p010_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Left: Flux1.dev and Right: QwenImage reward curve trained using Flow-GRPO-Fast algorithm under differ￾ent settings (w JAGG, w/o JAGG, Ablation) [PITH_FULL_IMAGE:figures/full_fig_p010_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

97 extracted references · 1 canonical work pages

  1. [1]

    2024 , eprint=

    DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models , author=. 2024 , eprint=

    2024

  2. [2]

    2026 , eprint=

    LongR: Unleashing Long-Context Reasoning via Reinforcement Learning with Dense Utility Rewards , author=. 2026 , eprint=

    2026

  3. [3]

    2026 , eprint=

    Beyond SFT-to-RL: Pre-alignment via Black-Box On-Policy Distillation for Multimodal RL , author=. 2026 , eprint=

    2026

  4. [4]

    2025 , eprint=

    DanceGRPO: Unleashing GRPO on Visual Generation , author=. 2025 , eprint=

    2025

  5. [5]

    2025 , eprint=

    Flow-GRPO: Training Flow Matching Models via Online RL , author=. 2025 , eprint=

    2025

  6. [6]

    2020 , eprint=

    Denoising Diffusion Probabilistic Models , author=. 2020 , eprint=

    2020

  7. [7]

    2022 , eprint=

    Denoising Diffusion Implicit Models , author=. 2022 , eprint=

    2022

  8. [8]

    2022 , eprint=

    Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow , author=. 2022 , eprint=

    2022

  9. [9]

    2024 , eprint=

    Diffusion Models for Reinforcement Learning: A Survey , author=. 2024 , eprint=

    2024

  10. [10]

    Enhancing Deep Reinforcement Learning: A Tutorial on Generative Diffusion Models in Network Optimization , year=

    Du, Hongyang and Zhang, Ruichen and Liu, Yinqiu and Wang, Jiacheng and Lin, Yijing and Li, Zonghang and Niyato, Dusit and Kang, Jiawen and Xiong, Zehui and Cui, Shuguang and Ai, Bo and Zhou, Haibo and Kim, Dong In , journal=. Enhancing Deep Reinforcement Learning: A Tutorial on Generative Diffusion Models in Network Optimization , year=

  11. [11]

    2024 , eprint=

    Training Diffusion Models with Reinforcement Learning , author=. 2024 , eprint=

    2024

  12. [12]

    DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Models , url =

    Fan, Ying and Watkins, Olivia and Du, Yuqing and Liu, Hao and Ryu, Moonkyung and Boutilier, Craig and Abbeel, Pieter and Ghavamzadeh, Mohammad and Lee, Kangwook and Lee, Kimin , booktitle =. DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Models , url =

  13. [13]

    2025 , eprint=

    DiffPO: Diffusion-styled Preference Optimization for Efficient Inference-Time Alignment of Large Language Models , author=. 2025 , eprint=

    2025

  14. [14]

    DiffPPO: Reinforcement Learning Fine-Tuning of Diffusion Models for Text-to-Image Generation , year=

    Xiao, Zhuo and Jiang, Ping and Zhou, Mingjie and Zhang, Junzi and Huang, Zijian and Yang, Jianfeng , booktitle=. DiffPPO: Reinforcement Learning Fine-Tuning of Diffusion Models for Text-to-Image Generation , year=

  15. [15]

    Computer Vision – ECCV 2024: 18th European Conference, Milan, Italy, September 29–October 4, 2024, Proceedings, Part XXV , pages =

    Chen, Chaofeng and Wang, Annan and Wu, Haoning and Liao, Liang and Sun, Wenxiu and Yan, Qiong and Lin, Weisi , title =. Computer Vision – ECCV 2024: 18th European Conference, Milan, Italy, September 29–October 4, 2024, Proceedings, Part XXV , pages =. 2024 , isbn =. doi:10.1007/978-3-031-72698-9_11 , abstract =

    work page doi:10.1007/978-3-031-72698-9_11 2024

  16. [16]

    Training Diffusion Models Towards Diverse Image Generation with Reinforcement Learning , year=

    Miao, Zichen and Wang, Jiang and Wang, Ze and Yang, Zhengyuan and Wang, Lijuan and Qiu, Qiang and Liu, Zicheng , booktitle=. Training Diffusion Models Towards Diverse Image Generation with Reinforcement Learning , year=

  17. [17]

    2025 , eprint=

    A Survey of Reinforcement Learning for Large Reasoning Models , author=. 2025 , eprint=

    2025

  18. [18]

    2023 , eprint=

    Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation , author=. 2023 , eprint=

    2023

  19. [19]

    2026 , eprint=

    UniGRPO: Unified Policy Optimization for Reasoning-Driven Visual Generation , author=. 2026 , eprint=

    2026

  20. [20]

    2025 , eprint=

    BranchGRPO: Stable and Efficient GRPO with Structured Branching in Diffusion Models , author=. 2025 , eprint=

    2025

  21. [21]

    2026 , eprint=

    MixGRPO: Unlocking Flow-based GRPO Efficiency with Mixed ODE-SDE , author=. 2026 , eprint=

    2026

  22. [22]

    2026 , eprint=

    PCPO: Proportionate Credit Policy Optimization for Aligning Image Generation Models , author=. 2026 , eprint=

    2026

  23. [23]

    2025 , eprint=

    GRPO-Guard: Mitigating Implicit Over-Optimization in Flow Matching via Regulated Clipping , author=. 2025 , eprint=

    2025

  24. [24]

    2026 , eprint=

    Reinforcement Learning Meets Masked Generative Models: Mask-GRPO for Text-to-Image Generation , author=. 2026 , eprint=

    2026

  25. [25]

    2025 , eprint=

    AR-GRPO: Training Autoregressive Image Generation Models via Reinforcement Learning , author=. 2025 , eprint=

    2025

  26. [26]

    2026 , eprint=

    UDM-GRPO: Stable and Efficient Group Relative Policy Optimization for Uniform Discrete Diffusion Models , author=. 2026 , eprint=

    2026

  27. [27]

    2025 , eprint=

    Consolidating Reinforcement Learning for Multimodal Discrete Diffusion Models , author=. 2025 , eprint=

    2025

  28. [28]

    2026 , eprint=

    V-GRPO: Online Reinforcement Learning for Denoising Generative Models Is Easier than You Think , author=. 2026 , eprint=

    2026

  29. [29]

    2026 , eprint=

    DiffusionNFT: Online Diffusion Reinforcement with Forward Process , author=. 2026 , eprint=

    2026

  30. [30]

    2026 , eprint=

    Rethinking the Design Space of Reinforcement Learning for Diffusion Models: On the Importance of Likelihood Estimation Beyond Loss Design , author=. 2026 , eprint=

    2026

  31. [31]

    2026 , eprint=

    Manifold-Aware Exploration for Reinforcement Learning in Video Generation , author=. 2026 , eprint=

    2026

  32. [32]

    2026 , eprint=

    Stabilizing Reinforcement Learning for Diffusion Language Models , author=. 2026 , eprint=

    2026

  33. [33]

    2026 , eprint=

    Improving Reasoning for Diffusion Language Models via Group Diffusion Policy Optimization , author=. 2026 , eprint=

    2026

  34. [34]

    2025 , eprint=

    Inpainting-Guided Policy Optimization for Diffusion Large Language Models , author=. 2025 , eprint=

    2025

  35. [35]

    2023 , eprint=

    Building Normalizing Flows with Stochastic Interpolants , author=. 2023 , eprint=

    2023

  36. [36]

    2025 , eprint=

    Stochastic Interpolants: A Unifying Framework for Flows and Diffusions , author=. 2025 , eprint=

    2025

  37. [37]

    2023 , eprint=

    Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis , author=. 2023 , eprint=

    2023

  38. [38]

    2023 , eprint=

    Testing the General Deductive Reasoning Capacity of Large Language Models Using OOD Examples , author=. 2023 , eprint=

    2023

  39. [39]

    2022 , eprint=

    CLIPScore: A Reference-free Evaluation Metric for Image Captioning , author=. 2022 , eprint=

    2022

  40. [40]

    2025 , eprint=

    FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space , author=. 2025 , eprint=

    2025

  41. [41]

    2024 , howpublished=

    Black Forest Labs , title=. 2024 , howpublished=

    2024

  42. [42]

    2025 , eprint=

    Qwen-Image Technical Report , author=. 2025 , eprint=

    2025

  43. [43]

    2025 , eprint=

    HunyuanVideo: A Systematic Framework For Large Video Generative Models , author=. 2025 , eprint=

    2025

  44. [44]

    2018 , eprint=

    Optimization Methods for Large-Scale Machine Learning , author=. 2018 , eprint=

    2018

  45. [45]

    2026 , eprint=

    Rewards as Labels: Revisiting RLVR from a Classification Perspective , author=. 2026 , eprint=

    2026

  46. [46]

    2025 , eprint=

    Improving Video Generation with Human Feedback , author=. 2025 , eprint=

    2025

  47. [47]

    2025 , eprint=

    From Reusing to Forecasting: Accelerating Diffusion Models with TaylorSeers , author=. 2025 , eprint=

    2025

  48. [48]

    2024 , eprint=

    -DiT: A Training-Free Acceleration Method Tailored for Diffusion Transformers , author=. 2024 , eprint=

    2024

  49. [49]

    2025 , eprint=

    Real-Time Video Generation with Pyramid Attention Broadcast , author=. 2025 , eprint=

    2025

  50. [50]

    The Fourteenth International Conference on Learning Representations , year=

    ScalingCache: Extreme Acceleration of DiTs through Difference Scaling and Dynamic Interval Caching , author=. The Fourteenth International Conference on Learning Representations , year=

  51. [51]

    S.; Boffi, N

    Albergo, M. S.; Boffi, N. M.; and Vanden-Eijnden, E. 2025. Stochastic Interpolants: A Unifying Framework for Flows and Diffusions. arXiv:2303.08797

    Pith/arXiv arXiv 2025

  52. [52]

    S.; and Vanden-Eijnden, E

    Albergo, M. S.; and Vanden-Eijnden, E. 2023. Building Normalizing Flows with Stochastic Interpolants. arXiv:2209.15571

    Pith/arXiv arXiv 2023

  53. [53]

    Black, K.; Janner, M.; Du, Y.; Kostrikov, I.; and Levine, S. 2024. Training Diffusion Models with Reinforcement Learning. arXiv:2305.13301

    Pith/arXiv arXiv 2024

  54. [54]

    E.; and Nocedal, J

    Bottou, L.; Curtis, F. E.; and Nocedal, J. 2018. Optimization Methods for Large-Scale Machine Learning. arXiv:1606.04838

    Pith/arXiv arXiv 2018

  55. [55]

    Chen, C.; Wang, A.; Wu, H.; Liao, L.; Sun, W.; Yan, Q.; and Lin, W. 2024 a . Enhancing Diffusion Models with Text-Encoder Reinforcement Learning. In Computer Vision – ECCV 2024: 18th European Conference, Milan, Italy, September 29–October 4, 2024, Proceedings, Part XXV, 182–198. Berlin, Heidelberg: Springer-Verlag. ISBN 978-3-031-72697-2

    2024

  56. [56]

    Chen, P.; Shen, M.; Ye, P.; Cao, J.; Tu, C.; Bouganis, C.-S.; Zhao, Y.; and Chen, T. 2024 b . -DiT: A Training-Free Acceleration Method Tailored for Diffusion Transformers. arXiv:2406.01125

    Pith/arXiv arXiv 2024

  57. [57]

    T.; Quek, T.; Poria, S.; and Liu, Z

    Chen, R.; Chai, W.; Yang, Z.; Zhang, X.; Zhou, J. T.; Quek, T.; Poria, S.; and Liu, Z. 2025. DiffPO: Diffusion-styled Preference Optimization for Efficient Inference-Time Alignment of Large Language Models. arXiv:2503.04240

    Pith/arXiv arXiv 2025

  58. [58]

    Choi, J.; Zhu, Y.; Guo, W.; Molodyk, P.; Yuan, B.; Bai, J.; Xin, Y.; Tao, M.; and Chen, Y. 2026. Rethinking the Design Space of Reinforcement Learning for Diffusion Models: On the Importance of Likelihood Estimation Beyond Loss Design. arXiv:2602.04663

    Pith/arXiv arXiv 2026

  59. [59]

    Du, H.; Zhang, R.; Liu, Y.; Wang, J.; Lin, Y.; Li, Z.; Niyato, D.; Kang, J.; Xiong, Z.; Cui, S.; Ai, B.; Zhou, H.; and Kim, D. I. 2024. Enhancing Deep Reinforcement Learning: A Tutorial on Generative Diffusion Models in Network Optimization. IEEE Communications Surveys & Tutorials, 26(4): 2611--2646

    2024

  60. [60]

    Fan, Y.; Watkins, O.; Du, Y.; Liu, H.; Ryu, M.; Boutilier, C.; Abbeel, P.; Ghavamzadeh, M.; Lee, K.; and Lee, K. 2023. DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Models. In Oh, A.; Naumann, T.; Globerson, A.; Saenko, K.; Hardt, M.; and Levine, S., eds., Advances in Neural Information Processing Systems, volume 36, 79858--79885. C...

    2023

  61. [61]

    Gu, L.; He, J.; Su, L.; He, K.; Wang, W.; and Liu, Y. 2026. ScalingCache: Extreme Acceleration of DiTs through Difference Scaling and Dynamic Interval Caching. In The Fourteenth International Conference on Learning Representations

    2026

  62. [62]

    L.; and Choi, Y

    Hessel, J.; Holtzman, A.; Forbes, M.; Bras, R. L.; and Choi, Y. 2022. CLIPScore: A Reference-free Evaluation Metric for Image Captioning. arXiv:2104.08718

    Pith/arXiv arXiv 2022

  63. [63]

    Ho, J.; Jain, A.; and Abbeel, P. 2020. Denoising Diffusion Probabilistic Models. arXiv:2006.11239

    Pith/arXiv arXiv 2020

  64. [64]

    Kirstain, Y.; Polyak, A.; Singer, U.; Matiana, S.; Penna, J.; and Levy, O. 2023. Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation. arXiv:2305.01569

    Pith/arXiv arXiv 2023

  65. [65]

    Labs, B. F. 2024. FLUX. https://github.com/black-forest-labs/flux

    2024

  66. [66]

    Labs, B. F.; Batifol, S.; Blattmann, A.; Boesel, F.; Consul, S.; Diagne, C.; Dockhorn, T.; English, J.; English, Z.; Esser, P.; Kulal, S.; Lacey, K.; Levi, Y.; Li, C.; Lorenz, D.; Müller, J.; Podell, D.; Rombach, R.; Saini, H.; Sauer, A.; and Smith, L. 2025. FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space. arXiv:2...

    Pith/arXiv arXiv 2025

  67. [67]

    Lee, J.; and Ye, J. C. 2026. PCPO: Proportionate Credit Policy Optimization for Aligning Image Generation Models. arXiv:2509.25774

    arXiv 2026

  68. [68]

    Li, J.; Cui, Y.; Huang, T.; Ma, Y.; Fan, C.; Cheng, Y.; Yang, M.; Zhong, Z.; and Bo, L. 2026. MixGRPO: Unlocking Flow-based GRPO Efficiency with Mixed ODE-SDE. arXiv:2507.21802

    Pith/arXiv arXiv 2026

  69. [69]

    Li, Y.; Wang, Y.; Zhu, Y.; Zhao, Z.; Lu, M.; She, Q.; and Zhang, S. 2025. BranchGRPO: Stable and Efficient GRPO with Structured Branching in Diffusion Models. arXiv:2509.06040

    arXiv 2025

  70. [70]

    Liu, J.; Liu, G.; Liang, J.; Li, Y.; Liu, J.; Wang, X.; Wan, P.; Zhang, D.; and Ouyang, W. 2025 a . Flow-GRPO: Training Flow Matching Models via Online RL. arXiv:2505.05470

    Pith/arXiv arXiv 2025

  71. [71]

    Liu, J.; Ye, Z.; Yuan, L.; Zhu, S.; Gao, Y.; Wu, J.; Li, K.; Wang, X.; Nie, X.; Huang, W.; and Ouyang, W. 2026. UniGRPO: Unified Policy Optimization for Reasoning-Driven Visual Generation. arXiv:2603.23500

    arXiv 2026

  72. [72]

    Liu, J.; Zou, C.; Lyu, Y.; Chen, J.; and Zhang, L. 2025 b . From Reusing to Forecasting: Accelerating Diffusion Models with TaylorSeers. arXiv:2503.06923

    Pith/arXiv arXiv 2025

  73. [73]

    Liu, X.; Gong, C.; and Liu, Q. 2022. Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow. arXiv:2209.03003

    Pith/arXiv arXiv 2022

  74. [74]

    Luo, Y.; Hu, X.; Fan, K.; Sun, H.; Chen, Z.; Xia, B.; Zhang, T.; Chang, Y.; and Wang, X. 2026. Reinforcement Learning Meets Masked Generative Models: Mask-GRPO for Text-to-Image Generation. arXiv:2510.13418

    arXiv 2026

  75. [75]

    Ma, T.; Zhang, M.; Wang, Y.; and Ye, Q. 2025. Consolidating Reinforcement Learning for Multimodal Discrete Diffusion Models. arXiv:2510.02880

    arXiv 2025

  76. [76]

    Miao, Z.; Wang, J.; Wang, Z.; Yang, Z.; Wang, L.; Qiu, Q.; and Liu, Z. 2024. Training Diffusion Models Towards Diverse Image Generation with Reinforcement Learning. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 10844--10853

    2024

  77. [77]

    Ping, B.; Chen, Z.; Yu, Y.; Hui, T.; Yan, J.; and Chang, B. 2026. LongR: Unleashing Long-Context Reasoning via Reinforcement Learning with Dense Utility Rewards. arXiv:2602.05758

    arXiv 2026

  78. [78]

    Rojas, K.; Lin, J.; Rasul, K.; Schneider, A.; Nevmyvaka, Y.; Tao, M.; and Deng, W. 2026. Improving Reasoning for Diffusion Language Models via Group Diffusion Policy Optimization. arXiv:2510.08554

    arXiv 2026

  79. [79]

    K.; Wu, Y.; and Guo, D

    Shao, Z.; Wang, P.; Zhu, Q.; Xu, R.; Song, J.; Bi, X.; Zhang, H.; Zhang, M.; Li, Y. K.; Wu, Y.; and Guo, D. 2024. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. arXiv:2402.03300

    Pith/arXiv arXiv 2024

  80. [80]

    Song, J.; Meng, C.; and Ermon, S. 2022. Denoising Diffusion Implicit Models. arXiv:2010.02502

    Pith/arXiv arXiv 2022

Showing first 80 references.