Pith. sign in

REVIEW 2 major objections 4 minor 63 references

Rewarding groups of images for covering every target category, rather than a single best sample, is the paper's proposed fix for text-to-image mode collapse and demographic skew.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 00:35 UTC pith:GTQKPWLZ

load-bearing objection A solid, honestly-scoped diversity-RL recipe with strong controlled experiments; the fairness claim is credible but rests on automatic evaluators, which the paper openly flags. the 2 major comments →

arxiv 2607.14962 v1 pith:GTQKPWLZ submitted 2026-07-16 cs.LG cs.AIcs.CV

Multi-Axis Max@K Reinforcement Learning for Representative Diversity in Text-to-Image Generation

Ku Onoda , Paavo Parmas , Hiroki Furuta , Soichiro Nishimori , Yuta Oshima , Shohei Taniguchi , Yutaka Matsuo This is my paper
classification cs.LG cs.AIcs.CV
keywords text-to-image generationreinforcement learningtarget-mode coverageset-level diversitymax@K policy gradientsperceived-appearance fairnessdiffusion modelscredit assignment
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper asks why repeated samples from a text-to-image model for one prompt cover so few of the visually distinct modes that the prompt permits, and why person-centric prompts skew demographically. It answers that the problem should be treated as coverage of a predefined set of target modes, and proposes multi-axis max@K, a reinforcement-learning objective that scores a batch by the best representative on each mode and only then sums across modes. The claim is that this objective's credit assignment—giving a sample credit on a category only if removing it lowers that category's batch maximum—moves probability mass toward underrepresented modes. A sympathetic reader would care because the same training mechanism, without any redesign, lifts perceived-appearance fairness scores by 0.23–0.36 across three automatic evaluators on held-out prompts while preserving image quality and text alignment. Controlled experiments with synthetic Gaussian modes and deterministic pixel-based color rewards support the mechanism before the fairness results.

Core claim

The paper's central claim, on its own terms, is that representative diversity is a property of a set of samples, not a single image. For a fixed prompt, each generated image gets one score per target mode, and the objective scores a k-sample group by taking, for each mode, the maximum score within the group and then summing those mode-wise maxima. A sample receives positive training credit on a mode only if removing it lowers that mode's group maximum, so samples in the same batch can specialize to different modes. The paper shows that in the hard-label case the expected objective equals the probability-weighted sum of modes being covered at least once, which is strictly concave with a unifo

What carries the argument

The load-bearing object is the multi-axis max@K objective G_k(S_k,c)=sum_d w_d(c) max_{x_i in S_k} r_d(x_i,c): per-mode scores are maxed over the sample set first and summed only afterward. Around it the paper builds an axis-preserving credit rule—leave-one-out marginal A_i^d=[r_d(x_i,c)-max_{j!=i} r_d(x_j,c)]_+—together with an expected-improvement estimator over sampled comparison subsets and a leave-two-out baseline, then aggregates standardized per-axis advantages into one policy weight for the diffusion update. This ordering is what lets different samples serve as representatives of different modes; scalar-first aggregation cannot do this, and the paper shows k=1 removes the set-level e

Load-bearing premise

The whole pipeline leans on a frozen prompt-ensemble classifier whose per-category scores must be a faithful ordering of perceived appearance during training rollouts; if those scores reward the wrong images, the fairness gains are an artifact of the reward model, and the method can also only amplify modes the base generator already has nonzero probability of producing.

What would settle it

Swap the frozen reward classifier for a different perceived-appearance scorer (or human annotations) and rerun the same training; if held-out fairness gains under an independent evaluator disappear or invert, the effect was an artifact of the original classifier rather than a property of the multi-axis credit rule. A second check: start from a base generator with zero measured mass on one target mode and see whether that mode ever appears after training—the paper predicts it will not.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the central claim holds, any text-to-image model can be steered toward better within-prompt mode coverage by adding a fixed axis scorer, without changing inference-time sampling or sacrificing quality.
  • Because the same objective improved deterministic color-mode coverage by 57% over base in a controlled setting, the mechanism is not tied to learned perception classifiers; it is a general credit-assignment rule.
  • The fairness gains transfer to evaluators not used in training, which the paper treats as evidence the improvement is in the generated images, not only in the reward classifier.
  • The representative-window size k controls strength of the coverage incentive; independent evaluator scores saturate near k equal to the number of target modes, while single-sample credit (k=1) leaves the base distribution essentially unchanged.
  • Proposition 1 implies the objective's optimum is uniform mode probabilities under hard mutually exclusive labels, so if one accepts the coverage framing, the target distribution is principled rather than an ad-hoc penalty.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper does not test: applying the same set-level credit to other evaluable axes—viewpoint, object subtype, artistic style—could give a general representative-diversity training knob; the paper demonstrates it only for perceived race/gender product cells and color modes.
  • If a target mode has zero mass in the base generator, the method cannot invent it; a practitioner should therefore pair this objective with a base model that already has some support for each desired mode, or add modes progressively.
  • A human-rating study would settle whether the fairness-score gains correspond to visibly diverse batches or merely to reward-classifier compliance; the paper explicitly leaves human validation to future work.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper formalizes target-mode coverage in text-to-image generation as a multi-axis max@K objective, G_k(S_k,c) = Σ_d w_d(c) max_{x_i∈S_k} r_d(x_i,c), and trains a diffusion/flow generator with group-based RL using axis-preserving leave-one-out credit. The authors derive the population-level coverage formula in Proposition 1, present a practical EI+L2O credit estimator for training groups larger than the representative window, and validate the mechanism in three stages: a synthetic 9-mode Gaussian mixture, SD3.5-M with deterministic pixel-based color rewards, and perceived-appearance fairness on race5 and race5×gender2 axes. On 200 held-out occupation prompts, they report Fairness Score improvements of 0.23–0.36 over the base model across three automatic evaluators (CLIP, VLM, FairFace), while maintaining measured image quality and text alignment.

Significance. If the empirical claims hold, the paper provides a clean, reward-agnostic set-level credit-assignment mechanism that lets different samples specialize to different target modes—a useful primitive for diversity-oriented RL post-training. The evidence is well controlled: Proposition 1 gives a closed-form population target; the toy and color-reward experiments isolate the credit mechanism under known rewards; the fairness experiment reports mean±std over three seeds and uses two evaluators not used in training. The authors are also transparent about limitations, including the requirement that modes already have nonzero support and the reliance on automatic classifiers. The main risk is the gap between the phrase 'perceived-appearance fairness' and the automatic classifiers used to operationalize it.

major comments (2)
  1. [§6.1, App. G Eq. (23), Table 1] The fairness claim is operationalized entirely through automatic classifiers. The training reward and the CLIP evaluation column are the same frozen prompt ensemble, so that column partly measures reward over-optimization; the VLM and FairFace columns mitigate this but all three are web-trained automatic models. Section 7 states that human validation remains future work, yet the abstract and Section 6 frame the result as 'perceived-appearance fairness.' This is load-bearing: without human grounding, the strongest interpretation of the fairness claim is unsupported. I recommend either adding a small human evaluation on a sample of generated images or consistently rephrasing the claims to 'automatic-evaluator fairness' and explicitly noting that human perception may differ.
  2. [§7, Table 10] The method can only reweight modes that the base generator already samples. Table 10 shows base FairFace mass of 3.3% for Black and 5.5% for Indian; the reported gains are obtained by amplifying small but nonzero support. Section 7 states this, but the abstract and conclusion present the 0.23–0.36 improvement without the caveat. For fairness applications this is an important scope limitation: the method cannot create absent modes. Please make this limitation prominent in the abstract/conclusion and avoid implying general demographic debiasing.
minor comments (4)
  1. [Figure 6] The y-axis label 'Fairness Score' with the legend 'CLIP (reward)' is slightly confusing; consider labeling the reward-side curve as 'CLIP reward-side score' to distinguish it from the independent evaluators.
  2. [Appendix C, Eq. (17)] The Lagrange-multiplier derivation is terse. Defining λ and writing the constraint explicitly would make the optimum condition easier to check.
  3. [Table 1 caption] The abbreviations Pick., Aes., C-T are defined in the running text but not in the caption; please spell them out in the caption for self-containedness.
  4. [§4.3, Eq. (9)] The relation between the EI+L2O estimator and the leave-one-out objective in Eq. (8) is stated but not derived. A brief derivation or a more precise pointer to the cited estimator would improve rigor.

Circularity Check

1 steps flagged

The CLIP evaluation column is the training reward itself, so part of the reported gain is reward over-optimization; independent VLM and FairFace evaluators keep the central claim partially independent.

specific steps
  1. fitted input called prediction [Appendix G, Eq. (23); Sec. 6.1 'Evaluation'; Sec. 6.2 'Representative credit outperforms...', Table 1a]
    "The CLIP reward and the CLIP evaluator use a frozen ViT-L/14 checkpoint ... [Eq. 23] ... the CLIP column alone could reflect over-fitting to the reward classifier. The two independent evaluators show the same trend."

    The training advantage is computed from rd(x,c)=softmax(...) in Eq. (23), and the CLIP Fairness Score in Table 1a uses the same frozen CLIP prompt-ensemble scores. Thus the CLIP column measures the very reward that Eq. (6) optimizes; an increase in that column is expected from optimizing the objective, not an independent validation of perceived-appearance fairness. The paper itself acknowledges this and relies on Qwen2.5-VL and FairFace as out-of-reward evaluators. Those evaluators supply independent evidence, so the circularity is confined to the CLIP column and does not make the whole claim vacuous.

full rationale

The main derivation chain is self-contained rather than circular. Eq. (3) defines the multi-axis max@K set objective; Eq. (7) is the exact leave-one-out marginal contribution of the axis-wise maximum; Eqs. (9)-(12) specify a complete EI+L2O estimator; and Prop. 1's E[G_k]=Σ_d w_d[1-(1-q_d)^k] follows from independent hard-label draws without importing the result being proved. The toy and deterministic-color experiments use training-side rewards also as evaluation metrics, but the paper explicitly frames these as controlled validations of the credit-assignment mechanism, not as external predictions. The self-citations to ReMax and to Takashiro et al. are acknowledged related work, and the estimator they cite is written out in full in the paper, so they are not load-bearing in the sense of importing an unverified uniqueness or ansatz. The genuine overlap is CLIP: Eq. (23) defines both the reward and the CLIP evaluator, so the CLIP Fairness Score partly reflects reward over-optimization rather than independent evidence. However, the central fairness claim is also supported by Qwen2.5-VL and FairFace, which were not used during training and show the same qualitative improvement, so the result does not reduce by construction to its inputs. The lack of human validation is a correctness/validity limitation explicitly stated in Sec. 7, not itself a circularity.

Axiom & Free-Parameter Ledger

6 free parameters · 6 axioms · 0 invented entities

The method's theoretical core is a concave coverage objective (Prop. 1) requiring only standard probability; the empirical contribution rests on two domain assumptions the paper itself flags: a frozen imperfect classifier as the reward signal, and nonzero base-model support for every target mode. No new physical entities are introduced; the free parameters are the usual RL hyperparameters plus the externally chosen uniform target distribution.

free parameters (6)
  • representative window k = k=D: 5 (race5), 10 (race5×gender2), 7 (color), 9 (toy)
    Controls the strength of multi-sample credit; k=1 removes the set effect (Eq. 4 marginal gain k(1-q)^{k-1}). Chosen as k=D via the 'smallest window that can host one representative per mode' argument (App. C.2) and ablated in Fig. 6 and App. H; CLIP reward-side score keeps rising past k=D while independent evaluators saturate.
  • KL weight β = 0.05
    Chosen hyperparameter in the diffusion RL objective (Eq. 6), fixed across all learned rows (App. E Table 2).
  • candidate group size m = 16
    Rollout group per prompt; EI+L2O credit is estimated over k-subsets of this group (Eqs. 9-10).
  • CLIP softmax temperature τ = 1.0
    Scales the reward-side softmax in Eq. 23; set without tuning.
  • uniform target distribution over modes = w_d=1, target q_d=1/D
    The fairness target is uniform coverage of the predefined perceived-race/gender taxonomy; this is an externally chosen normative target, not derived (Sec. 6.1, App. F). Non-uniform weights are analyzed but not used.
  • color reward soft-pixel partition = temperature-softmax over 7 RGB logits
    Hand-designed deterministic color axes (App. D); the seven rewards sum to 1 per image, making the axes a competing partition.
axioms (6)
  • domain assumption The frozen CLIP prompt ensemble provides faithful per-mode perceived-attribute scores during training
    The sole training reward signal is r_d(x,c) from Eq. 23; the paper validates balanced accuracy 0.781 (held-out 0.767) vs FairFace 0.791 on FairFace-val (App. G Table 4).
  • domain assumption Each target mode has nonzero probability under the base policy
    Stated in Sec. 7: 'can only credit modes that the generator already samples.' Failure would invalidate coverage gains for absent modes; the base model has small but nonzero support (FairFace Black 3.3%, Indian 5.5%, App. J Table 10).
  • domain assumption Uniform mode distribution is the correct fairness target
    The objective's uniform optimum (Prop. 1) is adopted as the fairness criterion; Appendix F acknowledges it is a simplified measurement taxonomy.
  • standard math Flow-GRPO stochastic sampler has the same marginals as the deterministic flow
    Eq. (14) converts the ODE to an SDE 'with the same marginal distributions' following Flow-GRPO [33]; per-step log-probs and KL are then computable.
  • standard math Samples in a set are independent and identically distributed from πθ
    Needed for Prop. 1 (Eq. 4) and for the subset-averaging estimator (Eq. 9).
  • domain assumption Mutually exclusive hard-label modes with rewards that sum to one
    The problem framing (one image = one mode) is what makes scalar-first aggregation degenerate (Sec. 3.2); the perceived-race/gender soft rewards approximate this via softmax/outer product.

pith-pipeline@v1.3.0-alltime-deepseek · 23779 in / 21755 out tokens · 215403 ms · 2026-08-02T00:35:13.431972+00:00 · methodology

0 comments
read the original abstract

Text-to-image (T2I) models can synthesize realistic, prompt-aligned images, yet samples generated for the same prompt often cover only a small subset of visually distinct modes. This limits the diversity of images, and for person-centric prompts, can reflect or amplify demographic skew. We formalize this problem as coverage of a predefined set of semantically specified modes, which we call target-mode coverage. We then propose multi-axis max@K, a group-based reinforcement learning objective for improving such coverage in diffusion-based T2I models. Given a group of samples and one score per target category, multi-axis max@K first takes the maximum score across samples for each category and then sums these category-wise maxima. The resulting credit assignment gives a sample positive weight on a category only when it increases that category's group-wise maximum, allowing different samples to contribute to different categories. We first validate the credit-assignment mechanism on a synthetic mixture and on SD3.5-M using deterministic pixel-based color rewards. We then evaluate the same objective on perceived-appearance fairness. Across three automatic evaluators on held-out prompts, multi-axis max@K improves the Fairness Score by 0.23-0.36 relative to the base model, while maintaining image quality and text alignment.

Figures

Figures reproduced from arXiv: 2607.14962 by Hiroki Furuta, Ku Onoda, Paavo Parmas, Shohei Taniguchi, Soichiro Nishimori, Yutaka Matsuo, Yuta Oshima.

Figure 1
Figure 1. Figure 1: Diversifying color modes in T2I generation. Columns correspond to seven pixel-based reward functions, each representing a different rule-based color mode. Multi-axis max@K encourages the sampled batch to cover these color modes with different samples. Preprint. arXiv:2607.14962v1 [cs.LG] 16 Jul 2026 [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Multi-axis max@K. For each prompt, an m-sample on-policy candidate group is scored on D reward axes, forming a D × m reward matrix. Multi-axis max@K compares samples separately on each axis before aggregating axes, so different samples can receive credit for improving different target-mode representatives. The resulting axis-wise credits are summed into per-sample policy weights and used in the diffusion R… view at source ↗
Figure 3
Figure 3. Figure 3: Multi-axis max@K redistributes probability mass across a nine-mode target. A controlled policy over nine mutually exclusive Gaussian modes isolates the credit-assignment mechanism before image experiments. (A) Starting from the same skewed base distribution, a single-mode scalar reward concentrates probability mass on one mode, whereas multi-axis max@K spreads mass across the target modes. (B) The represen… view at source ↗
Figure 5
Figure 5. Figure 5: Perceived-race composi￾tion. FairFace [24] top-label shares on the race5 axis over the held￾out prompts; methods are sorted by Fairness Score (Table 1a). Qualitative view [PITH_FULL_IMAGE:figures/full_fig_p010_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Representative-window sweep on the race5 axis. Fairness Score is plotted against the training representative window k under the fixed race5 protocol. FairFace and VLM saturate near k=5, whereas the reward-side CLIP score contin￾ues to increase. 10 [PITH_FULL_IMAGE:figures/full_fig_p010_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Weights steer the marginal; k controls the strength. Single-mode boost (ρ=8 on the bold-ring mode g ⋆ ), uniform otherwise. Top: on-policy sampling cloud (color = mode). Bottom: weight wd (gray) vs. trained mass pd (blue), dotted line = 1/D; the boosted mode is the black-edged bar. The weights are fixed across columns, yet p(g ⋆ ) rises above 1/D at small/moderate k and returns to 1/D as k grows. 18 [PITH… view at source ↗
Figure 8
Figure 8. Figure 8: The rarest mode approaches the uniform target as k approaches D. Uniform weights, graded-skew start, for D ∈ {4, 6, 9} (rows). Top of each block: sampling cloud; bottom: per-mode mass pd (dotted = 1/D; the rarest mode is the bold ring / black-edged bar). Smaller windows can already lift the rarest mode close to uniform for small D, but the gap below 1/D becomes larger as D grows and the tail mode becomes r… view at source ↗
Figure 9
Figure 9. Figure 9: Qualitative examples for the rule-based color modes. For each prompt and method, we generate 64 images and show the image with the highest score for each rule-based color axis. Columns correspond to red, green, blue, warm, cool, bright achromatic, and dark achromatic. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Random 16-sample batches for the rule-based color modes. For the same prompts as [PITH_FULL_IMAGE:figures/full_fig_p021_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Full held-out batches for five occupation prompts. Each panel shows all M=16 generated images for SD3.5-M (Base) and Ours, without representative selection [PITH_FULL_IMAGE:figures/full_fig_p028_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Matched non-person OOD examples. Each panel compares SD3.5-M (base) and Ours on the same non-person prompt and matched seeds. 28 [PITH_FULL_IMAGE:figures/full_fig_p028_12.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

63 extracted references · 18 linked inside Pith

  1. [1]

    Consistency-diversity-realism pareto fronts of conditional image generative models.arXiv preprint arXiv:2406.10429, 2024

    Pietro Astolfi, Marlene Careil, Melissa Hall, Oscar Mañas, Matthew Muckley, Jakob Verbeek, Adriana Romero-Soriano, and Michal Drozdzal. Consistency-diversity-realism pareto fronts of conditional image generative models.arXiv preprint arXiv:2406.10429, 2024

    Pith/arXiv arXiv 2024

  2. [2]

    The best of N worlds: Aligning reinforcement learning with best-of-N sampling via max@k optimisation.arXiv preprint arXiv:2510.23393, 2025

    Farid Bagirov, Mikhail Arkhipov, Ksenia Sycheva, Evgeniy Glukhov, and Egor Bogomolov. The best of N worlds: Aligning reinforcement learning with best-of-N sampling via max@k optimisation.arXiv preprint arXiv:2510.23393, 2025

    arXiv 2025

  3. [3]

    Vector policy optimization: Training for diversity improves test-time search.arXiv preprint arXiv:2605.22817, 2026

    Ryan Bahlous-Boldi, Isha Puri, Idan Shenfeld, Akarsh Kumar, Mehul Damani, Sebastian Risi, Omar Khattab, Zhang-Wei Hong, and Pulkit Agrawal. Vector policy optimization: Training for diversity improves test-time search.arXiv preprint arXiv:2605.22817, 2026

    Pith/arXiv arXiv 2026

  4. [4]

    Qwen2.5-VL technical report.arXiv preprint arXiv:2502.13923, 2025

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-VL technical report.ar...

    Pith/arXiv arXiv 2025

  5. [5]

    Easily accessible text-to-image generation amplifies demographic stereotypes at large scale

    Federico Bianchi, Pratyusha Kalluri, Esin Durmus, Faisal Ladhak, Myra Cheng, Debora Nozza, Tatsunori Hashimoto, Dan Jurafsky, James Zou, and Aylin Caliskan. Easily accessible text-to-image generation amplifies demographic stereotypes at large scale. InFAccT, 2023

    2023

  6. [6]

    Training diffusion models with reinforcement learning

    Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement learning. InICLR, 2024

    2024

  7. [7]

    SEGA: Instructing text-to-image models using semantic guidance

    Manuel Brack, Felix Friedrich, Dominik Hintersdorf, Lukas Struppek, Patrick Schramowski, and Kristian Kersting. SEGA: Instructing text-to-image models using semantic guidance. InNeurIPS, 2023

    2023

  8. [8]

    HoloFair: Unified T2I fairness evaluation and Fair-GRPO debiasing.arXiv preprint arXiv:2605.24687, 2026

    Ruyi Chen, Lu Zhou, Xiaogang Xu, Chiyu Zhang, Jiafei Wu, and Liming Fang. HoloFair: Unified T2I fairness evaluation and Fair-GRPO debiasing.arXiv preprint arXiv:2605.24687, 2026

    Pith/arXiv arXiv 2026

  9. [9]

    TIBET: Identifying and evaluating biases in text-to-image generative models

    Aditya Chinchure et al. TIBET: Identifying and evaluating biases in text-to-image generative models. In ECCV, 2024

    2024

  10. [10]

    Inference-aware fine-tuning for best-of-n sampling in large language models.arXiv preprint arXiv:2412.15287, 2024

    Yinlam Chow, Guy Tennenholtz, Izzeddin Gur, Vincent Zhuang, Bo Dai, Sridhar Thiagarajan, Craig Boutilier, Rishabh Agarwal, Aviral Kumar, and Aleksandra Faust. Inference-aware fine-tuning for best-of-n sampling in large language models.arXiv preprint arXiv:2412.15287, 2024

    arXiv 2024

  11. [11]

    Debiasing vision-language models via biased prompts.arXiv preprint arXiv:2302.00070, 2023

    Ching-Yao Chuang, Varun Jampani, Yuanzhen Li, Antonio Torralba, and Stefanie Jegelka. Debiasing vision-language models via biased prompts.arXiv preprint arXiv:2302.00070, 2023

    Pith/arXiv arXiv 2023

  12. [12]

    OpenBias: Open-set bias detection in text-to-image generative models

    Moreno D’Incà et al. OpenBias: Open-set bias detection in text-to-image generative models. InCVPR, 2024

    2024

  13. [13]

    Scaling rectified flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, and Robin Rombach. Scaling rectified flow transformers for high-resolution image synthesis. InICML, 2024. 12

    2024

  14. [14]

    DPOK: Reinforcement learning for fine-tuning text-to-image diffusion models

    Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Mo- hammad Ghavamzadeh, Kangwook Lee, and Kimin Lee. DPOK: Reinforcement learning for fine-tuning text-to-image diffusion models. InNeurIPS, 2023

    2023

  15. [15]

    Fair diffusion: Instructing text-to-image generation models on fairness

    Felix Friedrich, Manuel Brack, Lukas Struppek, Dominik Hintersdorf, Patrick Schramowski, Sasha Luccioni, and Kristian Kersting. Fair diffusion: Instructing text-to-image generation models on fairness. arXiv preprint arXiv:2302.10893, 2023

    Pith/arXiv arXiv 2023

  16. [16]

    FairImagen: Post- processing for bias mitigation in text-to-image models

    Zihao Fu, Ryan Brown, Shun Shao, Kai Rawal, Eoin Delaney, and Chris Russell. FairImagen: Post- processing for bias mitigation in text-to-image models. InNeurIPS, 2025

    2025

  17. [17]

    Improving dynamic object interactions in text-to-video generation with AI feedback.arXiv preprint arXiv:2412.02617, 2024

    Hiroki Furuta, Heiga Zen, Dale Schuurmans, Aleksandra Faust, Yutaka Matsuo, Percy Liang, and Sherry Yang. Improving dynamic object interactions in text-to-video generation with AI feedback.arXiv preprint arXiv:2412.02617, 2024

    Pith/arXiv arXiv 2024

  18. [18]

    Unified concept editing in diffusion models

    Rohit Gandikota, Hadas Orgad, Yonatan Belinkov, Joanna Materzy´nska, and David Bau. Unified concept editing in diffusion models. InWACV, 2024

    2024

  19. [19]

    Using reward uncertainty to induce diverse behaviour in reinforcement learning.arXiv preprint arXiv:2606.03962, 2026

    Anthony GX-Chen, Ankit Anand, Gheorghe Comanici, Zaheer Abbas, Eser Aygün, David Smalling, Shibl Mourad, Doina Precup, André Barreto, and Mark Rowland. Using reward uncertainty to induce diverse behaviour in reinforcement learning.arXiv preprint arXiv:2606.03962, 2026

    Pith/arXiv arXiv 2026

  20. [20]

    Polychromic objectives for reinforcement learning

    Jubayer Ibn Hamid, Ifdita Hasan Orney, Ellen Xu, Chelsea Finn, and Dorsa Sadigh. Polychromic objectives for reinforcement learning. InICLR, 2026

    2026

  21. [21]

    TempFlow-GRPO: When timing matters for GRPO in flow models.arXiv preprint arXiv:2508.04324, 2025

    Xiaoxuan He, Siming Fu, Yuke Zhao, Wanli Li, Jian Yang, Dacheng Yin, et al. TempFlow-GRPO: When timing matters for GRPO in flow models.arXiv preprint arXiv:2508.04324, 2025

    Pith/arXiv arXiv 2025

  22. [22]

    Debiasing diffusion model: Enhancing fairness through latent representation learning in stable diffusion model.arXiv preprint arXiv:2503.12536, 2025

    Lin-Chun Huang, Ching Chieh Tsao, Fang-Yi Su, and Jung-Hsien Chiang. Debiasing diffusion model: Enhancing fairness through latent representation learning in stable diffusion model.arXiv preprint arXiv:2503.12536, 2025

    Pith/arXiv arXiv 2025

  23. [23]

    FairGen: Enhancing fairness in text-to-image diffusion models via self-discovering latent directions.arXiv preprint arXiv:2412.18810, 2024

    Yilei Jiang, Weihong Li, Yiyuan Zhang, Minghong Cai, and Xiangyu Yue. FairGen: Enhancing fairness in text-to-image diffusion models via self-discovering latent directions.arXiv preprint arXiv:2412.18810, 2024

    arXiv 2024

  24. [24]

    FairFace: Face attribute dataset for balanced race, gender, and age for bias measurement and mitigation

    Kimmo Kärkkäinen and Jungseock Joo. FairFace: Face attribute dataset for balanced race, gender, and age for bias measurement and mitigation. InWACV, 2021

    2021

  25. [25]

    Rethinking training for de-biasing text-to-image generation: Unlocking the potential of stable diffusion

    Eunji Kim, Siwon Kim, Minjun Park, Rahim Entezari, and Sungroh Yoon. Rethinking training for de-biasing text-to-image generation: Unlocking the potential of stable diffusion. InCVPR, 2025

    2025

  26. [26]

    Pick-a-Pic: An open dataset of user preferences for text-to-image generation

    Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. Pick-a-Pic: An open dataset of user preferences for text-to-image generation. InNeurIPS, 2023

    2023

  27. [27]

    Emergence of exploration in policy gradient reinforcement learning via resetting

    Sotetsu Koyamada, Paavo Parmas, Tadashi Kozuno, and Shin Ishii. Emergence of exploration in policy gradient reinforcement learning via resetting. OpenReview submission to ICLR 2023, 2022

    2023

  28. [28]

    Improved precision and recall metric for assessing generative models

    Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall metric for assessing generative models. InNeurIPS, 2019

    2019

  29. [29]

    Holistic evaluation of text-to-image models

    Tony Lee et al. Holistic evaluation of text-to-image models. InNeurIPS Datasets and Benchmarks, 2023

    2023

  30. [30]

    SetPO: Set-level policy optimization for diversity-preserving LLM reasoning

    Chenyi Li, Yuan Zhang, Bo Wang, Guoqing Ma, Wei Tang, Haoyang Huang, and Nan Duan. SetPO: Set-level policy optimization for diversity-preserving LLM reasoning. InICML, 2026

    2026

  31. [31]

    Fair text-to-image diffusion via fair mapping

    Jia Li, Lijie Hu, Jingfeng Zhang, Tianhang Zheng, Hua Zhang, and Di Wang. Fair text-to-image diffusion via fair mapping. InAAAI, 2025

    2025

  32. [32]

    DiverseGRPO: Mitigating mode collapse in image generation via diversity-aware GRPO.arXiv preprint arXiv:2512.21514, 2025

    Henglin Liu, Huijuan Huang, Jing Wang, Chang Liu, Xiu Li, and Xiangyang Ji. DiverseGRPO: Mitigating mode collapse in image generation via diversity-aware GRPO.arXiv preprint arXiv:2512.21514, 2025

    arXiv 2025

  33. [33]

    Flow-GRPO: Training flow matching models via online RL

    Jie Liu, Gongye Liu, Jiajun Liang, Yangguang Li, Jiaheng Liu, Xintao Wang, Pengfei Wan, Di Zhang, and Wanli Ouyang. Flow-GRPO: Training flow matching models via online RL. InNeurIPS, 2025

    2025

  34. [34]

    Improving video generation with human feedback

    Jie Liu, Gongye Liu, Jiajun Liang, Ziyang Yuan, Xiaokun Liu, Mingwu Zheng, Xiele Wu, Qiulin Wang, Menghan Xia, Xintao Wang, Xiaohong Liu, Fei Yang, Pengfei Wan, Di Zhang, Kun Gai, Yujiu Yang, and Wanli Ouyang. Improving video generation with human feedback. InNeurIPS, 2025. 13

    2025

  35. [35]

    Beyond the Dirac Delta: Mitigating diversity collapse in reinforcement fine-tuning for versatile image generation.arXiv preprint arXiv:2601.12401, 2026

    Jinmei Liu, Haoru Li, Zhenhong Sun, Chaofeng Chen, Yatao Bian, Bo Wang, et al. Beyond the Dirac Delta: Mitigating diversity collapse in reinforcement fine-tuning for versatile image generation.arXiv preprint arXiv:2601.12401, 2026

    arXiv 2026

  36. [36]

    Stable bias: Evaluating societal representations in diffusion models

    Sasha Luccioni, Christopher Akiki, Margaret Mitchell, and Yacine Jernite. Stable bias: Evaluating societal representations in diffusion models. InNeurIPS Datasets and Benchmarks, 2023

    2023

  37. [37]

    Training diffusion models towards diverse image generation with reinforcement learning

    Zichen Miao, Jiang Wang, Ze Wang, Zhengyuan Yang, Lijuan Wang, Qiang Qiu, and Zicheng Liu. Training diffusion models towards diverse image generation with reinforcement learning. InCVPR, pages 10844–10853, 2024

    2024

  38. [38]

    Reliable fidelity and diversity metrics for generative models

    Muhammad Ferjad Naeem, Seong Joon Oh, Youngjung Uh, Yunjey Choi, and Jaejun Yoo. Reliable fidelity and diversity metrics for generative models. InICML, 2020

    2020

  39. [39]

    Retry policy gradients in continuous action spaces.arXiv preprint arXiv:2606.05888, 2026

    Soichiro Nishimori and Paavo Parmas. Retry policy gradients in continuous action spaces.arXiv preprint arXiv:2606.05888, 2026

    Pith/arXiv arXiv 2026

  40. [40]

    Emergence of exploration in policy gradient reinforcement learning via retrying.arXiv preprint arXiv:2606.00151, 2026

    Soichiro Nishimori, Paavo Parmas, Sotetsu Koyamada, Tadashi Kozuno, Toshinori Kitamura, Shin Ishii, and Yutaka Matsuo. Emergence of exploration in policy gradient reinforcement learning via retrying.arXiv preprint arXiv:2606.00151, 2026

    Pith/arXiv arXiv 2026

  41. [41]

    Diversity-aware max@k optimization for improving best-of-N performance in image generation with diffusion models (in Japanese)

    Ku Onoda, Yuta Oshima, Shohei Taniguchi, Soichiro Nishimori, Paavo Parmas, Hiroki Furuta, and Yutaka Matsuo. Diversity-aware max@k optimization for improving best-of-N performance in image generation with diffusion models (in Japanese). InThe 40th Annual Conference of the Japanese Society for Artificial Intelligence (JSAI), 2026. 2K4-GS-7b-06

    2026

  42. [42]

    Inference-time text-to-video alignment with diffusion latent beam search

    Yuta Oshima, Masahiro Suzuki, Yutaka Matsuo, and Hiroki Furuta. Inference-time text-to-video alignment with diffusion latent beam search. InNeurIPS, 2025

    2025

  43. [43]

    MultiBanana: A challenging benchmark for multi-reference text-to-image generation

    Yuta Oshima, Daiki Miyake, Kohsei Matsutani, Yusuke Iwasawa, Masahiro Suzuki, Yutaka Matsuo, and Hiroki Furuta. MultiBanana: A challenging benchmark for multi-reference text-to-image generation. In CVPR, pages 448–460, 2026

    2026

  44. [44]

    Balancing act: Distribution-guided debiasing in diffusion models

    Rishubh Parihar et al. Balancing act: Distribution-guided debiasing in diffusion models. InCVPR, 2024

    2024

  45. [45]

    OrderGrad: Optimizing beyond the mean with order-statistic policy gradient estimation.arXiv preprint arXiv:2606.06096, 2026

    Paavo Parmas, Yongmin Kim, Kohsei Matsutani, Shota Takashiro, Soichiro Nishimori, Takeshi Kojima, Yusuke Iwasawa, and Yutaka Matsuo. OrderGrad: Optimizing beyond the mean with order-statistic policy gradient estimation.arXiv preprint arXiv:2606.06096, 2026

    Pith/arXiv arXiv 2026

  46. [46]

    Escaping the mode: Multi-answer reinforcement learning in LMs

    Isha Puri, Mehul Damani, Idan Shenfeld, Marzyeh Ghassemi, Jacob Andreas, and Yoon Kim. Escaping the mode: Multi-answer reinforcement learning in LMs. InICML, 2026

    2026

  47. [47]

    From scale to speed: Adaptive test-time scaling for image editing

    Xiangyan Qu, Zhenlong Yuan, Jing Tang, Rui Chen, Datao Tang, Meng Yu, Lei Sun, Yancheng Bai, Xiangxiang Chu, Gaopeng Gou, Gang Xiong, and Yujun Cai. From scale to speed: Adaptive test-time scaling for image editing. InCVPR, pages 23272–23282, 2026

    2026

  48. [48]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. InICML, 2021

    2021

  49. [49]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. InCVPR, 2022

    2022

  50. [50]

    Seyedmorteza Sadat, Jakob Buhmann, Derek Bradley, Otmar Hilliges, and Romann M. Weber. CADS: Unleashing the diversity of diffusion models through condition-annealed sampling. InICLR, 2024

    2024

  51. [51]

    Mehdi S. M. Sajjadi, Olivier Bachem, Mario Lucic, Olivier Bousquet, and Sylvain Gelly. Assessing generative models via precision and recall. InNeurIPS, 2018

    2018

  52. [52]

    LAION-5B: An open large-scale dataset for training next generation image-text models

    Christoph Schuhmann et al. LAION-5B: An open large-scale dataset for training next generation image-text models. InNeurIPS Datasets and Benchmarks, 2022

    2022

  53. [53]

    Finetuning text-to-image diffusion models for fairness

    Xudong Shen, Chao Du, Tianyu Pang, Min Lin, Yongkang Wong, and Mohan Kankanhalli. Finetuning text-to-image diffusion models for fairness. InICLR, 2024

    2024

  54. [54]

    On advantage estimates for Max@K policy gradients.arXiv preprint arXiv:2606.06080, 2026

    Shota Takashiro, Soichiro Nishimori, Paavo Parmas, Yongmin Kim, Kohsei Matsutani, Gouki Minegishi, Yusuke Iwasawa, Takeshi Kojima, and Yutaka Matsuo. On advantage estimates for Max@K policy gradients.arXiv preprint arXiv:2606.06080, 2026. 14

    Pith/arXiv arXiv 2026

  55. [55]

    Finite-time regret analysis of retry-aware bandits.arXiv preprint arXiv:2605.20854, 2026

    Bingkui Tong, Junpei Komiyama, Soichiro Nishimori, and Paavo Parmas. Finite-time regret analysis of retry-aware bandits.arXiv preprint arXiv:2605.20854, 2026

    Pith/arXiv arXiv 2026

  56. [56]

    Beyond the prompt: Gender bias in text-to-image models, with a case study on hospital professions.arXiv preprint arXiv:2510.00045, 2025

    Franck Vandewiele, Remi Synave, Samuel Delepoulle, and Remi Cozot. Beyond the prompt: Gender bias in text-to-image models, with a case study on hospital professions.arXiv preprint arXiv:2510.00045, 2025

    arXiv 2025

  57. [57]

    Pass@K policy optimization: Solving harder reinforcement learning problems.arXiv preprint arXiv:2505.15201, 2025

    Christian Walder and Deep Karkhanis. Pass@K policy optimization: Solving harder reinforcement learning problems.arXiv preprint arXiv:2505.15201, 2025

    Pith/arXiv arXiv 2025

  58. [58]

    Diffusion model alignment using direct preference optimization

    Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization. InCVPR, 2024

    2024

  59. [59]

    RewardDance: Reward scaling in visual generation.arXiv preprint arXiv:2509.08826, 2025

    Jie Wu, Yu Gao, Zilyu Ye, Ming Li, Liang Li, Hanzhong Guo, Jie Liu, Zeyue Xue, Xiaoxia Hou, Wei Liu, Yan Zeng, and Weilin Huang. RewardDance: Reward scaling in visual generation.arXiv preprint arXiv:2509.08826, 2025

    Pith/arXiv arXiv 2025

  60. [60]

    ImageReward: Learning and evaluating human preferences for text-to-image generation

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. ImageReward: Learning and evaluating human preferences for text-to-image generation. InNeurIPS, 2023

    2023

  61. [61]

    DanceGRPO: Unleashing GRPO on visual generation.arXiv preprint arXiv:2505.07818, 2025

    Zeyue Xue, Jie Wu, Yu Gao, Fangyuan Kong, Lingting Zhu, Mengzhao Chen, et al. DanceGRPO: Unleashing GRPO on visual generation.arXiv preprint arXiv:2505.07818, 2025

    Pith/arXiv arXiv 2025

  62. [62]

    ITI-GEN: Inclusive text-to-image generation

    Cheng Zhang, Xuanbai Chen, Siqi Chai, Chen Henry Wu, Dmitry Lagun, Thabo Beeler, and Fernando De la Torre. ITI-GEN: Inclusive text-to-image generation. InICCV, 2023

    2023

  63. [63]

    a photo of the face of a person

    Shu Zhang, Xinyi Yang, Yihao Feng, Can Qin, Chia-Chih Chen, Ning Yu, Zeyuan Chen, Huan Wang, Silvio Savarese, Stefano Ermon, Caiming Xiong, and Ran Xu. HIVE: Harnessing human feedback for instructional visual editing.CVPR, 2024. 15 Appendix. Supporting Details A Background on Diffusion RL This appendix summarizes how a text-to-image diffusion or flow mode...

    2024