{"id":"a8ebef10-2c27-4c8f-8d95-8a179e422952","arxiv_id":"2504.14534","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"SUDO fine-tunes text-to-image diffusion models with a direct preference objective built from self-generated pairs, where the losing image is a random mismatched image from the training data.","lead":"SUDO is a new fine-tuning method for text-to-image diffusion models that replaces human-annotated preference pairs with automatically constructed pairs, using each image as the 'good' example and a random image from the training set as the 'bad' one. It claims better image quality and text alignment than supervised fine-tuning and standard DPO, using only standard image-caption data.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The core preference premise is not supported: the only effective 'downgrade' is a random text-image mismatch, not an image-quality degradation, and no account is given of why this should improve generation.","rationale":"The paper's central claim is that a self-supervised DPO formulation improves text-to-image generation. For that to hold, the constructed preference pair must convey signal about what makes images better. The evidence shows that only one construction works, and that construction is not a degradation of the winner. Table 3 makes this precise: blur and random grid, which are true quality degradations, perform worse than SFT, while random image performs much better. This is not a minor detail. It means Eq. 8's loss is not functioning as 'direct preference optimization' in the standard sense; it is a contrastive objective that pushes the denoiser to distinguish matched from mismatched text-image pairs. That may be a useful objective, but the paper does not analyze it or bound its behavior, and the 'image-level learning' narrative is not directly evidenced. The reader's condition that code and error bars be provided is appropriate, but there is a more substantive condition: the mechanism must be tested. The SDXL results also show that the 'consistent enhancement' claim is too strong, since Aesthetics is flat and DPO is better on some alignment metrics. I therefore keep the conditional verdict rather than moving it; the acceptance criteria should include the proposed disjoint-pool and seed-variability experiments.","tokens_in":11329,"tokens_out":8084,"duration_ms":77909,"concrete_test":"Train SUDO with identical Pick-a-Pic V2 data and hyperparameters, but draw the losing image from a disjoint image pool (e.g., MS-COCO train or a held-out LAION subset) rather than from the training set; include a same-prompt different-seed losing image as a control. If the reported gains over SFT on CLIP, HPS, and ImageReward disappear or reverse, the central premise that random in-dataset images are a valid self-supervised losing signal is not general. Also rerun the main table with at least three random seeds and report mean and standard deviation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Section 3.2 (Eq. 8), SUDO treats a randomly selected training image as the losing sample x^sl = Downgrade(x^w). This is not a downgrade of the winner: it replaces the image with an unrelated, usually high-quality image, destroying text-image correspondence rather than degrading image quality. The paper's own ablation (Table 3) shows that actual degradations (blur, random grid) are harmful and only 'random image' beats SFT; the text even concludes that degrading image quality is detrimental, which is inconsistent with calling the successful operation a downgrade. Consequently, the claimed connection to DPO-style preference optimization is unestablished. The objective likely acts as a contrastive text-image alignment regularizer, yet no theoretical or empirical analysis explains why that should improve global quality. This matters because the headline claim of 'consistently enhancing model performance' is also overstated: on SDXL, Table 2 shows Aesthetics is essentially flat (HPDv2: 61.30 vs 61.34 base; Pick-a-Pic: 59.97 vs 60.01), and DPO beats SUDO on PickScore/CLIP in several settings. No error bars or multiple seeds are reported, so 'significant' is not quantified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces SUDO, a self-supervised direct preference optimization method for fine-tuning text-to-image diffusion models. In each training iteration, the input image is treated as the winning sample and a randomly selected image from the training set is used as the losing sample; the DPO-style loss in Eq. (8) is then added to the standard MSE loss. Experiments on Stable Diffusion 1.5 and SDXL, evaluated on Pick-a-Pic V2, PartiPrompts, and HPDv2 with five automatic metrics, are reported to show that SUDO improves over SFT and the base models and is competitive with Diffusion-DPO, without using preference annotations.","tokens_in":11610,"tokens_out":4495,"duration_ms":38208,"significance":"If the reported results are robust, SUDO offers a practical alternative to SFT for improving text-to-image models with no additional human preference labels, which is a valuable contribution. The paper provides a wide range of quantitative comparisons, ablations on the downgrade operation, and qualitative examples. However, the mechanism behind the method is not clearly established, the reported gains are not consistent across all metrics and models, and the statistical basis of the claims is weak because all numbers are point estimates without error bars or significance tests.","major_comments":[{"comment":"The only 'Downgrade' operation that improves performance is random text-image mismatch, not an image-quality degradation; blur and random grid both perform worse than SFT. This directly contradicts the paper's assertion that the losing sample is obtained by 'degrading the winner' and undermines the interpretation of SUDO as a direct preference optimization method over image quality. The authors should either provide an explicit analysis of why this operation helps (for example, as a contrastive text-image alignment regularizer) or substantially reframe the method and its claims away from 'preference optimization' and 'global image-level learning.'","section":"Section 3.2, Eq. (8) and Table 3"},{"comment":"The claim of 'consistently enhancing model performance' and of 'significant enhancements' compared to DPO is overstated. On SDXL, SUDO's PickScore is lower than DPO on all three evaluation sets (22.34 vs 22.57 on Pick-a-Pic V2, 22.79 vs 22.90 on PartiPrompts, 22.98 vs 23.18 on HPDv2), and SUDO is below the base model on Aesthetics on Pick-a-Pic (59.97 vs 60.01). Win rates also favor DPO on several metrics (e.g., Pick-a-Pic V2 PickScore win rate 60.80 vs 72.60). The text should be revised to describe SUDO as competitive with DPO on some metrics and superior on others, not uniformly better.","section":"Table 2"},{"comment":"No error bars, multiple seeds, or significance tests are reported for any of the quantitative results. The terms 'significant', 'substantially higher', and 'far exceeds' are used throughout without statistical quantification. The authors should report standard deviations over at least three seeds or paired significance tests for the main comparisons, especially for the win rates, before claiming consistent improvements.","section":"Section 4.2 and Table 2"}],"minor_comments":[{"comment":"The description of SUDO's data requirement as 'standard image-text pairs' is imprecise because the training data is the Pick-a-Pic V2 dataset, which was collected through user preferences; only the preference annotations are unused. This should be clarified to avoid implying the dataset is arbitrary image-text pairs.","section":"Table 1 and Section 4.1"},{"comment":"The sentence 'We provide qualitative comparisons in Figure 4 and Figure 1 for SD1.5 and SDXL' is incorrect; the SDXL qualitative results appear in Figure 5 in the main text and Appendix Figures 1 and 2, not in Figure 1. Please correct the figure cross-references.","section":"Section 4.3"},{"comment":"Equation (8) is typeset with broken line breaks and subscripts in the submitted manuscript, which makes the loss definition hard to read. In addition, there are typographical errors such as 'wining' instead of 'winning', 'encourages' instead of 'encourages', and 'demonstraing' instead of 'demonstrating'.","section":"Section 3.2 and Eq. (8)"},{"comment":"The 'Win rate' rows are not defined in the caption or text; please specify how a win is determined for each metric, as this is important for interpreting the pairwise comparisons.","section":"Table 2"},{"comment":"The limitation that the random-image strategy 'may prove ineffective' for very small datasets is stated without supporting evidence; a small-scale experiment varying dataset size would make this claim more concrete.","section":"Section 5"}],"recommendation":"major_revision","confidential_remarks":"The manuscript addresses a timely and practically relevant problem in text-to-image diffusion fine-tuning, and the empirical framework is broadly sensible. However, the gap between the reported method's interpretation and its actual operation is substantial, and the quantitative claims are overclaimed relative to the tables. I believe the paper can be repaired with a major revision that reframes the contribution, adds statistical support, and tempers the claims; I do not see a load-bearing error that would require rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Hi [Colleague],\n\nQuick take on arXiv:2504.14534 (SUDO). The paper's idea is neat: instead of collecting human preference pairs for DPO, construct the 'losing' image by randomly picking another image from the training set and pairing it with the prompt. That is genuinely new as far as I know, and the experiments are extensive: SD1.5 and SDXL, three eval sets, five metrics, plus ablations. The method is simple, the writing is clear, and the code link is provided. So the package is a plausible contribution to the fine-tuning toolbox.\n\nThe soft spot is the one the stress-test flags, and it's real. Calling the random image a 'downgrade' is a stretch. A random training image is not a degraded version of the winner; it's a mismatched sample that breaks text-image correspondence. The paper's own ablation shows that actual degradations (blur, random grid) hurt, and the text even says that degrading image quality is detrimental. So the benefit comes specifically from the random-mismatch operation, and the DPO framing doesn't explain why that should improve global quality. The objective probably acts as a contrastive text-image alignment regularizer. That could be a fine method, but the current story overclaims the 'preference optimization' connection.\n\nAlso worth flagging: all numbers are point estimates. No error bars, no significance tests, no multiple seeds. On SDXL the gains are modest on Aesthetics (essentially flat), and DPO beats SUDO on PickScore/CLIP in several rows. The post-hoc selection of 'random image' after comparing options on the same eval sets (Table 3) is a selection risk, though not disqualifying.\n\nThe paper is not circular in a damaging way: the DPO loss is from prior work, and the hyperparameters are tuning choices. The limitation section is honest about the dataset-size issue.\n\nVerdict: worth engaging with, but it needs a serious referee who will push for error bars and a clearer story about what the loss is actually doing. The central claim is plausible but not stringently supported.\n\nBest,\n[Name]","headline":"Useful self-supervised pair construction for diffusion DPO, but the 'downgrade' framing is misleading and the numbers lack error bars.","tokens_in":12083,"tokens_out":2616,"would_cite":true,"duration_ms":22504,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SUDO lifts text-to-image diffusion models with self-made preference pairs, replacing costly human annotation.","keywords":["text-to-image generation","diffusion models","direct preference optimization","self-supervised learning","fine-tuning","Stable Diffusion","image-level optimization","preference pairs"],"falsifier":"A controlled comparison could settle it: fine-tune with SUDO but replace the random-image loser with a same-prompt image generated by the base model, a true near-miss negative; if the gains disappear or reverse, the reported improvement comes from contrastive regularization against unrelated images rather than from preference optimization.","tokens_in":11134,"feed_emoji":"🖼️","tokens_out":6313,"duration_ms":47853,"temperature":0.7,"pith_summary":"The paper proposes SUDO, a fine-tuning method for text-to-image diffusion models that replaces human-annotated preference pairs with self-generated ones. Taking the training image as the 'winning' sample and a randomly selected, mismatched image as the 'losing' sample, SUDO applies a direct preference optimization objective at the image level while keeping the standard pixel-level mean-squared-error loss. The claim is that this joint optimization improves both local detail and global image quality, and the experiments on Stable Diffusion 1.5 and SDXL across three evaluation sets show consistent gains over supervised fine-tuning, with most metrics at or above the human-annotated DPO baseline. If correct, SUDO offers a drop-in replacement for SFT that needs no extra data collection or annotation.","feed_headline":"SUDO lifts diffusion image models with self-made preference pairs","feed_subtitle":"Randomly mismatched images act as 'losing' samples, improving quality and text alignment without human labels.","key_machinery":"The load-bearing object is the self-supervised preference pair created by the Downgrade operation. In each iteration the correct image $x^w$ is the winner and $x^{sl} = \\mathrm{Downgrade}(x^w)$ is a randomly sampled image from the training set that does not correspond to the prompt. The loss is $$L_{\\mathrm{SUDO}} = -\\log\\$\\sigma$\\!\\left(C\\left[\\|\\epsilon_\\$\\theta$(x^w_t,t)-\\epsilon^w\\|$_2^{2}$ - \\|\\epsilon_\\$\\theta$($x^{{sl}}$_t,t)-\\$epsilon^{{sl}}$\\|$_2^{2}$ - \\left(\\|\\epsilon_{\\mathrm{ref}}(x^w_t,t)-\\epsilon^w\\|$_2^{2}$ - \\|\\epsilon_{\\mathrm{ref}}($x^{{sl}}$_t,t)-\\$epsilon^{{sl}}$\\|$_2^{2}$\\right)\\right]\\right),$$ with $C$ a scale factor, and the total loss is $L = \\lambda_1 L_{\\mathrm{MSE}} + \\lambda_2 L_{\\mathrm{SUDO}}$. The objective widens the gap between the model's denoising error on the matching pair and its error on the mismatched pair, relative to the reference model, and this gap is what carries the image-level learning.","core_discovery":"The central discovery is that the expensive preference labels in DPO can be replaced by a free, self-supervised signal: any image that does not match the prompt is a valid losing sample. Formally, with $x^w$ the correct text-image pair and $x^{sl} = \\mathrm{Downgrade}(x^w)$ obtained by randomly drawing a mismatched image from the training set, SUDO maximizes the log-likelihood of preferring $x^w$ over $x^{sl}$ in a DPO-style objective. The paper reports that this objective, combined with the MSE loss, consistently improves the base models on Pick-a-Pic V2, PartiPrompts, and HPDv2, measured by PickScore, Aesthetics, CLIP, HPS v2, and ImageReward. It also reports that the random-image downgrade is essential: blurring or randomly swapping grid patches makes performance worse than SFT.","pith_inferences":["A testable implication the authors do not pursue: if the benefit comes from the model learning to separate matched from mismatched pairs, then harder negatives, images that are semantically related to the prompt but wrong, should give larger gains than uniformly random images.","Since the paper notes the random-image strategy may fail on very small training sets, a natural extension is curriculum or retrieved negatives from a larger pool, which would also clarify whether the signal is pure contrastive regularization rather than preference optimization.","The paper's framing as 'preference' optimization is one interpretation; equally consistent with the data is that SUDO acts as a form of data augmentation or regularization that prevents fine-tuning from overfitting to the pixel-level MSE loss. A comparison against a standard contrastive loss on the same pairs would separate these explanations."],"forward_implications":["SUDO can be applied to any text-to-image diffusion model without collecting human preference annotations, making image-level alignment as cheap as ordinary supervised fine-tuning.","On SD1.5, SUDO improves all five metrics over the base model on all three evaluation sets, with ImageReward rising from -14.81 to 71.00 on Pick-a-Pic V2, far above SFT's 45.03 and DPO's 4.13.","On SDXL, where SFT degrades the base model, SUDO still improves most metrics and reaches HPS win rates above 92 percent on all three sets.","The random-image downgrade is not interchangeable: blur and random-grid downgrades fall below SFT, so the specific choice of losing sample matters.","Removing the MSE loss leaves SUDO's gains largely intact, indicating the image-level preference term alone carries most of the benefit."],"supporting_citations":[{"why":"Supplies the DPO-for-diffusion objective that SUDO adapts, and the DPO baseline and implementation details it compares against.","marker":"[37]"},{"why":"Provides the direct preference optimization formulation that SUDO extends to self-supervised pairs.","marker":"[31]"},{"why":"Pick-a-Pic V2 is the training source (prompts and images) and one of the evaluation sets; SUDO uses only its text-image pairs.","marker":"[19]"},{"why":"Stable Diffusion 1.5 is one of the two base models fine-tuned and evaluated.","marker":"[32]"},{"why":"SDXL is the second base model fine-tuned and evaluated.","marker":"[28]"},{"why":"HPS v2 is used as an evaluation metric over the three test sets.","marker":"[38]"},{"why":"ImageReward is one of the five automatic preference metrics used to report gains.","marker":"[40]"},{"why":"CLIP score is used as the text-alignment metric in the evaluation.","marker":"[30]"}],"fun_headline_variants":["SUDO: Self-made preference pairs boost diffusion image quality","No human labels: SUDO trains diffusion models with mismatched images","SUDO's trick: random mismatched images as preference negatives","Self-supervised SUDO improves text-to-image diffusion without labels","Mismatched images teach diffusion models better than fine-tuning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a randomly selected training image is a valid 'losing' sample for any prompt, so widening the model's denoising-error gap between the correct pair and the mismatched pair improves generation quality; the paper offers no theoretical argument for why this particular signal helps, and its own ablations show other downgrades are harmful.","fun_headline_variants_meta":{"raw":{"variants":["SUDO: Self-made preference pairs boost diffusion image quality","No human labels: SUDO trains diffusion models with mismatched images","SUDO's trick: random mismatched images as preference negatives","Self-supervised SUDO improves text-to-image diffusion without labels","Mismatched images teach diffusion models better than fine-tuning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000211,"raw_usage":{"total_tokens":1421,"prompt_tokens":956,"completion_tokens":465,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":572,"completion_tokens_details":{"reasoning_tokens":378}},"tokens_in":572,"tokens_out":465,"duration_ms":4361,"temperature":1.0,"reasoning_tokens":378,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:45:47.261666+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A controlled comparison could settle it: fine-tune with SUDO but replace the random-image loser with a same-prompt image generated by the base model, a true near-miss negative; if the gains disappear or reverse, the reported improvement comes from contrastive regularization against unrelated images rather than from preference optimization.","supporting_citations":[{"cited_title":"Diffusion model align- ment using direct preference optimization","cited_arxiv_id":null,"evidence_quote":"Supplies the DPO-for-diffusion objective that SUDO adapts, and the DPO baseline and implementation details it compares against."},{"cited_title":"Direct preference optimization: Your language model is secretly a reward model","cited_arxiv_id":null,"evidence_quote":"Provides the direct preference optimization formulation that SUDO extends to self-supervised pairs."},{"cited_title":"Pick-a-pic: An open dataset of user preferences for text-to-image generation","cited_arxiv_id":null,"evidence_quote":"Pick-a-Pic V2 is the training source (prompts and images) and one of the evaluation sets; SUDO uses only its text-image pairs."},{"cited_title":"High-resolution image synthesis with latent diffusion models","cited_arxiv_id":null,"evidence_quote":"Stable Diffusion 1.5 is one of the two base models fine-tuned and evaluated."}],"review_version":1}