{"id":"df06824c-e35a-441f-99a1-d307f9fafc7d","arxiv_id":"2412.01794","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"An adapter module trained on IQA/IAA scores gives SDXL controllable quality-aware generation, improving perceived quality and enabling reference-based distortion transfer.","lead":"This paper adds a small adapter to SDXL that conditions image generation on user-specified quality scores from image quality assessment models, letting users generate sharper, cleaner images or deliberately degraded ones. It reports about 7-9% average objective quality gains, a 60% human win rate over the base model, and successful transfer of specific distortions from a reference image.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reference-based distortion-transfer SROCC is computed with ARNIQA, the same model whose activations condition the adapter, making the 0.80 vs 0.41 comparison partly self-fulfilling.","rationale":"The reader's weaker assumption focuses on the latent-vs-pixel training discrepancy, which the authors explicitly acknowledge and which the empirical results (including a human preference study) suggest does not prevent the adapter from working. The most load-bearing concern is instead the reference-based sub-claim: the SROCC 0.80 versus 0.41 comparison is computed with ARNIQA, the same model whose activations condition the adapter, creating a circularity that could inflate the apparent distortion-transfer advantage. The paper provides some independent support (the non-ARNIQA TOPIQ+LAION-AES variant still achieves high SROCC under ARNIQA, and qualitative examples show distinct distortions), so this is not a reason to reject the paper, but it is a concrete weakness in a headline quantitative result. My proposed test (using an independent evaluation metric and a distortion-type classifier) would settle the concern. The reader's verdict of CONDITIONAL, driven by missing artifacts and evaluation-circularity concerns, remains appropriate; my concern adds a specific instance of that circularity without changing the overall verdict.","tokens_in":31259,"tokens_out":5420,"duration_ms":58192,"concrete_test":"Recompute Table 2's SROCC on the KADID-10k setup using an IQA model not involved in adapter training (e.g., MANIQA or a FR metric such as DISTS computed between the generated image and the distortion reference), and additionally measure distortion-type classification accuracy with a classifier trained on KADID's 25 distortion labels. If the Ref.-based IQA-Adapter's margin over IP-Adapter under the independent metric and the type-accuracy metric shrinks to statistical parity, the superior distortion-transfer claim is largely an artifact of aligning with the conditioning model's activation space.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest quantitative claim in the reference-based section is that Ref.-based IQA-Adapter transfers distortion types with SROCC 0.80 versus 0.41 for IP-Adapter (Table 2). The SROCC is computed with ARNIQA scores on generated and reference images. The Ref.-based IQA-Adapter is trained to map ARNIQA activations into the denoising process, so it is directly optimized to make outputs align with ARNIQA's scoring. IP-Adapter and StyleCrafter have no access to ARNIQA at all, making the comparison asymmetric: the presence of the evaluator's feature space in the conditioning loop can inflate the measured correlation independently of whether true perceptual distortion transfer occurs. The paper partially mitigates this by reporting IQA-Adapter(TOPIQ+LAION-AES) with SROCC 0.76/0.79, which does not use ARNIQA embeddings and still outperforms IP-Adapter under ARNIQA evaluation; this suggests genuine transfer. However, SROCC on ARNIQA scores only measures ordering of severity, not whether the correct distortion type (blur vs JPEG vs noise) is transferred. A model that produces blurry images of varying strength could achieve high SROCC if ARNIQA orders blur severity monotonically, without ever reproducing JPEG artifacts. The qualitative figures illustrate type transfer, but the headline numeric comparison is vulnerable to circularity because the conditioning source and the evaluation metric are the same model.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes IQA-Adapter, a lightweight adapter for SDXL that conditions diffusion generation on target image quality/aesthetic (IQA/IAA) scores via an additional quality cross-attention layer, and a reference-based variant that conditions on the activation space of an IQA model (ARNIQA). The central claims are: (1) high-quality conditioning shifts the output distribution to a higher-quality subdomain with 7–9% average relative gains across objective metrics and a 60% human preference win rate over the base model; (2) low-quality conditioning acts as a controllable degradation model; (3) reference-based conditioning transfers distortion type and severity with SROCC 0.80 versus 0.41 for IP-Adapter. The paper also reports GenEval, FID, IS, CLIP, and a large subjective study, plus ablations of the separate attention design, adapter scale, negative guidance, and positional encoding.","tokens_in":31670,"tokens_out":4754,"duration_ms":51377,"significance":"If the results hold, this is a useful and reasonably novel contribution: it is one of the first systematic attempts to inject IQA/IAA knowledge into a diffusion generator through conditioning, and it ships an extensive empirical package, including a subjective study with 22,300+ responses across 1,017 users, a 21-model IQA/IAA sweep, and open code/weights. The separate-quality-attention architecture and the degradation-model interpretation are valuable. The paper's main high-quality-conditioning claim is supported by a human preference study and by cross-metric objective gains, which break the most naive form of circularity (training and evaluation on the same metric). The reference-based distortion-transfer claim, however, is partially undermined by an evaluator/conditioning overlap and by the lack of distortion-type-specific evidence. Overall the contribution is significant and worth publishing after the reference-based evaluation is strengthened and the statistical reporting is improved.","major_comments":[{"comment":"The headline SROCC comparison (Ref.-based IQA-Adapter 0.80 versus IP-Adapter 0.41) is partly circular. The Ref.-based IQA-Adapter is trained to map ARNIQA activations into the denoising process, and the SROCC is computed with ARNIQA scores on the generated images; IP-Adapter and StyleCrafter have no access to ARNIQA. The rows for IQA-Adapter(TOPIQ+LAION-AES) partially mitigate this, but the paper should either report SROCC for all methods using at least two independent IQA models (e.g., TOPIQ, MANIQA, DBCNN) or provide human judgments of distortion-type transfer. In addition, SROCC measures only ordering of overall severity, not whether the correct distortion type (blur vs JPEG vs noise) is reproduced; please add a per-distortion-type analysis or a qualitative/quantitative type-confusion matrix to support the 'transfers highly specific qualitative features' claim.","section":"Section 4.5, Table 2"},{"comment":"The paper claims 'consistent' quality gains of 7–9% and reports SROCC values in Table 2, but no confidence intervals or significance tests are provided for the relative gains or for the SROCC differences. The seed-consistency experiment in Supplementary Sec. 17.2 (2,500 generations) already provides the distribution needed to compute confidence intervals; please add error bars or CIs to Figure 3(a) and Table 2, and state whether the differences between IQA-Adapter variants and between IQA-Adapter and baselines are statistically significant. Without this, the ranking of methods in Figure 3(a) and the SROCC 0.80 vs 0.41 gap cannot be interpreted reliably.","section":"Section 4.2, Figure 3(a), and Table 2"},{"comment":"The paper acknowledges that IQA-Adapter training occurs in latent space while the supervision scores are computed in pixel space, and that the VAE decoder can introduce artifacts that the adapter cannot correct. This is a load-bearing assumption for the entire training objective, yet no quantification or verification is provided. Please add a direct check, for example: (a) the correlation between IQA scores computed on original images versus VAE-reconstructed images, or (b) a small experiment where the adapter is conditioned on scores computed from latents (or from VAE reconstructions) rather than from the clean pixel image, to show that the quality signal survives the latent bottleneck. As written, the limitation is asserted but its practical impact on the results is not measured.","section":"Supplementary Sec. 10 (Limitations)"},{"comment":"The RelGain metric in Eq. (1) assumes the same prompt and seed for x and x', but this is not meaningful for methods that change the prompt itself, such as BeautifulPrompt and Prompt Weighting (which modify the text). For those methods, the comparison is between images generated from different prompts, so the relative gain is not directly attributable to quality conditioning. Please specify how x is defined for these methods (e.g., the base model output on the original prompt) and whether the seed is held fixed, or otherwise separate the prompt-effect from the quality-effect to make the comparison fair.","section":"Section 4.2, Eq. (1)"}],"minor_comments":[{"comment":"The gradient-based guidance equation uses D(zt) without stating that backpropagation must pass through the VAE decoder; please clarify that this is a VAE decoder and that the gradient is with respect to zt through D.","section":"Eq. (2), Section 3.1"},{"comment":"The row label '3xARNIQA,LIQE-MIX(different datasets)' is unclear; please spell out which ARNIQA variants are combined and how the '3x' weighting is applied.","section":"Table 1 and Table 6"},{"comment":"The definition of win rate as 'share of image pairs on which it achieved the majority of votes' should specify how ties ('No difference' responses) are handled in the majority calculation.","section":"Section 4.3, Figure 3(b)"},{"comment":"The term 'Qualitative Attention' is slightly confusing; consider 'Quality Attention' to match the 'quality tokens' terminology used later.","section":"Section 3.2.1"},{"comment":"The best result in the SDEdit I2I section is IQA-Adapter(TOPIQ+LAION-AES) with SROCC 0.79, not the Ref.-based IQA-Adapter (0.69); please reflect this in the text, which currently emphasizes only the T2I result.","section":"Section 4.5, Table 2"},{"comment":"The training section states 'For Reference-based IQA-Adapter, we apply series of degradations to training images with a probability p = 0.1' but does not define the degradation model; it refers to [43] later, but please make the reference explicit at this point.","section":"Supplementary Sec. 9"}],"recommendation":"major_revision","confidential_remarks":"The paper's core contribution is sound and well-supported on the high-quality-conditioning side, but the reference-based distortion-transfer section needs a substantial rework to address the ARNIQA circularity and to provide distortion-type-specific evidence. The lack of confidence intervals is also a concern for a journal submission. With those revisions, the paper would be a solid fit for the journal. The authors should also be encouraged to release the training/inference code and weights as promised, as that will materially strengthen the reproducibility of the claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This one is worth a serious look. The core result is that a lightweight adapter with its own cross-attention layer can condition SDXL on continuous IQA/IAA scores, shifting output quality up or down in a controlled way. The 22,300-response human study and the cross-metric gains are the evidence that matters; they show the effect is not just the training metric rewarding its own reflection.\n\nWhat is actually new: separate quality cross-attention, qualitative negative guidance, and reference-based conditioning from IQA activations form a combination I have not seen before. The degradation-model direction is a nice bonus. The paper is also unusually thorough: 21 IQA/IAA models, ablations on architecture and positional encoding, computational overhead, seed consistency, and honest limitations. Supplementary Sec. 10 explicitly flags the latent-vs-pixel space mismatch and VAE decoder artifacts, which is more candor than most papers manage. The ablations also demonstrate that a simplified token-concatenation adapter loses qualitative control (SROCC drops from 0.97 to 0.26), which supports the design choices.\n\nThe soft spots are real but not load-bearing. First, evaluation circularity: gains on the target metric are expected by construction, but the cross-metric gains, FID/GenEval results, and the human study break that circularity and ground the central claim. Second, the reference-based SROCC 0.80 vs 0.41 is partly self-fulfilling because ARNIQA's activations condition the adapter and ARNIQA scores are used for SROCC. The mitigation is genuine: the TOPIQ+LAION-AES adapter, which never sees ARNIQA, still reaches 0.76/0.79 under ARNIQA evaluation. But SROCC only measures ordering of severity, not whether the correct distortion type is transferred; the qualitative figures carry that load. I would ask the authors for a per-distortion-type accuracy table and for SROCC computed with a different IQA model than the conditioning one. Several tables also lack confidence intervals, and the artifacts are promised but not yet released, which matters for a method paper.\n\nOverall, the central claim holds up. It is not a conceptual breakthrough, but it is a useful and reproducible-in-principle method with a clear application niche. It deserves a serious referee. If the authors release code and weights and address the reference-transfer circularity with a non-ARNIQA evaluation, this becomes a solid accept rather than a conditional one.","headline":"Solid applied contribution: quality-conditioned diffusion via an adapter is real and well-evaluated, but the headline reference-transfer number is partly circular and the code/weights are not out yet.","tokens_in":32026,"tokens_out":1825,"would_cite":true,"duration_ms":20533,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"IQA-Adapter conditions SDXL image generation on target quality scores, shifting outputs into a higher-quality subdomain while preserving prompt following.","keywords":["Image Quality Assessment","Image Aesthetic Assessment","IQA-Adapter","Diffusion models","Quality-aware generation","Cross-attention conditioning","Reference-based degradation transfer","SDXL"],"falsifier":"Train an IQA-Adapter exactly as described but evaluate it after swapping in a VAE decoder that removes compression artifacts and high-frequency noise; if the 7-9% relative quality gains and the SROCC 0.80 distortion transfer vanish, the central claim that pixel-space IQA knowledge transfers through latent-space conditioning would be falsified.","tokens_in":31077,"feed_emoji":"🎨","tokens_out":6008,"duration_ms":59486,"temperature":0.7,"pith_summary":"This paper tries to give diffusion image generators a dial for perceptual quality. The authors propose IQA-Adapter, a lightweight adapter inserted into SDXL that takes target scores from image quality and aesthetic assessment models and uses a separate cross-attention layer to steer generation. Conditioned on the 99th percentile of a target metric, the adapter shifts SDXL outputs into a higher-quality subdomain, with average 7-9% relative gains across objective metrics and a 60% win rate in a human preference study; at low percentiles it progressively degrades images, reproducing artifacts like JPEG compression and blur. The same architecture supports reference-based conditioning, where activations of an IQA model transfer specific, content-agnostic distortion types with SROCC 0.80, outperforming IP-Adapter's 0.41. If correct, this gives a way to control output quality without retraining the frozen base model, and it reverses the usual direction of IQA research by moving assessment knowledge into the generator.","feed_headline":"Quality-aware adapter lifts SDXL output quality 7-9 percent","feed_subtitle":"Conditioning on IQA scores also turns the base model into a controllable degradation generator.","key_machinery":"The load-bearing object is a separate qualitative cross-attention layer added alongside each textual cross-attention layer in SDXL's U-Net. Quality scores, standardized to zero mean and unit variance, are projected through a linear layer and LayerNorm into tokens $c_q$, then attended to with a query $Q=ZW_q$ shared with the textual attention and new keys and values $K'=c_qW'_k$, $V'=c_qW'_v$: $$\\mathrm{CrossAttn}(Z,c_t,c_q)=\\mathrm{Softmax}(QK^\\top/\\sqrt{d})V+\\$\\lambda$\\,\\mathrm{Softmax}(QK'^\\top/\\sqrt{d})V'.$$ The scale $\\lambda$ controls adapter strength, and qualitative negative guidance uses $q_{\\mathrm{neg}}=-\\delta\\,q$ inside classifier-free guidance. This design keeps quality information separate from prompt semantics, so the adapter can apply quality attributes content-agnostically. For reference-based conditioning, the qualitative tokens come from an IQA model's intermediate activations rather than from scores, exploiting layers whose semantic content is minimal.","core_discovery":"The paper's central claim is that a frozen diffusion model can learn complex qualitative relationships from an IQA model's outputs and internal activations through a lightweight adapter, and that this learned conditioning transfers across metrics and prompts. The authors show that inference-time gradient guidance inflates the target metric by exploiting model-specific vulnerabilities, producing adversarial patterns, whereas IQA-Adapter, trained only on scores with no backpropagation through the IQA model, yields consistent 7-9% relative quality gains across many metrics, retains GenEval prompt-following near the base level, and lets users modulate quality across a wide spectrum from the 1st to the 99th percentile. They further claim that the adapter's separate qualitative attention disentangles quality from text, and that when reference-based conditioning uses farthest-layer IQA activations, such as those of ARNIQA, the model transfers distortion type and strength rather than semantics, with distortion-transfer SROCC 0.80 in text-to-image generation versus 0.41 for IP-Adapter.","pith_inferences":["An implication the authors leave implicit is that the same separate-attention mechanism could condition on any continuous semantic attribute, such as safety, toxicity, or style intensity, by replacing the IQA scores with another predictor's outputs.","If the reference-based result generalizes beyond the KADID-10k distortion set, the adapter offers a cheap way to build paired degradation datasets at scale for restoration model pretraining, a testable extension the paper mentions only briefly.","The sharp contrast between gradient guidance and adapter training suggests that quality-conditioned generation could serve as an adversarial robustness probe for IQA models, exposing vulnerabilities that pure metric evaluation misses.","The latent-versus-pixel supervision gap flagged in the supplementary implies that adapting the same idea to a latent-native quality model would be a direct stress test of the transfer mechanism."],"forward_implications":["Quality-conditioned generation becomes a practical dial: users can request the 99th percentile of a target metric and get images that humans prefer roughly 60% of the time over the base model, or request low percentiles to obtain a controllable degradation model.","Because training only needs scores and not gradients, IQA-Adapter can be trained with non-differentiable metrics or human subjective scores, broadening the set of usable quality signals.","The adapter combines with existing quality-improvement methods: pairing it with DiffusionDPO and BeautifulPrompt yields the best GenEval overall scores in the paper's comparisons.","Reference-based distortion transfer with SROCC 0.80 suggests diffusion generators can serve as data generators for restoration and enhancement by synthesizing paired high-quality and degraded images, an application the paper identifies as future work.","Since only adapter weights change, the base model's generative capabilities remain close to the original, so quality control does not come at the cost of prompt following."],"supporting_citations":[{"why":"Supplies the frozen SDXL base model whose U-Net receives the adapter's separate cross-attention layer.","marker":"[5]"},{"why":"TOPIQ is the primary IQA model used both in training and in evaluating the adapter's high-quality conditioning.","marker":"[1]"},{"why":"ARNIQA supplies the activations for reference-based conditioning and the degradation augmentation used during training.","marker":"[43]"},{"why":"IP-Adapter is the main baseline in the reference-based distortion-transfer comparison, achieving SROCC 0.41.","marker":"[7]"},{"why":"DiffusionDPO is the fine-tuning baseline that the IQA-Adapter is shown to combine with for the best GenEval scores.","marker":"[58]"},{"why":"The DDPM denoising objective defines the training loss for the IQA-Adapter.","marker":"[66]"},{"why":"CC3M provides the roughly three million text-image pairs used for the main training stage of the adapter.","marker":"[67]"},{"why":"LAION-5B and its improved-aesthetics subset provide the fine-tuning data at 1024x1024 resolution.","marker":"[69]"},{"why":"SDEdit is used to set up the Image-to-Image evaluation of reference-based distortion transfer.","marker":"[83]"}],"fun_headline_variants":["IQA-Adapter lifts diffusion output quality by up to 10%","Train a quality adapter to shift diffusion images high or low","Reference-based IQA adapter transfers distortion patterns","Control diffusion quality from 1st to 99th percentile","Adapter turns diffusion model into controllable degrader"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that pixel-space IQA scores, computed on decoded images, remain a reliable supervision signal while the adapter learns entirely in SDXL's latent space; if the VAE's latents lose or distort the quality cues the IQA models respond to, the learned conditioning would be unstable and would not transfer.","fun_headline_variants_meta":{"raw":{"variants":["IQA-Adapter lifts diffusion output quality by up to 10%","Train a quality adapter to shift diffusion images high or low","Reference-based IQA adapter transfers distortion patterns","Control diffusion quality from 1st to 99th percentile","Adapter turns diffusion model into controllable degrader"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000293,"raw_usage":{"total_tokens":1728,"prompt_tokens":988,"completion_tokens":740,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":604,"completion_tokens_details":{"reasoning_tokens":660}},"tokens_in":604,"tokens_out":740,"duration_ms":7693,"temperature":1.0,"reasoning_tokens":660,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T00:55:15.021589+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train an IQA-Adapter exactly as described but evaluate it after swapping in a VAE decoder that removes compression artifacts and high-frequency noise; if the 7-9% relative quality gains and the SROCC 0.80 distortion transfer vanish, the central claim that pixel-space IQA knowledge transfers through latent-space conditioning would be falsified.","supporting_citations":[{"cited_title":"Diffusion model alignment using direct preference optimization","cited_arxiv_id":null,"evidence_quote":"DiffusionDPO is the fine-tuning baseline that the IQA-Adapter is shown to combine with for the best GenEval scores."},{"cited_title":"Laion-5b: an open large-scale dataset for training next generation image-text models","cited_arxiv_id":null,"evidence_quote":"LAION-5B and its improved-aesthetics subset provide the fine-tuning data at 1024x1024 resolution."},{"cited_title":"SDEdit: Guided image synthesis and editing with stochastic differ- ential equations","cited_arxiv_id":null,"evidence_quote":"SDEdit is used to set up the Image-to-Image evaluation of reference-based distortion transfer."}],"review_version":1}