Pith. sign in

REVIEW 4 major objections 6 minor 115 references

IQA-Adapter: Exploring Knowledge Transfer from Image Quality Assessment to Diffusion-based Generative Models

T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read IQA-Adapter conditions SDXL image generation on target quality scores, shifting outputs into a higher-quality subdomain while preserving prompt following.

desk verdict Solid applied contribution: quality-conditioned diffusion via an adapter is real and well-evaluated, but the headline reference-transfer number is partly circular and the code/weights are not out yet. read the letter →

arxiv 2412.01794 v2 pith:DALQ5M6B submitted 2024-12-02 cs.CV cs.AI

Khaled Abud , Sergey Lavrushkin , Alexey Kirillov , Dmitriy Vatolin This is my paper · ORCID
classification cs.CVcs.AI
keywords ImageQualityAssessmentAestheticIQA-AdapterDiffusionmodelsQuality-awaregenerationCross-attentionconditioningReference-baseddegradationtransferSDXL
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to give diffusion image generators a dial for perceptual quality. The authors propose IQA-Adapter, a lightweight adapter inserted into SDXL that takes target scores from image quality and aesthetic assessment models and uses a separate cross-attention layer to steer generation. Conditioned on the 99th percentile of a target metric, the adapter shifts SDXL outputs into a higher-quality subdomain, with average 7-9% relative gains across objective metrics and a 60% win rate in a human preference study; at low percentiles it progressively degrades images, reproducing artifacts like JPEG compression and blur. The same architecture supports reference-based conditioning, where activations of an IQA model transfer specific, content-agnostic distortion types with SROCC 0.80, outperforming IP-Adapter's 0.41. If correct, this gives a way to control output quality without retraining the frozen base model, and it reverses the usual direction of IQA research by moving assessment knowledge into the generator.

What carries the argument

The load-bearing object is a separate qualitative cross-attention layer added alongside each textual cross-attention layer in SDXL's U-Net. Quality scores, standardized to zero mean and unit variance, are projected through a linear layer and LayerNorm into tokens $c_q$, then attended to with a query $Q=ZW_q$ shared with the textual attention and new keys and values $K'=c_qW'_k$, $V'=c_qW'_v$: $$\mathrm{CrossAttn}(Z,c_t,c_q)=\mathrm{Softmax}(QK^\top/\sqrt{d})V+\$\lambda$\,\mathrm{Softmax}(QK'^\top/\sqrt{d})V'.$$ The scale $\lambda$ controls adapter strength, and qualitative negative guidance uses $q_{\mathrm{neg}}=-\delta\,q$ inside classifier-free guidance. This design keeps quality information separate from prompt semantics, so the adapter can apply quality attributes content-agnostically. For reference-based conditioning, the qualitative tokens come from an IQA model's intermediate activations rather than from scores, exploiting layers whose semantic content is minimal.

What would settle it

Train an IQA-Adapter exactly as described but evaluate it after swapping in a VAE decoder that removes compression artifacts and high-frequency noise; if the 7-9% relative quality gains and the SROCC 0.80 distortion transfer vanish, the central claim that pixel-space IQA knowledge transfers through latent-space conditioning would be falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that a frozen diffusion model can learn complex qualitative relationships from an IQA model's outputs and internal activations through a lightweight adapter, and that this learned conditioning transfers across metrics and prompts. The authors show that inference-time gradient guidance inflates the target metric by exploiting model-specific vulnerabilities, producing adversarial patterns, whereas IQA-Adapter, trained only on scores with no backpropagation through the IQA model, yields consistent 7-9% relative quality gains across many metrics, retains GenEval prompt-following near the base level, and lets users modulate quality across a wide spectrum from the 1st to the 99th percentile. They further claim that the adapter's separate qualitative attention disentangles quality from text, and that when reference-based conditioning uses farthest-layer IQA activations, such as those of ARNIQA, the model transfers distortion type and strength rather than semantics, with distortion-transfer SROCC 0.80 in text-to-image generation versus 0.41 for IP-Adapter.

Load-bearing premise

The method assumes that pixel-space IQA scores, computed on decoded images, remain a reliable supervision signal while the adapter learns entirely in SDXL's latent space; if the VAE's latents lose or distort the quality cues the IQA models respond to, the learned conditioning would be unstable and would not transfer.

Editorial extensions

If this is right

  • Quality-conditioned generation becomes a practical dial: users can request the 99th percentile of a target metric and get images that humans prefer roughly 60% of the time over the base model, or request low percentiles to obtain a controllable degradation model.
  • Because training only needs scores and not gradients, IQA-Adapter can be trained with non-differentiable metrics or human subjective scores, broadening the set of usable quality signals.
  • The adapter combines with existing quality-improvement methods: pairing it with DiffusionDPO and BeautifulPrompt yields the best GenEval overall scores in the paper's comparisons.
  • Reference-based distortion transfer with SROCC 0.80 suggests diffusion generators can serve as data generators for restoration and enhancement by synthesizing paired high-quality and degraded images, an application the paper identifies as future work.
  • Since only adapter weights change, the base model's generative capabilities remain close to the original, so quality control does not come at the cost of prompt following.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the authors leave implicit is that the same separate-attention mechanism could condition on any continuous semantic attribute, such as safety, toxicity, or style intensity, by replacing the IQA scores with another predictor's outputs.
  • If the reference-based result generalizes beyond the KADID-10k distortion set, the adapter offers a cheap way to build paired degradation datasets at scale for restoration model pretraining, a testable extension the paper mentions only briefly.
  • The sharp contrast between gradient guidance and adapter training suggests that quality-conditioned generation could serve as an adversarial robustness probe for IQA models, exposing vulnerabilities that pure metric evaluation misses.
  • The latent-versus-pixel supervision gap flagged in the supplementary implies that adapting the same idea to a latent-native quality model would be a direct stress test of the transfer mechanism.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes IQA-Adapter, a lightweight adapter for SDXL that conditions diffusion generation on target image quality/aesthetic (IQA/IAA) scores via an additional quality cross-attention layer, and a reference-based variant that conditions on the activation space of an IQA model (ARNIQA). The central claims are: (1) high-quality conditioning shifts the output distribution to a higher-quality subdomain with 7–9% average relative gains across objective metrics and a 60% human preference win rate over the base model; (2) low-quality conditioning acts as a controllable degradation model; (3) reference-based conditioning transfers distortion type and severity with SROCC 0.80 versus 0.41 for IP-Adapter. The paper also reports GenEval, FID, IS, CLIP, and a large subjective study, plus ablations of the separate attention design, adapter scale, negative guidance, and positional encoding.

Significance. If the results hold, this is a useful and reasonably novel contribution: it is one of the first systematic attempts to inject IQA/IAA knowledge into a diffusion generator through conditioning, and it ships an extensive empirical package, including a subjective study with 22,300+ responses across 1,017 users, a 21-model IQA/IAA sweep, and open code/weights. The separate-quality-attention architecture and the degradation-model interpretation are valuable. The paper's main high-quality-conditioning claim is supported by a human preference study and by cross-metric objective gains, which break the most naive form of circularity (training and evaluation on the same metric). The reference-based distortion-transfer claim, however, is partially undermined by an evaluator/conditioning overlap and by the lack of distortion-type-specific evidence. Overall the contribution is significant and worth publishing after the reference-based evaluation is strengthened and the statistical reporting is improved.

major comments (4)
  1. [Section 4.5, Table 2] The headline SROCC comparison (Ref.-based IQA-Adapter 0.80 versus IP-Adapter 0.41) is partly circular. The Ref.-based IQA-Adapter is trained to map ARNIQA activations into the denoising process, and the SROCC is computed with ARNIQA scores on the generated images; IP-Adapter and StyleCrafter have no access to ARNIQA. The rows for IQA-Adapter(TOPIQ+LAION-AES) partially mitigate this, but the paper should either report SROCC for all methods using at least two independent IQA models (e.g., TOPIQ, MANIQA, DBCNN) or provide human judgments of distortion-type transfer. In addition, SROCC measures only ordering of overall severity, not whether the correct distortion type (blur vs JPEG vs noise) is reproduced; please add a per-distortion-type analysis or a qualitative/quantitative type-confusion matrix to support the 'transfers highly specific qualitative features' claim.
  2. [Section 4.2, Figure 3(a), and Table 2] The paper claims 'consistent' quality gains of 7–9% and reports SROCC values in Table 2, but no confidence intervals or significance tests are provided for the relative gains or for the SROCC differences. The seed-consistency experiment in Supplementary Sec. 17.2 (2,500 generations) already provides the distribution needed to compute confidence intervals; please add error bars or CIs to Figure 3(a) and Table 2, and state whether the differences between IQA-Adapter variants and between IQA-Adapter and baselines are statistically significant. Without this, the ranking of methods in Figure 3(a) and the SROCC 0.80 vs 0.41 gap cannot be interpreted reliably.
  3. [Supplementary Sec. 10 (Limitations)] The paper acknowledges that IQA-Adapter training occurs in latent space while the supervision scores are computed in pixel space, and that the VAE decoder can introduce artifacts that the adapter cannot correct. This is a load-bearing assumption for the entire training objective, yet no quantification or verification is provided. Please add a direct check, for example: (a) the correlation between IQA scores computed on original images versus VAE-reconstructed images, or (b) a small experiment where the adapter is conditioned on scores computed from latents (or from VAE reconstructions) rather than from the clean pixel image, to show that the quality signal survives the latent bottleneck. As written, the limitation is asserted but its practical impact on the results is not measured.
  4. [Section 4.2, Eq. (1)] The RelGain metric in Eq. (1) assumes the same prompt and seed for x and x', but this is not meaningful for methods that change the prompt itself, such as BeautifulPrompt and Prompt Weighting (which modify the text). For those methods, the comparison is between images generated from different prompts, so the relative gain is not directly attributable to quality conditioning. Please specify how x is defined for these methods (e.g., the base model output on the original prompt) and whether the seed is held fixed, or otherwise separate the prompt-effect from the quality-effect to make the comparison fair.
minor comments (6)
  1. [Eq. (2), Section 3.1] The gradient-based guidance equation uses D(zt) without stating that backpropagation must pass through the VAE decoder; please clarify that this is a VAE decoder and that the gradient is with respect to zt through D.
  2. [Table 1 and Table 6] The row label '3xARNIQA,LIQE-MIX(different datasets)' is unclear; please spell out which ARNIQA variants are combined and how the '3x' weighting is applied.
  3. [Section 4.3, Figure 3(b)] The definition of win rate as 'share of image pairs on which it achieved the majority of votes' should specify how ties ('No difference' responses) are handled in the majority calculation.
  4. [Section 3.2.1] The term 'Qualitative Attention' is slightly confusing; consider 'Quality Attention' to match the 'quality tokens' terminology used later.
  5. [Section 4.5, Table 2] The best result in the SDEdit I2I section is IQA-Adapter(TOPIQ+LAION-AES) with SROCC 0.79, not the Ref.-based IQA-Adapter (0.69); please reflect this in the text, which currently emphasizes only the T2I result.
  6. [Supplementary Sec. 9] The training section states 'For Reference-based IQA-Adapter, we apply series of degradations to training images with a probability p = 0.1' but does not define the degradation model; it refers to [43] later, but please make the reference explicit at this point.

Circularity Check

2 steps flagged · score 4.0 of 10

Reference-based distortion-transfer SROCC is partly self-fulfilling because ARNIQA serves as both the conditioning embedding and the evaluation metric; the central quality-conditioning claim is otherwise independently supported.

  1. fitted input called prediction [Sec. 3.3 (architecture) and Sec. 4.5 / Table 2 (evaluation)]
    "In particular, we used the ARNIQA [43] IQA model to obtain qualitative embeddings ... To evaluate the qualitative alignment between generated and reference images, we measure Spearman’s correlation coefficient between target IQA metric values on generated images and distortion references. ... Ref.-based IQA-Adapter 0.80"

    ARNIQA supplies both the conditioning signal and the evaluation score. The Reference-based adapter is trained by conditioning denoising on ARNIQA activations of the image being reconstructed, so at inference its output is effectively steered toward the reference's ARNIQA embedding; ranking those outputs by ARNIQA's scalar score then measures proximity to the training objective rather than independent perceptual distortion transfer. IP-Adapter and StyleCrafter never see ARNIQA, making the 0.80-vs-0.41 gap asymmetric. The circularity is partial: the non-ARNIQA-conditioned IQA-Adapter(TOPIQ+LAION-AES) row (0.76) and the qualitative figures indicate genuine transfer remains.

  2. other [Sec. 3.4 (training) and Sec. 4.2 (high-quality conditioning evaluation)]
    "the quality scores are estimated by passing each image through a target IQA/IAA model ... For IQA-Adapter, high-quality conditioning is achieved by setting the input to the 99-th percentile of the target metric's values from the training dataset ... RelGain = 1/N Σ f(x′_i) − f(x_i) / f(x_i) · 100%"

    The conditioning percentile and the evaluation function use the same IQA model f that produced the training labels. Thus the 99th-percentile condition is an in-distribution interpolation of the adapter's supervision, so measuring outputs with the same f is a consistency check rather than an independent test. The paper does not rely on this alone: cross-metric gains (Figure 3a), GenEval, FID, and the human preference study are independent of the training metric and support the central claim.

full rationale

The paper's main quality-conditioning claim is not circular: IQA-Adapter is trained with a denoising objective on scores computed by IQA models, and its improvements are validated by cross-metric gains, GenEval, FID/CLIP, and a 60% human-preference win rate, all of which are external to the training labels. The reference-based experiment, however, contains a partially circular measurement: its headline SROCC 0.80 is computed with ARNIQA, the same model whose activations condition the Reference-based adapter, making the comparison with IP-Adapter and StyleCrafter asymmetric. That circularity is confined to one supporting measurement: the IQA-Adapter(TOPIQ+LAION-AES) control still achieves SROCC 0.76/0.79 under ARNIQA evaluation without using ARNIQA conditioning, and the qualitative figures illustrate distortion-type transfer. The pixel-space/latent-space mismatch acknowledged in Supp. Sec. 10 is a robustness limitation, not a circular step. Overall, circularity is partial and localized, giving a score of 4.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The method does not introduce physical or ontological entities. It relies on supervised labels from IQA models, on the assumption that quality is a content-agnostic attribute, and on the latent-pixel fidelity assumption. Hand-chosen inference parameters (adapter scale, negative guidance gap, target percentile) shape all reported results.

free parameters (4)
  • Adapter scale lambda = 0.5
    Set by hand for all main experiments; controls the strength of quality conditioning and strongly affects reported quality gains (Supplementary Sec. 11.3).
  • Negative guidance gap delta = 0.3 for +Neg. G. variants, 0 otherwise
    Chosen by hand to control the distance between positive and negative quality conditions in classifier-free guidance (Sec. 3.2.1).
  • Target quality percentile = 99th percentile of training set
    Defines the high-quality conditioning used in Sec. 4.2; arbitrary but constant across experiments and grounded in the training distribution.
  • Degradation augmentation probability p = 0.1
    Used only in reference-based IQA-Adapter training to expand coverage of the IQA activation space (Sec. 3.4).
assumptions (5)
  • domain assumption IQA/IAA scores are valid, transferable labels for human-perceived quality.
    Used to generate training targets in Sec. 3.4 and to define high-quality conditioning; the method inherits the biases and blind spots of these models, as acknowledged in Sec. 10.
  • domain assumption Quality is content-agnostic and can be separated from textual semantics via a separate cross-attention layer.
    Core design of Sec. 3.2.1; the ablation in Supplementary Sec. 11.1 supports this by showing prompt-following degrades without the separate attention.
  • domain assumption Farthest-layer activations of ARNIQA contain negligible semantic information.
    Basis of the reference-based IQA-Adapter (Sec. 3.3); used to prevent content leakage and evaluated via CLIP-I with distortion reference in Sec. 4.5.
  • standard math Standard DDPM training and classifier-free guidance equations extend to the adapter when conditions are randomly dropped.
    Training objective in Sec. 3.4 and inference equations in Sec. 3.2.1 rely on Ho et al. and classifier guidance; no new derivation is provided.
  • domain assumption Pixel-space quality scores can supervise latent-space denoising.
    Training in Sec. 3.4 computes IQA on pixel images but optimizes the adapter in VAE latent space; the authors explicitly list this discrepancy as a limitation in Sec. 10.

how reviews work

0 comments
Cite this review

Pith. "Pith review of IQA-Adapter: Exploring Knowledge Transfer from Image Quality Assessment to Diffusion-based Generative Models." pith.science (2026). https://pith.science/paper/DALQ5M6B

@misc{pith2026241201794,
  author       = {Pith},
  title        = {Pith review of: IQA-Adapter: Exploring Knowledge Transfer from Image Quality Assessment to Diffusion-based Generative Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DALQ5M6B}},
  note         = {Machine review of arXiv:2412.01794}
}
read the original abstract

Diffusion-based models have recently revolutionized image generation, achieving unprecedented levels of fidelity. However, consistent generation of high-quality images remains challenging partly due to the lack of conditioning mechanisms for perceptual quality. In this work, we propose methods to integrate image quality assessment (IQA) models into diffusion-based generators, enabling quality-aware image generation. We show that diffusion models can learn complex qualitative relationships from both IQA models' outputs and internal activations. First, we experiment with gradient-based guidance to optimize image quality directly and show this method has limited generalizability. To address this, we introduce IQA-Adapter, a novel framework that conditions generation on target quality levels by learning the implicit relationship between images and quality scores. When conditioned on high target quality, IQA-Adapter can shift the distribution of generated images towards a higher-quality subdomain, and, inversely, it can be used as a degradation model, generating progressively more distorted images when provided with a lower-quality signal. Under high-quality condition, IQA-Adapter achieves up to a 10% improvement across multiple objective metrics, as confirmed by a user preference study, while preserving generative diversity and content. Furthermore, we extend IQA-Adapter to a reference-based conditioning scenario, utilizing the rich activation space of IQA models to transfer highly specific, content-agnostic qualitative features between images.

Figures

Figures reproduced from arXiv: 2412.01794 by the authors.

Figure 1
Figure 1. Quality-aware image generation with IQA-Adapter. All images are generated with the SDXL base model, the same prompt, and the seed. The IQA-Adapter is trained with TOPIQ [1] and LAION-Aesthetics [2] metrics. with a range of IQA and IAA models with diverse archi￾tectures and training datasets, demonstrating the adapt￾ability of our approach to different quality and aesthetic metrics and the generalization of quality f… view at source ↗
Figure 2
Figure 2. Overall architecture of the proposed IQA-Adapter. Yellow arrows depict IQA/IAA knowledge flow into the diffusion-based generator. tor using reinforcement learning approaches to maximize quality. Q-Refine [59] utilizes an IQA model to detect low￾quality regions in the image and inpaints them using off￾the-shelf image enhancement models without any qualita￾tive knowledge transfer to the generator. In contrast to prior… view at source ↗
Figure 3
Figure 3. (a) Quality improvement relative to the base model (in %) for the IQA-Adapters trained on different IQA/IAA models. All IQA [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (22 more)
Figure 4
Figure 4. Figure 4: (a) Distributions of quality scores for images generated with the IQA-Adapter conditioned on different target quality levels (1-99 [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Examples of Reference-based IQA-Adapter conditioning. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: (a) Examples of adversarial patterns appearing under high [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Results of the IQA-Adapter modulation on input quality-condition for different types of input preprocessing with positional [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: The relationship between image-quality scores (evaluated by the HYPER-IQA, TOPIQ and LIQE metrics) and the adapter scale [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: Quality improvement relative to base model (in %) for the IQA-Adapters trained on different IQA/IAA models and other [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: Quality improvement relative to base model (in %) for the IQA-Adapters trained on different IQA/IAA models and other [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]
Figure 11
Figure 11. Figure 11: The relationship between input quality-condition (represented as a percentile of target IQA/IAA model on the training dataset) [PITH_FULL_IMAGE:figures/full_fig_p019_11.png]
Figure 12
Figure 12. Figure 12: Overall scheme of the subjective study described in Sections [PITH_FULL_IMAGE:figures/full_fig_p022_12.png]
Figure 13
Figure 13. Figure 13: FR IQA metrics distances between images generated with the IQA-Adapter conditioned on different target-quality levels. The [PITH_FULL_IMAGE:figures/full_fig_p023_13.png]
Figure 14
Figure 14. Figure 14: Distributions of relative gains defined in [PITH_FULL_IMAGE:figures/full_fig_p023_14.png]
Figure 15
Figure 15. Figure 15: Correlations between input quality-conditions (represented as a percentile of target IQA/IAA model on the training dataset) and [PITH_FULL_IMAGE:figures/full_fig_p025_15.png]
Figure 16
Figure 16. Figure 16: Ablation experiment: generations with IQA-Adapter with Neg. guidance enabled (1st row), without Neg. guidance (2nd row), [PITH_FULL_IMAGE:figures/full_fig_p025_16.png]
Figure 17
Figure 17. Figure 17: Visualization of generations with different target-quality conditions with IQA-Adapters trained with different IQA/IAA models. [PITH_FULL_IMAGE:figures/full_fig_p026_17.png]
Figure 18
Figure 18. Figure 18: Additional visualizations of IQA-Adapter quality-modulation with different aspect ratios. [PITH_FULL_IMAGE:figures/full_fig_p027_18.png]
Figure 19
Figure 19. Figure 19: Example of images generated with and without strong negative guidance ( [PITH_FULL_IMAGE:figures/full_fig_p027_19.png]
Figure 20
Figure 20. Figure 20: Examples of images generated with and without IQA-Adapter with the same prompt. The seeds are equal for corresponding [PITH_FULL_IMAGE:figures/full_fig_p028_20.png]
Figure 21
Figure 21. Figure 21: Examples of images generated with IQA-Adapter conditioned on [PITH_FULL_IMAGE:figures/full_fig_p028_21.png]
Figure 22
Figure 22. Figure 22: The comparison of adversarial examples generated with the gradient-based method (middle column) alongside outputs from the [PITH_FULL_IMAGE:figures/full_fig_p029_22.png]
Figure 23
Figure 23. Figure 23: Comparison of different generation quality improvement methods. [PITH_FULL_IMAGE:figures/full_fig_p030_23.png]
Figure 24
Figure 24. Figure 24: Reference-based Image Editing with SDEdit using a diffusion model equipped with Reference-based IQA-Adapter and IP [PITH_FULL_IMAGE:figures/full_fig_p031_24.png]
Figure 25
Figure 25. Figure 25: Text-to-Image generation with qualitative reference. First row denotes generations with Reference-based IQA-Adapter and [PITH_FULL_IMAGE:figures/full_fig_p032_25.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

115 extracted references · 56 canonical work pages

  1. [1]

    Topiq: A top-down approach from semantics to distortions for im- age quality assessment

    Chaofeng Chen, Jiadi Mo, Jingwen Hou, Haoning Wu, Liang Liao, Wenxiu Sun, Qiong Yan, and Weisi Lin. Topiq: A top-down approach from semantics to distortions for im- age quality assessment. IEEE Transactions on Image Pro- cessing, 2024. 2, 6, 3

    2024

  2. [2]

    Laion aesthetics predictor, 2023

    Christoph Schuhmann. Laion aesthetics predictor, 2023. date of access: November 14, 2024. 2, 3

    2023

  3. [3]

    Improving image generation with bet- ter captions

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with bet- ter captions. https://cdn.openai.com/papers/dall-e-3.pdf,

  4. [4]

    Flux github repo

    Black Forest Labs. Flux github repo. https://github.com/black-forest-labs/flux, 2024. 1, 2

    2024

  5. [5]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. In ICLR, 2024. 1, 2, 4

    2024

  6. [6]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In ICCV, 2023. 1, 2

    2023

  7. [7]

    Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models. arXiv preprint arxiv:2308.06721,

    arXiv

  8. [8]

    Omnigen: Unified image generation

    S Xiao, Y Wang, J Zhou, H Yuan, X Xing, and R Yan. Omnigen: Unified image generation. arXiv preprint arXiv:2409.11340, 2024. 1

    arXiv 2024

Show all 115 references
  1. [9]

    Imagere- ward: Learning and evaluating human preferences for text- to-image generation

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagere- ward: Learning and evaluating human preferences for text- to-image generation. NeurIPS, 36, 2024. 1

    2024

  2. [10]

    Pick-a-pic: An open dataset of user preferences for text-to-image generation

    Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. Pick-a-pic: An open dataset of user preferences for text-to-image generation. 2023

    2023

  3. [11]

    Better aligning text-to-image models with human preference, 2023

    Xiaoshi Wu, Keqiang Sun, Feng Zhu, Rui Zhao, and Hong- sheng Li. Better aligning text-to-image models with human preference, 2023. 1

    2023

  4. [12]

    Dp-iqa: Utilizing diffusion prior for blind image quality assessment in the wild

    Honghao Fu, Yufei Wang, Wenhan Yang, and Bihan Wen. Dp-iqa: Utilizing diffusion prior for blind image quality assessment in the wild. arXiv preprint arXiv:2405.19996,

  5. [13]

    Genziqa: Generalized image quality assessment us- ing prompt-guided latent diffusion models

    Diptanu De, Shankhanil Mitra, and Rajiv Soundarara- jan. Genziqa: Generalized image quality assessment us- ing prompt-guided latent diffusion models. arXiv preprint arXiv:2406.04654, 2024

    2024

  6. [14]

    Feature denoising diffusion model for blind image quality assessment

    Xudong Li, Jingyuan Zheng, Runze Hu, Yan Zhang, Ke Li, Yunhang Shen, Xiawu Zheng, Yutao Liu, ShengChuan Zhang, Pingyang Dai, et al. Feature denoising diffusion model for blind image quality assessment. arXiv preprint arXiv:2401.11949, 2024

    2024 arXiv

  7. [15]

    Diffusion model based vi- sual compensation guidance and visual difference analysis for no-reference image quality assessment

    Zhaoyang Wang, Bo Hu, Mingyang Zhang, Jie Li, Leida Li, Maoguo Gong, and Xinbo Gao. Diffusion model based vi- sual compensation guidance and visual difference analysis for no-reference image quality assessment. arXiv preprint arXiv:2402.14401, 2024. 1, 2

    2024 arXiv

  8. [16]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In Interna- tional conference on machine learning , pages 8821–8831. Pmlr, 2021. 2

    2021

  9. [17]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 4

    2022

  10. [18]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Sali- mans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural in- forma...

    2022

  11. [19]

    Playground v3: Improving text-to- image alignment with deep-fusion large language models

    Bingchen Liu, Ehsan Akhgari, Alexander Visheratin, Aleks Kamko, Linmiao Xu, Shivam Shrirao, Joao Souza, Suhail Doshi, and Daiqing Li. Playground v3: Improving text-to- image alignment with deep-fusion large language models. arXiv preprint arXiv:2409.10695, 2024. 2

    2024 arXiv

  12. [20]

    Yaart: Yet another art rendering technology.arXiv preprint arXiv:2404.05666, 2024

    Sergey Kastryulin, Artem Konev, Alexander Shishenya, Eugene Lyapustin, Artem Khurshudov, Alexander Tselousov, Nikita Vinokurov, Denis Kuznedelev, Alexan- der Markovich, Grigoriy Livshits, et al. Yaart: Yet another art rendering technology.arXiv preprint arXiv:2404.05666, 2024

    2024 arXiv

  13. [21]

    Emu: Enhanc- ing image generation models using photogenic needles in a haystack

    Xiaoliang Dai, Ji Hou, Chih-Yao Ma, Sam Tsai, Jialiang Wang, Rui Wang, Peizhao Zhang, Simon Vandenhende, Xi- aofang Wang, Abhimanyu Dubey, et al. Emu: Enhanc- ing image generation models using photogenic needles in a haystack. arXiv preprint arXiv:2309.15807, 2023

    2023 arXiv

  14. [22]

    Im- agen 3

    Jason Baldridge, Jakob Bauer, Mukul Bhutani, Nicole Brichtova, Andrew Bunner, Kelvin Chan, Yichang Chen, Sander Dieleman, Yuqing Du, Zach Eaton-Rosen, et al. Im- agen 3. arXiv preprint arXiv:2408.07009, 2024

    2024

  15. [23]

    Scaling recti- fied flow transformers for high-resolution image synthesis, march 2024

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling recti- fied flow transformers for high-resolution image synthesis, march 2024. URL http://arxiv. org/abs/2403.03206

    2024 arXiv

  16. [24]

    Pixart- \sigma: Weak-to-strong train- ing of diffusion transformer for 4k text-to-image genera- tion

    Junsong Chen, Chongjian Ge, Enze Xie, Yue Wu, Lewei Yao, Xiaozhe Ren, Zhongdao Wang, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart- \sigma: Weak-to-strong train- ing of diffusion transformer for 4k text-to-image genera- tion. arXiv preprint arXiv:2403.04692, 2024. 2

    2024 arXiv

  17. [25]

    9 LoRA: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 9 LoRA: Low-rank adaptation of large language models. In ICLR, 2022. 2

    2022

  18. [26]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In CVPR, pages 22500–22510, 2023. 2

    2023

  19. [27]

    An image is worth one word: Personalizing text-to-image generation using textual inversion

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit Haim Bermano, Gal Chechik, and Daniel Cohen-or. An image is worth one word: Personalizing text-to-image generation using textual inversion. In ICLR, 2023. 2, 4

    2023

  20. [28]

    T2i-adapter: Learn- ing adapters to dig out more controllable ability for text-to- image diffusion models

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i-adapter: Learn- ing adapters to dig out more controllable ability for text-to- image diffusion models. In AAAI, volume 38, pages 4296– 4304, 2024. 2

    2024

  21. [29]

    Erasing concepts from diffusion models

    Rohit Gandikota, Joanna Materzy ´nska, Tingrui Zhou, An- tonio Torralba, and David Bau. Erasing concepts from diffusion models. In ECCV, 2024. arXiv preprint arXiv:2311.12092. 2

    2024 arXiv

  22. [30]

    Stylecrafter: Enhancing stylized text-to-video generation with style adapter

    Gongye Liu, Menghan Xia, Yong Zhang, Haoxin Chen, Jinbo Xing, Xintao Wang, Yujiu Yang, and Ying Shan. Stylecrafter: Enhancing stylized text-to-video generation with style adapter. arXiv preprint arXiv:2312.00330, 2023. 2, 3, 8

    2023 arXiv

  23. [31]

    Movie gen: A cast of media foundation models

    Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjan- dra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih-Yao Ma, Ching-Yao Chuang, et al. Movie gen: A cast of media foundation models. arXiv preprint arXiv:2410.13720, 2024. 2

    arXiv

  24. [32]

    Diffusion for world modeling: Visual details matter in atari

    Eloi Alonso, Adam Jelley, Vincent Micheli, Anssi Kan- ervisto, Amos Storkey, Tim Pearce, and Franc ¸ois Fleuret. Diffusion for world modeling: Visual details matter in atari. arXiv preprint arXiv:2405.12399, 2024. 2

    2024 arXiv

  25. [33]

    Image quality assessment: from error visibility to structural similarity

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image pro- cessing, 13(4):600–612, 2004. 2, 9

    2004

  26. [34]

    Multi- scale structural similarity for image quality assessment

    Zhou Wang, Eero P Simoncelli, and Alan C Bovik. Multi- scale structural similarity for image quality assessment. In The Thrity-Seventh Asilomar Conference on Signals, Sys- tems & Computers, 2003 , volume 2, pages 1398–1402. Ieee, 2003

    2003

  27. [35]

    A visual information fi- delity approach to video quality assessment

    Hamid R Sheikh and Alan C Bovik. A visual information fi- delity approach to video quality assessment. In The first in- ternational workshop on video processing and quality met- rics for consumer electronics, volume 7, pages 2117–2128. sn, 2005

    2005

  28. [36]

    No-reference image quality assessment in the spatial domain

    Anish Mittal, Anush Krishna Moorthy, and Alan Con- rad Bovik. No-reference image quality assessment in the spatial domain. IEEE Transactions on image processing , 21(12):4695–4708, 2012

    2012

  29. [37]

    completely blind

    Anish Mittal, Rajiv Soundararajan, and Alan C Bovik. Making a “completely blind” image quality analyzer. IEEE Signal processing letters, 20(3):209–212, 2012

    2012

  30. [38]

    Blind image quality assessment: From natural scene statistics to perceptual quality

    Anush Krishna Moorthy and Alan Conrad Bovik. Blind image quality assessment: From natural scene statistics to perceptual quality. IEEE transactions on Image Processing, 20(12):3350–3364, 2011

    2011

  31. [39]

    A feature- enriched completely blind image quality evaluator

    Lin Zhang, Lei Zhang, and Alan C Bovik. A feature- enriched completely blind image quality evaluator. IEEE Transactions on Image Processing , 24(8):2579–2591,

  32. [40]

    Blindly assess image qual- ity in the wild guided by a self-adaptive hyper network

    Shaolin Su, Qingsen Yan, Yu Zhu, Cheng Zhang, Xin Ge, Jinqiu Sun, and Yanning Zhang. Blindly assess image qual- ity in the wild guided by a self-adaptive hyper network. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition , pages 3667–3676, 2020. 2, 3

    2020

  33. [41]

    Musiq: Multi-scale image quality transformer

    Junjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar, and Feng Yang. Musiq: Multi-scale image quality transformer. In Proceedings of the IEEE/CVF international conference on computer vision, pages 5148–5157, 2021. 3

    2021

  34. [42]

    Blind im- age quality assessment using a deep bilinear convolutional neural network

    A Deep Bilinear Convolutional Neural Network. Blind im- age quality assessment using a deep bilinear convolutional neural network. 3

  35. [43]

    Arniqa: Learning distortion mani- fold for image quality assessment

    Lorenzo Agnolucci, Leonardo Galteri, Marco Bertini, and Alberto Del Bimbo. Arniqa: Learning distortion mani- fold for image quality assessment. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 189–198, 2024. 4, 5, 6, 8, 3

    2024

  36. [44]

    Blind image quality assessment via vision- language correspondence: A multitask learning perspec- tive

    Weixia Zhang, Guangtao Zhai, Ying Wei, Xiaokang Yang, and Kede Ma. Blind image quality assessment via vision- language correspondence: A multitask learning perspec- tive. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition , pages 14071–14081, 2023

    2023

  37. [45]

    No-reference image quality assessment via transformers, relative ranking, and self-consistency

    S Alireza Golestaneh, Saba Dadsetan, and Kris M Kitani. No-reference image quality assessment via transformers, relative ranking, and self-consistency. InProceedings of the IEEE/CVF winter conference on applications of computer vision, pages 1220–1230, 2022. 3

    2022

  38. [46]

    Maniqa: Multi-dimension attention network for no- reference image quality assessment

    Sidi Yang, Tianhe Wu, Shuwei Shi, Shanshan Lao, Yuan Gong, Mingdeng Cao, Jiahao Wang, and Yujiu Yang. Maniqa: Multi-dimension attention network for no- reference image quality assessment. In CVPR, pages 1191– 1200, 2022. 6, 3

    2022

  39. [47]

    Con- volutional neural networks for no-reference image quality assessment

    Le Kang, Peng Ye, Yi Li, and David Doermann. Con- volutional neural networks for no-reference image quality assessment. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1733–1740,

  40. [48]

    Ex- ploring clip for assessing the look and feel of images

    Jianyi Wang, Kelvin CK Chan, and Chen Change Loy. Ex- ploring clip for assessing the look and feel of images. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 37, pages 2555–2563, 2023. 6, 3

    2023

  41. [49]

    Nima: Neural im- age assessment

    Hossein Talebi and Peyman Milanfar. Nima: Neural im- age assessment. IEEE transactions on image processing , 27(8):3998–4011, 2018. 3

    2018

  42. [50]

    Koniq-10k: An ecologically valid database for deep learning of blind image quality assessment

    Vlad Hosu, Hanhe Lin, Tamas Sziranyi, and Dietmar Saupe. Koniq-10k: An ecologically valid database for deep learning of blind image quality assessment. IEEE Transac- tions on Image Processing, 29:4041–4056, 2020. 3

    2020

  43. [51]

    Pipal: a large-scale image quality assessment dataset for perceptual image restoration

    Gu Jinjin, Cai Haoming, Chen Haoyu, Ye Xiaoxing, Jimmy S Ren, and Dong Chao. Pipal: a large-scale image quality assessment dataset for perceptual image restoration. In ECCV, pages 633–651. Springer, 2020. 3 10

    2020

  44. [52]

    Perceptual quality assessment of smartphone pho- tography

    Yuming Fang, Hanwei Zhu, Yan Zeng, Kede Ma, and Zhou Wang. Perceptual quality assessment of smartphone pho- tography. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3677–3686,

  45. [53]

    Massive on- line crowdsourced study of subjective and objective pic- ture quality

    Deepti Ghadiyaram and Alan C Bovik. Massive on- line crowdsourced study of subjective and objective pic- ture quality. IEEE Transactions on Image Processing , 25(1):372–387, 2015. 3

    2015

  46. [54]

    From patches to pictures (paq-2-piq): Mapping the perceptual space of pic- ture quality

    Zhenqiang Ying, Haoran Niu, Praful Gupta, Dhruv Maha- jan, Deepti Ghadiyaram, and Alan Bovik. From patches to pictures (paq-2-piq): Mapping the perceptual space of pic- ture quality. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3575–3585,

  47. [55]

    Ava: A large-scale database for aesthetic visual analysis

    Naila Murray, Luca Marchesotti, and Florent Perronnin. Ava: A large-scale database for aesthetic visual analysis. In 2012 IEEE conference on computer vision and pattern recognition, pages 2408–2415. IEEE, 2012. 3

    2012

  48. [56]

    Kadid-10k: A large-scale artificially distorted iqa database

    Hanhe Lin, Vlad Hosu, and Dietmar Saupe. Kadid-10k: A large-scale artificially distorted iqa database. In 2019 Eleventh International Conference on Quality of Multime- dia Experience (QoMEX), pages 1–3. IEEE, 2019. 8, 3

    2019

  49. [57]

    Photo aesthetics ranking network with attributes and content adaptation

    Shu Kong, Xiaohui Shen, Zhe Lin, Radomir Mech, and Charless Fowlkes. Photo aesthetics ranking network with attributes and content adaptation. In Computer Vision– ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part I 14, pages...

    2016

  50. [58]

    Diffusion model alignment using direct preference optimization

    Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caim- ing Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization. InProceed- ings of the IEEE/CVF Conference on Computer Vision ...

    2024

  51. [59]

    Q-refine: A perceptual qual- ity refiner for ai-generated image

    Chunyi Li, Haoning Wu, Zicheng Zhang, Hongkun Hao, Kaiwei Zhang, Lei Bai, Xiaohong Liu, Xiongkuo Min, Weisi Lin, and Guangtao Zhai. Q-refine: A perceptual qual- ity refiner for ai-generated image. In 2024 IEEE Interna- tional Conference on Multimedia and Expo (ICME), pages 1–6...

    2024

  52. [60]

    Vmix: Improving text-to-image diffusion model with cross-attention mixing control

    Shaojin Wu, Fei Ding, Mengqi Huang, Wei Liu, and Qian He. Vmix: Improving text-to-image diffusion model with cross-attention mixing control. arXiv preprint arXiv:2412.20800, 2024. 2

    2024 arXiv

  53. [61]

    BeautifulPrompt: Towards au- tomatic prompt engineering for text-to-image synthesis

    Tingfeng Cao, Chengyu Wang, Bingyan Liu, Ziheng Wu, Jinhui Zhu, and Jun Huang. BeautifulPrompt: Towards au- tomatic prompt engineering for text-to-image synthesis. In Mingxuan Wang and Imed Zitouni, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural La...

    2023

  54. [62]

    Compel library

    Damian Stewart. Compel library. https://github. com/damian0815/compel, 2023. 2, 6

    2023

  55. [63]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alex Nichol. Diffusion models beat gans on image synthesis. In Proceedings of the 35th In- ternational Conference on Neural Information Processing Systems, NIPS ’21, Red Hook, NY , USA, 2024. Curran As- sociates Inc. 3

    2024

  56. [64]

    T2i- adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, Ying Shan, and Xiaohu Qie. T2i- adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. arXiv preprint arXiv:2302.08453, 2023. 3

    2023 arXiv

  57. [65]

    Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. Layer normalization. https://arxiv.org/abs/1607.06450,

    arXiv

  58. [66]

    De- noising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. De- noising diffusion probabilistic models. arXiv preprint arxiv:2006.11239, 2020. 4

    2006 arXiv

  59. [67]

    Conceptual captions: A cleaned, hypernymed, im- age alt-text dataset for automatic image captioning

    Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. Conceptual captions: A cleaned, hypernymed, im- age alt-text dataset for automatic image captioning. In Pro- ceedings of ACL, 2018. 5

    2018

  60. [68]

    Laion-5b 170k subset of images with aes- thetics score > 6.5

    Bhargav Desai. Laion-5b 170k subset of images with aes- thetics score > 6.5. https://huggingface.co/ datasets / bhargavsdesai / laion _ improved _ aesthetics_6.5plus_with_images, 2022. 5

    2022

  61. [69]

    Laion-5b: an open large-scale dataset for training next generation image-text models

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts- man, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Je- nia Jitsev. L...

    2024

  62. [70]

    Blind image quality assessment using a deep bilin- ear convolutional neural network

    Weixia Zhang, Kede Ma, Jia Yan, Dexiang Deng, and Zhou Wang. Blind image quality assessment using a deep bilin- ear convolutional neural network. IEEE Transactions on Circuits and Systems for Video Technology , 30(1):36–47,

  63. [71]

    Convo- lutional neural networks for no-reference image quality as- sessment

    Le Kang, Peng Ye, Yi Li, and David Doermann. Convo- lutional neural networks for no-reference image quality as- sessment. In CVPR, pages 1733–1740, 2014. 6

    2014

  64. [72]

    Blindly assess image qual- ity in the wild guided by a self-adaptive hyper network

    Shaolin Su, Qingsen Yan, Yu Zhu, Cheng Zhang, Xin Ge, Jinqiu Sun, and Yanning Zhang. Blindly assess image qual- ity in the wild guided by a self-adaptive hyper network. In CVPR, June 2020. 6

    2020

  65. [73]

    Musiq: Multi-scale image quality transformer

    Junjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar, and Feng Yang. Musiq: Multi-scale image quality transformer. In ICCV, pages 5148–5157, 2021. 6

    2021

  66. [74]

    No-reference image quality assessment via transformers, relative ranking, and self-consistency

    S Alireza Golestaneh, Saba Dadsetan, and Kris M Kitani. No-reference image quality assessment via transformers, relative ranking, and self-consistency. In Proceedings of the IEEE/CVF Winter Conference on Applications of Com- puter Vision, pages 3209–3218, 2022. 6

    2022

  67. [75]

    Blind image quality assessment via vision- language correspondence: A multitask learning perspec- tive

    Weixia Zhang, Guangtao Zhai, Ying Wei, Xiaokang Yang, and Kede Ma. Blind image quality assessment via vision- language correspondence: A multitask learning perspec- tive. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, page 14071–14081, 2023. 6, 3

    2023

  68. [76]

    Dataset of user-generated prompts col- lected from lexica.art website

    Gustavo Santana. Dataset of user-generated prompts col- lected from lexica.art website. date of access: November 14, 2024. 6 11

    2024

  69. [77]

    Scaling autoregressive mod- els for content-rich text-to-image generation

    Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, Ben Hutchin- son, Wei Han, Zarana Parekh, Xin Li, Han Zhang, Jason Baldridge, and Yonghui Wu. Scaling autoregressive mod- els for content...

    2022

  70. [78]

    Geneval: An object-focused framework for evaluating text- to-image alignment

    Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. Geneval: An object-focused framework for evaluating text- to-image alignment. NeurIPS, 36, 2024. 6, 7

    2024

  71. [79]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll ´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV, pages 740–755. Springer, 2014. 6

    2014

  72. [80]

    Gans trained by a two time-scale update rule converge to a local nash equi- librium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equi- librium. NeurIPS, 30, 2017. 6

    2017

  73. [81]

    Improved techniques for training gans

    Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. NeurIPS, 29, 2016. 7

    2016

  74. [82]

    Clipscore: A reference-free eval- uation metric for image captioning

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free eval- uation metric for image captioning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Lan- guage Processing, pages 7514–7528, 2021. 7

    2021

  75. [83]

    SDEdit: Guided image synthesis and editing with stochastic differ- ential equations

    Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. SDEdit: Guided image synthesis and editing with stochastic differ- ential equations. In International Conference on Learning Representations, 2022. 8

    2022

  76. [84]

    Blip-2: bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: bootstrapping language-image pre-training with frozen image encoders and large language models. In Pro- ceedings of the 40th International Conference on Machine Learning, ICML’23. JMLR.org, 2023. 8

    2023

  77. [85]

    Pytorch library for cam methods

    Jacob Gildenblat and contributors. Pytorch library for cam methods. https://github.com/jacobgil/ pytorch-grad-cam, 2021. 1

    2021

  78. [86]

    IQA-PyTorch: Pytorch toolbox for image quality assessment

    Chaofeng Chen and Jiadi Mo. IQA-PyTorch: Pytorch toolbox for image quality assessment. [Online]. Avail- able: https://github.com/chaofengc/IQA- PyTorch, 2022. 2

    2022

  79. [87]

    calflops: a flops and params calculate tool for neural networks in pytorch framework, 2023

    xiaoju ye. calflops: a flops and params calculate tool for neural networks in pytorch framework, 2023. 2

    2023

  80. [88]

    A statistical evaluation of recent full reference image quality assessment algorithms

    Hamid R Sheikh, Muhammad F Sabir, and Alan C Bovik. A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Transactions on image pro- cessing, 15(11):3440–3451, 2006. 3

    2006

  81. [89]

    Most appar- ent distortion: full-reference image quality assessment and the role of strategy

    Eric C Larson and Damon M Chandler. Most appar- ent distortion: full-reference image quality assessment and the role of strategy. Journal of electronic imaging , 19(1):011006–011006, 2010. 3

    2010

  82. [90]

    No-reference blur assessment of digital pictures based on multifeature classi- fiers

    Alexandre Ciancio, Eduardo AB da Silva, Amir Said, Ramin Samadani, Pere Obrador, et al. No-reference blur assessment of digital pictures based on multifeature classi- fiers. IEEE Transactions on image processing , 20(1):64– 75, 2010. 3

    2010

  83. [91]

    Human pref- erence score v2: A solid benchmark for evaluating hu- man preferences of text-to-image synthesis

    Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human pref- erence score v2: A solid benchmark for evaluating hu- man preferences of text-to-image synthesis. arXiv preprint arXiv:2306.09341, 2023. 3

    2023 arXiv

  84. [92]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. https://arxiv.org/abs/1711.05101, 2019. 2

    2019 arXiv

  85. [93]

    Accelerate: Training and inference at scale made simple, efficient and adaptable

    Sylvain Gugger, Lysandre Debut, Thomas Wolf, Philipp Schmid, Zachary Mueller, Sourab Mangrulkar, Marc Sun, and Benjamin Bossan. Accelerate: Training and inference at scale made simple, efficient and adaptable. https:// github.com/huggingface/accelerate, 2022. 2

    2022

  86. [94]

    Srinivasan, Matthew Tancik, Jonathan T

    Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: representing scenes as neural radiance fields for view syn- thesis. Commun. ACM, 65(1):99–106, December 2021. 4

    2021

  87. [95]

    Roformer: Enhanced transformer with rotary position embedding

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomput., 568(C), March 2024. 4

    2024

  88. [96]

    A study on the evaluation of generative models.arXiv preprint arXiv:2206.10935, 2022

    Eyal Betzalel, Coby Penso, Aviv Navon, and Ethan Fe- taya. A study on the evaluation of generative models.arXiv preprint arXiv:2206.10935, 2022. 7

    2022 arXiv

  89. [97]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, 2018. 9

    2018

  90. [98]

    Simoncelli

    Keyan Ding, Kede Ma, Shiqi Wang, and Eero P. Simoncelli. Image quality assessment: Unifying structure and texture similarity. CoRR, abs/2004.07728, 2020. 9

    2004 arXiv

  91. [99]

    Pieapp: Perceptual image-error assessment through pairwise preference

    Ekta Prashnani, Hong Cai, Yasamin Mostofi, and Pradeep Sen. Pieapp: Perceptual image-error assessment through pairwise preference. In CVPR, June 2018. 9

    2018

  92. [100]

    Msu video super-resolution quality metrics benchmark 2023

    Artem Borisov, Evgeney Bogatyrev, Egor Kashkarov, and Dmitriy Vatolin. Msu video super-resolution quality metrics benchmark 2023. URL: https : //videoprocessing.ai/benchmarks/super- resolution-metrics.html , 2023. Date of access: 2024-11-19. 10 12 IQA-Adapter: Exploring Knowle...

    2023

  93. [101]

    Contents Here we briefly summarize the contents of all sections in this supplementary file: • Section 7: Discussion of the possible use-cases of IQA- Adapter and Future Work; • Section 8: A detailed summary of all IQA/IAA models used in this study; • Section 9: Details on IQA-...

  94. [102]

    Discussion and Future Work 7.1. IQA-Adapter as a degradation model As most IQA models are trained to assess distorted images, they can reliably detect noise, compression, blur, and other artifacts on images during IQA-Adapter training. There- fore, this knowledge is transferre...

  95. [103]

    (a) Examples of adversarial patterns appearing under high gradient-based guidance scale

    Employed IQA/IAA methods Table 3 provides a detailed summary of all IQA/IAA meth- ods used in this study, along with their training datasets 1 Base Model ARNIQA (KONIQ) HYPER-IQA TOPIQ (AVA) TRES (FLIVE) DBCNN (a) Base Model TOPIQ (KonIQ) LAION-Aesthetics TOPIQ + LAION-Aesthet...

  96. [104]

    During training on CC3M, the images were center- cropped to a resolution of 512 × 512

    IQA-Adapter training The IQA-Adapters were trained on the CC3M dataset, which consists of approximately 3 million text-image pairs, for 24,000 steps, followed by fine-tuning on a subset of the LAION-5B dataset, containing 170,000 images, for 3,000 steps. During training on CC3...

    2006

  97. [105]

    However, the extent of this knowledge transfer is inherently constrained by the capabilities and limitations of current IQA/IAA models

    Limitations IQA-Adapter serves as a guiding mechanism for transfer- ring knowledge from the IQA/IAA domain to generative models. However, the extent of this knowledge transfer is inherently constrained by the capabilities and limitations of current IQA/IAA models. Most existin...

  98. [106]

    adaptive

    Ablation Study In this section, we report the results of our experiments with different architectural elements and hyperparameters of the IQA-Adapter. We compare our base design with a ”sim- plified” model (Sec. 11.1) and a more sophisticated ap- proach with Positional Encodin...

  99. [107]

    High-quality conditioning: more results 12.1. Gradient-based guidance Figure 9(b) presents the relative gain in metric scores when using the gradient-based approach to optimize image qual- ity during generation for prompts from PartiPrompts [77]. Unlike IQA-Adapter, direct opt...

  100. [108]

    Among the 25 evaluated IQA-Adapters, five outperform the Base Model in terms of the overall score

    Evaluating Generative Capabilities: more results Table 6 provides the complete results on the GenEval benchmark. Among the 25 evaluated IQA-Adapters, five outperform the Base Model in terms of the overall score. Notably, even the weakest IQA-Adapter surpasses the Base Model in...

  101. [109]

    Figure 15 shows estimated correla- tions for each trained IQA-Adapter

    Alignment with qualitative condition: more results To further evaluate the relationship between the input qual- ity conditions provided to the IQA-Adapter during image generation and the quality of the resulting images, we an- alyzed correlations between the target quality and...

  102. [110]

    Left is better

    IQA-Adapter as a degradation model 15.1. Examples of progressive quality degradation Figures 17 and 18 illustrate the generation results for dif- ferent percentiles of metric scores on the training dataset. As the percentile decreases, the generated images begin to exhibit var...

  103. [111]

    Overall scheme of the subjective study described in Sections 4.3 and 16

    Validation Payment (If a person passed verification quesstions) Figure 12. Overall scheme of the subjective study described in Sections 4.3 and 16. age shifts, making it more robust. Lastly, PieAPP demon- strates strong correlations with subjective scores, particu- larly for t...

  104. [112]

    equal quality

    Subjective Study Our subjective study employed 300 randomly sampled user- generated prompts from the Lexica.art dataset. We used Subjectify.us platform for the evaluation. Overall scheme of the subjective study and the example of the user interface is demonstrated on Figure 12...

  105. [113]

    Computational Overhead In Table 8, we report time measurements for different gen- eration methods used in this work

    Additional Experiments 17.1. Computational Overhead In Table 8, we report time measurements for different gen- eration methods used in this work. All evaluations were 10 Percentile Percentile Percentile Percentile Percentile DISTS↓ LPIPS↓ PieAPP↓ SSIM↑ Figure 13. FR IQA metric...

  106. [114]

    over-stylized

    Quality-conditioning and Adversarial Ro- bustness of IQA models Figure 22 presents a comparison of images generated by the base model (left column), the gradient-based method (middle column), and the IQA-Adapter (right column), alongside GradCAM visualizations of the target IQ...

  107. [115]

    over-stylisation

    Reference-based IQA-Adapter: more visu- alizations Figure 24 demonstrates the comparison of Reference-based IQA-Adapter and IP-Adapter in image editing task. Fig- ure 25 shows the results on Text-to-Image generation task with similar distortion references. It can be seen that ...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.