REVIEW 3 major objections 5 minor 32 references
Hierarchical Fusion and Joint Aggregation: A Multi-Level Feature Representation Method for AIGC Image Quality Assessment
T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read The paper argues that replacing a single top-level visual feature with a hierarchy of features from CLIP and ResNet, fused by learnable queries, yields better AIGC image quality assessment for both perceptual quality and text-to-image…
desk verdict Solid multi-level fusion idea, internal ablations carry it, but single-seed comparisons do not support the 'outstanding performance' claim. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the learnable query-based fusion block, used in two variants. A GLF Block takes a set of learnable queries and runs two successive cross-attention operations plus a feed-forward network with residual connections: the first attention pulls global information from the CLIP feature level, the second pulls local detail from the corresponding ResNet feature level. A PEF Block does the same but with the text prompt's semantic tokens as the first conditioning signal and the visual features as the second. Each level's output queries are concatenated along the token dimension, globally averaged, and fed to an MLP regression head. The four chosen layers of CLIP and four stages of ResNet provide the hierarchy, with an Adapter converting 2D CNN feature maps into 1D token sequences.
What would settle it
Run the same training on AGIQA-3K under ten different random seeds and report mean and standard deviation of SRCC/PLCC; the claim would be undercut if the reported margin over SF-IQA on quality SRCC (0.9039 vs 0.9024) falls within one standard deviation or if the seed sweep reverses the ranking.
Extended reading notes
Core claim
The central discovery, as the authors state it, is that a multi-level visual representation paradigm outperforms single-level feature use across two AIGC assessment tasks. The networks take the AIGC image (and, for correspondence, the text prompt) and extract four levels of features from the CLIP image encoder's 3rd, 6th, 9th, and 12th layers, plus four stages of ResNet50 for local detail. At each level, learnable queries are refined through cross-attention: in MGLF-Net, first absorbing global information from CLIP and then local details from ResNet; in MPEF-Net, first absorbing prompt semantics and then attending to the visual features. The refined queries are concatenated, pooled, and regressed to a Mean Opinion Score. Experiments on three benchmarks report SRCC/PLCC values that are best or second-best among the compared methods, and the ablations indicate that removing the CNN branch, the Transformer branch, the prompt embedding, or the intermediate levels each lowers performance, confirming the role of multi-level fusion.
Load-bearing premise
The reported gains rest on one fixed 80/20 train/test split with a single random seed, and the layer choices and query counts were tuned on the same benchmark datasets, so the margins over strong baselines could be within run-to-run variation.
Editorial extensions
If this is right
- If the paradigm is correct, quality models for AIGC should extract features from multiple depths of the backbone rather than only the final layer.
- The success of MGLF-Net implies that global Transformer and local CNN information are complementary for perceptual quality, so combining them at every level is a better recipe than either alone.
- The success of MPEF-Net implies that injecting prompt semantics before each visual fusion stage improves text-to-image correspondence judgments.
- The same three-stage pipeline can be reused for other no-reference quality tasks where both low-level artifacts and high-level semantics matter.
Reading between the lines
- The query-count difference in the ablations, 4 queries for quality and 8 for correspondence, hints that correspondence is the harder fusion problem; one could test whether the optimal query count scales with prompt complexity.
- A natural extension is to test the same multi-level fusion on AIGC video or on images from unseen generators, since the benchmark images come from a limited set of text-to-image models.
- Because the fusion blocks are lightweight and task-agnostic, they could be plugged into other CLIP-based predictors, such as aesthetic scoring or image-text retrieval re-ranking, with little change.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a multi-level visual representation paradigm for AIGC image quality assessment, instantiated by two networks: MGLF-Net for perceptual quality, which fuses global features from CLIP-B/32 with local features from ResNet50 via learnable queries, and MPEF-Net for text-to-image correspondence, which embeds prompt semantics into the visual fusion process. Features are drawn from layers 3, 6, 9, and 12 of the CLIP image encoder and stages 1 through 4 of ResNet50, fused hierarchically, aggregated by global average pooling, and regressed to MOS with an MLP. Experiments are reported on AGIQA-1K, AGIQA-3K, and AIGCIQA2023 using SRCC and PLCC, with ablations on AGIQA-3K. The central claim is that the multi-level paradigm achieves outstanding performance on both perceptual quality and T2I correspondence tasks.
Significance. If the comparative performance claim were firmly established, the proposed paradigm would be a useful contribution: the dual-backbone hierarchical fusion design is well motivated, the prompt-embedded fusion mechanism is task-appropriate, and the ablations in Table IV consistently show gains of the multi-level variant over single-level features (e.g., Qual SRCC 0.9039 vs 0.8902; Corr SRCC 0.8410 vs 0.8258). The manuscript documents its implementation details and a fixed random seed, which aids reproducibility. However, the evidence supporting the headline 'outstanding performance' claim is currently too weak: the comparisons are based on one fixed split with no variance estimates, the margins over the strongest baselines are often within typical run-to-run noise, and on several metrics the proposed method is not the best. The internal ablations are the strongest part of the empirical evidence; the cross-method ranking needs substantially stronger experimental support.
major comments (3)
- [Section III-A and Tables I-III] The comparative claim of outstanding performance is not established because each dataset uses a single fixed 80/20 split with no error bars, multiple seeds, or statistical significance tests. The reported margins over the strongest baselines are very small: on AGIQA-3K Qual the SRCC gain over SF-IQA is 0.0015 (0.9039 vs 0.9024) while the PLCC is lower (0.9310 vs 0.9314); on AGIQA-3K Corr the proposed method is worse than SF-IQA on both SRCC and PLCC (0.8410/0.8968 vs 0.8454/0.9072); on AIGCIQA2023 Corr the gain over AMFF-Net is 0.0015 in SRCC and 0.0006 in PLCC. These deltas are far smaller than the seed-to-seed variation typically observed in split-based IQA evaluation, so the ranking could reverse under a different split. Please report results over multiple random splits (e.g., at least 5 seeds) as mean +/- std, include a significance test, and clarify whether the baseline numbers are re-run under the identical protocol or taken from their original papers with different splits.
- [Section II-A and Section III-B] The design hyperparameters -- CLIP layers {3,6,9,12}, ResNet50 stages 1-4, and the number of learnable queries (4 for GLF, 8 for PEF) -- are selected using the same benchmark datasets on which the final results are reported. This creates an optimistic selection bias for the comparative claim. Please either select these hyperparameters on a held-out validation split or show that the performance ranking against baselines is insensitive to reasonable variations in these choices.
- [Tables I-III] The proposed method is not consistently the best even in the reported tables. On AGIQA-1K (Table III) the PLCC is 0.8874, below CONTRIQUE, IP-IQA, and MoE-AGIQA-v2 at 0.8922. On AGIQA-3K (Table II) the Corr metrics are below SF-IQA. On AIGCIQA2023 (Table I) the Corr gains over AMFF-Net are negligible. The abstract and conclusion should either be toned down to claim competitiveness rather than outstanding performance, or the experiments need to be extended so that the claim is supported by statistically meaningful margins.
minor comments (5)
- [Table IV] The table heading contains a typo, 'ABALATION', which should be 'ABLATION'.
- [Table IV] The 'Full Model' row is identical to the '4 queries' row for Qual and to the '8 queries' row for Corr; please clarify the exact query configuration used for each of the two final networks, since MGLF-Net and MPEF-Net use different query counts.
- [Section II-B, Eqs. (2)-(4)] The symbol Q is reused for both the input and the output of the cross-attention and FFN stages without distinguishing superscripts or subscripts, which makes the equations ambiguous; please use distinct notation such as Q^i, Q'^i, Q''^i.
- [General] The text and reference list contain formatting artifacts (e.g., broken hyphenation in 'AI -generated' and line breaks inside references); the manuscript should be carefully proofread and reformatted.
- [Implementation Details] The paper mentions a fixed random seed for reproducibility, but it does not state whether code or trained models will be released; please add an availability statement.
Circularity Check
No circularity: the reported MOS predictions are evaluated on held-out 80/20 splits, and no fitted parameter is renamed as a prediction.
full rationale
The paper's derivation chain is a supervised regression pipeline: multi-level features are extracted from CLIP-B/32 and ResNet50 (Eq. 1), refined by learnable-query cross-attention blocks (Eqs. 2-4), aggregated, and regressed to MOS (Eqs. 5-7). No equation defines the target MOS in terms of the model's own outputs, and the network is trained against human-annotated labels. The experimental protocol is explicitly split-based: 'Each dataset was split into training and test sets using an 8/2 ratio. We maintained a fixed random seed throughout the data splitting and model training processes to ensure reproducibility.' Therefore the reported SRCC/PLCC values are genuine held-out predictions, not constructions. The hyperparameters (CLIP layers 3/6/9/12 and the query counts 4/8) are chosen based on the same benchmarks and only one split/seed is reported, which is a robustness and potential-optimism concern rather than a circularity concern. The paper does not rely on self-citations by the present authors, invokes no uniqueness theorem from prior work, and does not smuggle in its architecture via an unexamined citation. No fitted parameter is renamed as a prediction, and no equation is equivalent to its input by construction. Hence no circular step can be exhibited.
Assumptions & free parameters
free parameters (5)
- Number of learnable queries N_Q (GLF) =
4
- Number of learnable queries N_Q (PEF) =
8
- CLIP encoder layer selection =
3rd, 6th, 9th, 12th
- ResNet50 stage selection =
Stages 1-4
- Training hyperparameters (learning rate, weight decay, batch size, epochs) =
lr=1e-5, wd=1e-5, batch=16, epochs=30
assumptions (4)
- domain assumption Human MOS labels in AGIQA-1K, AGIQA-3K, and AIGCIQA2023 are valid ground truth for perceptual quality and text-image correspondence.
- domain assumption The selected intermediate layers of CLIP and ResNet capture complementary information needed for AIGC quality assessment.
- domain assumption Pre-trained CLIP and ResNet50 backbones provide suitable feature spaces for AIGC IQA.
- domain assumption The 80/20 split is representative of the dataset distribution.
Cite this review
Pith. "Pith review of Hierarchical Fusion and Joint Aggregation: A Multi-Level Feature Representation Method for AIGC Image Quality Assessment." pith.science (2026). https://pith.science/paper/Q62WOW6C
@misc{pith2026250717182,
author = {Pith},
title = {Pith review of: Hierarchical Fusion and Joint Aggregation: A Multi-Level Feature Representation Method for AIGC Image Quality Assessment},
year = {2026},
howpublished = {\url{https://pith.science/paper/Q62WOW6C}},
note = {Machine review of arXiv:2507.17182}
}
read the original abstract
The quality assessment of AI-generated content (AIGC) faces multi-dimensional challenges, that span from low-level visual perception to high-level semantic understanding. Existing methods generally rely on single-level visual features, limiting their ability to capture complex distortions in AIGC images. To address this limitation, a multi-level visual representation paradigm is proposed with three stages, namely multi-level feature extraction, hierarchical fusion, and joint aggregation. Based on this paradigm, two networks are developed. Specifically, the Multi-Level Global-Local Fusion Network (MGLF-Net) is designed for the perceptual quality assessment, extracting complementary local and global features via dual CNN and Transformer visual backbones. The Multi-Level Prompt-Embedded Fusion Network (MPEF-Net) targets Text-to-Image correspondence by embedding prompt semantics into the visual feature fusion process at each feature level. The fused multi-level features are then aggregated for final evaluation. Experiments on benchmarks demonstrate outstanding performance on both tasks, validating the effectiveness of the proposed multi-level visual assessment paradigm.
Reference graph
Works this paper leans on
-
[1]
High-resolution image synthesis with latent diffusion models,
R. Rombach, A. Blattmann, D. Lorenz, P. Esser, et al., “High-resolution image synthesis with latent diffusion models,” in Proceedings of the IEE E/CVF Conference on Computer Vision and Pattern Recognition (CVP R), 2022, pp. 10684–10695
work page 2022
-
[2]
Zero-shot text-to-image generation,
A. Ramesh, M. Pavlov, G. Goh, S. Gray, et al., “Zero-shot text-to-image generation,” in Proceedings of the 38th International Conference on Ma chine Learning (ICML), 2021, pp. 8821–8831
work page 2021
-
[3]
Hierarchical text-con ditional image generation with clip latents,
A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, et al., “Hierarchical text-con ditional image generation with clip latents,” arXiv preprint arXiv:2204.0 6125, 2022
work page 2022
-
[4]
Making a ‘completely bli nd’ image quality analyzer,
A. Mittal, R. Soundararajan, and A. C. Bovik, “Making a ‘completely bli nd’ image quality analyzer,” IEEE Signal Processing Letters, vol. 20, no. 3, pp. 209–212, 2012
work page 2012
-
[5]
NIMA: Neural image assessment,
H. Talebi and P. Milanfar, “NIMA: Neural image assessment,” IEEE Tra nsactions on Image Processing, vol. 27, no. 8, pp. 3998–4011, 2018
work page 2018
-
[6]
Hybrid no-reference quality m etric for singly and multiply distorted images,
K. Gu, G. Zhai, X. Yang, and W. Zhang, “Hybrid no-reference quality m etric for singly and multiply distorted images,” IEEE Transactions on Br oadcasting, vol. 60, no. 3, pp. 555–567, 2014
work page 2014
-
[7]
Blind image quality assessm ent: A natural scene statistics approach in the DCT domain,
M. A. Saad, A. C. Bovik, and C. Charrier, “Blind image quality assessm ent: A natural scene statistics approach in the DCT domain,” IEEE Trans actions on Image Processing, vol. 21, no. 8, pp. 3339–3352, 2012
work page 2012
-
[8]
Blind image quality assessment: From natural scene statistics to perceptual quality,
A. K. Moorthy and A. C. Bovik, “Blind image quality assessment: From natural scene statistics to perceptual quality,” IEEE Transactions on Ima ge Processing, vol. 20, no. 12, pp. 3350–3364, 2011
work page 2011
Show all 32 references
-
[9]
Quality predi ction of AI generated images and videos: Emerging trends and opportuni ties,
A. Ghildyal, Y. Chen, S. Zadtootaghaj, N. Barman, et al., “Quality predi ction of AI generated images and videos: Emerging trends and opportuni ties,” arXiv preprint arXiv:2410.08534, 2024
2024 arXiv
-
[10]
Aigc image quality assessmen t via image-prompt correspondence,
F. Peng, H. Fu, A. Ming, C. Wang, et al., “Aigc image quality assessmen t via image-prompt correspondence,” in Proceedings of the IEEE/CVF C onference on Computer Vision and Pattern Recognition (CVPR), 2024, p p. 6432–6441
2024
-
[11]
CLIP-AGIQA: boosting the p erformance of ai-generated image quality assessment with clip,
Z. Tang, Z. Wang, B. Peng, and J. Dong, “CLIP-AGIQA: boosting the p erformance of ai-generated image quality assessment with clip,” in Proc eedings of the International Conference on Pattern Recognition (ICPR), 2024, pp. 48–61
2024
-
[12]
Adaptive mixed-scale feature fusion network for blind AI-generated image quality assessment,
T. Zhou, S. Tan, W. Zhou, Y. Luo, et al., “Adaptive mixed-scale feature fusion network for blind AI-generated image quality assessment,” IEEE Transactions on Broadcasting, vol. 70, no. 3, pp. 833–843, 2024
2024
-
[13]
Tier: Text-image encoder-base d regression for aigc image quality assessment,
J. Yuan, X. Cao, J. Che, Q. Wang, et al., “Tier: Text-image encoder-base d regression for aigc image quality assessment,” arXiv preprint arXiv:24 01.03854, 2024
2024
-
[14]
Do vision tran sformers see like convolutional neural networks?,
M. Raghu, T. Unterthiner, S. Kornblith, C. Zhang, et al., “Do vision tran sformers see like convolutional neural networks?,” in Advances in Neura l Information Processing Systems (NeurIPS), vol. 34, 2021, pp. 12116–1 2128
2021
-
[15]
Boosting image quality assessm ent through efficient transformer adaptation with local feature enhancem ent,
K. Xu, L. Liao, J. Xiao, C. Chen, et al., “Boosting image quality assessm ent through efficient transformer adaptation with local feature enhancem ent,” in Proceedings of the IEEE/CVF Conference on Computer Vision a nd Pattern Recognition (CVPR), 2024, pp. 2662–2672
2024
-
[16]
Visualizing and understanding convolution al networks,
M. D. Zeiler and R. Fergus, “Visualizing and understanding convolution al networks,” in European Conference on Computer Vision (ECCV), 201 4, pp. 818–833
-
[17]
Understanding the effective r eceptive field in deep convolutional neural networks,
W. Luo, Y. Li, R. Urtasun, and R. Zemel, “Understanding the effective r eceptive field in deep convolutional neural networks,” in Advances in Ne ural Information Processing Systems (NeurIPS), vol. 29, 2016
2016
-
[18]
Understanding neural networks through deep visualization,
J. Yosinski, J. Clune, A. Nguyen, T. Fuchs, et al., “Understanding neural networks through deep visualization,” arXiv preprint arXiv:1506.0657 9, 2015
2015
-
[19]
Learning transfer able visual models from natural language supervision,
A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, et al., “Learning transfer able visual models from natural language supervision,” in Proceedings o f the International Conference on Machine Learning (ICML), 2021, pp. 8748–8763
2021
-
[20]
Deep residual learning for image r ecognition,
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image r ecognition,” in Proceedings of the IEEE Conference on Computer Visio n and Pattern Recognition (CVPR), 2016, pp. 770–778
2016
-
[21]
A perceptual quality assessment exploration for aigc images,
Z. Zhang, C. Li, W. Sun, X. Liu, et al., “A perceptual quality assessment exploration for aigc images,” in 2023 IEEE International Conference on Multimedia and Expo Workshops (ICMEW), 2023, pp. 440–445
2023
-
[22]
Agiqa-3k: An open database for ai-generated image quality assessment,
C. Li, Z. Zhang, H. Wu, W. Sun, et al., “Agiqa-3k: An open database for ai-generated image quality assessment,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 34, no. 8, pp. 6833–6846, 2023
2023
-
[23]
Aigciqa2023: A large-scale i mage quality assessment database for ai generated images: from the pers pectives of quality, authenticity and correspondence,
J. Wang, H. Duan, J. Liu, S. Chen, et al., “Aigciqa2023: A large-scale i mage quality assessment database for ai generated images: from the pers pectives of quality, authenticity and correspondence,” in CAAI Internati onal Conference on Artificial Intelligence, 2023, pp. 46–57
2023
-
[24]
Blind image quality assessment using a deep bilinear convolutional neural network,
W. Zhang, K. Ma, J. Yan, D. Deng, and Z. Wang, “Blind image quality assessment using a deep bilinear convolutional neural network,” IEEE T ransactions on Circuits and Systems for Video Technology, vol. 30, no. 1, pp. 36–47, 2018
2018
-
[25]
A multi-dimensional aesthetic quality assessment model for mobile game images,
T. Wang, W. Sun, X. Min, W. Lu, et al., “A multi-dimensional aesthetic quality assessment model for mobile game images,” in 2021 Internation al Conference on Visual Communications and Image Processing (VCI P), 2021, pp. 1–5
2021
-
[26]
Blindly assess image quality in the wild guided by a self-adaptive hyper network,
S. Su, Q. Yan, Y. Zhu, C. Zhang, et al., “Blindly assess image quality in the wild guided by a self-adaptive hyper network,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (C VPR), 2020, pp. 3667–3676
2020
-
[27]
Blind quality assessment for in-th e-wild images via hierarchical feature fusion and iterative mixed databas e training,
W. Sun, X. Min, D. Tu, S. Ma, et al., “Blind quality assessment for in-th e-wild images via hierarchical feature fusion and iterative mixed databas e training,” IEEE Journal of Selected Topics in Signal Processing, vol. 1 7, no. 6, pp. 1178–1192, 2023
2023
-
[28]
Image quality assessment using contrastive learning,
P. C. Madhusudana, N. Birkbeck, Y. Wang, B. Adsumilli, et al., “Image quality assessment using contrastive learning,” IEEE Transactions on Im age Processing, vol. 31, pp. 4149–4161, 2022
2022
-
[29]
Bringing textual prompt to ai-generated ima ge quality assessment,
B. Qu, H. Li, and W. Gao, “Bringing textual prompt to ai-generated ima ge quality assessment,” in 2024 IEEE International Conference on Multi media and Expo (ICME), 2024, pp. 1–6
2024
-
[30]
Moe-agiqa: Mixture-of-expert s boosted visual perception-driven and semantic-aware quality assessme nt for ai-generated images,
J. Yang, J. Fu, W. Zhang, W. Cao, et al., “Moe-agiqa: Mixture-of-expert s boosted visual perception-driven and semantic-aware quality assessme nt for ai-generated images,” in Proceedings of the IEEE/CVF Conferenc e on Computer Vision and Pattern Recognition (CVPR), 2024, pp. 6...
2024
-
[31]
Sf-iqa: Quality and similarity integr ation for ai generated image quality assessment,
Z. Yu, F. Guan, Y. Lu, X. Li, et al., “Sf-iqa: Quality and similarity integr ation for ai generated image quality assessment,” in Proceedings of the I EEE/CVF Conference on Computer Vision and Pattern Recognition (CV PR), 2024, pp. 6692–6701
2024
-
[32]
Pscr: Patches sampling-based co ntrastive regression for aigc image quality assessment,
J. Yuan, X. Cao, L. Cao, J. Lin, et al., “Pscr: Patches sampling-based co ntrastive regression for aigc image quality assessment,” arXiv preprint a rXiv:2312.05897, 2023
2023 arXiv
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.