Pith. sign in

REVIEW 3 major objections 6 minor 63 references

This challenge report claims that instruction-tuned large multimodal models can perform open-ended visual quality comparison across image pairs and groups, with the top ensemble reaching 75.7% top-1 accuracy on a new 4,000-question benchmar

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A challenge report finding that ensemble-tuned LMMs reach 75.7% accuracy on a 4,000-question visual quality comparison benchmark, only ~0.2 points above a Qwen2.5-VL-72B baseline.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection A useful, well-documented challenge report that overstates its case: the top methods are statistically indistinguishable from the Qwen2.5-VL-72B baseline. the 3 major comments →

arxiv 2509.09190 v1 pith:JSIFMBA3 submitted 2025-09-11 cs.CV

VQualA 2025 Challenge on Visual Quality Comparison for Large Multimodal Models: Methods and Results

Hanwei Zhu , Haoning Wu , Zicheng Zhang , Lingyu Zhu , Yixuan Li , Peilin Chen , Shiqi Wang , Chris Wei Zhou
show 21 more authors
Linhan Cao Wei Sun Xiangyang Zhu Weixia Zhang Yucheng Zhu Jing Liu Dandan Zhu Guangtao Zhai Xiongkuo Min Zhichao Zhang Xinyue Li Shubo Xu Anh Dao Yifan Li Hongyuan Yu Jiaojiao Yi Yiding Tian Yupeng Wu Feiran Sun Lijuan Liao Song Jiang
This is my paper
classification cs.CV
keywords visual quality comparisonlarge multimodal modelsimage quality assessmentinstruction tuningbenchmarkmultiple-choice questionsensemble votingcoarse-to-fine pairing
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper presents the Visual Quality Comparison track of the VQualA 2025 challenge. It introduces MICBench, a benchmark of 4,000 multiple-choice questions on pairs, triplets, and quadruplets of images, paired with the Co-Instruct-562K instruction-tuning corpus. The central claim is that instruction-tuned large multimodal models can reason about fine-grained visual quality differences across multiple images, rather than merely regress a scalar quality score. The top ensemble of three models reaches 75.7% top-1 accuracy, which the paper reads as evidence that multimodal reasoning plus instruction tuning is a viable path for open-domain quality comparison. A sympathetic reader would care because it moves image quality assessment from scalar scores toward interpretable, comparative judgments.

Core claim

The discovery the paper argues for is that open-ended visual quality comparison is learnable by instruction-tuned large multimodal models: fine-tuning on Co-Instruct-562K enables models to answer comparative questions like 'which image is sharper?' with explicit rationales, and aggregating several such models by majority voting improves accuracy. On MICBench's 2,000-question test set, the top ensemble achieves 0.7570 top-1 accuracy overall, with notably higher accuracy on three-image (0.8452) and four-image (0.8458) comparisons than on pairs (0.6713). The paper presents these results as demonstrating substantial advancements in visual quality comparison powered by multimodal reasoning and in

What carries the argument

The load-bearing machinery is a coarse-to-fine pairing protocol that constructs comparable image groups—same content and distortion type with different levels, same content and level with different distortion types, or real distortions within the same MOS interval—and casts comparison as multiple-choice questions. The training side uses Co-Instruct-562K, a 562K-sample instruction corpus built by merging human single-image descriptions (Merge2Compare) and using a large multimodal teacher model to generate pseudo-labels on unlabeled groups (Teach2Compare). The evaluation side is MICBench, a 4,000-MCQ benchmark spanning two-, three-, and four-image groups with 'which', 'yes/no', and 'what/how'

Load-bearing premise

The benchmark's 4,000 questions and the training corpus are constructed from the same source databases and the same pairing protocol, so the central assumption is that top-1 accuracy on MICBench measures generalizable open-domain visual quality comparison ability rather than in-distribution performance on benchmark families the organizers already built and released.

What would settle it

Assemble a fresh comparison benchmark from image datasets and distortion types not used in Co-Instruct-562K or MICBench, then run the winning ensemble without any additional fine-tuning. If its accuracy drops to near chance or to the level of an untrained baseline while in-distribution MICBench accuracy remains near 75%, the leaderboard metric is measuring familiarity with the data pipeline, not generalizable comparison ability.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Instruction-tuned multimodal models can handle multi-image comparisons beyond pairs, including triplets and quadruplets, without task-specific quality heads.
  • Ensembling several fine-tuned models through majority voting improves top-1 accuracy over any single model and was the winning strategy in this challenge.
  • MICBench enables granular analysis by group size, distortion type, and question format, allowing separate tracking of coarse- versus fine-grained comparison skill.
  • Large base models without challenge-specific training can reach near the top-team performance level, suggesting that model scale and general instruction tuning are strong contributors.
  • The combination of Co-Instruct-562K training and MICBench evaluation offers a reusable protocol for future work on open-domain quality comparison.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: If MICBench and Co-Instruct-562K are built from the same source databases and the same coarse-to-fine pairing protocol, then top-1 accuracy on MICBench may partly measure in-distribution fit to benchmark families the organizers already designed, rather than fully generalizable open-domain comparison ability.
  • Beyond the paper: The higher accuracy on three- and four-image questions than on pairs is counterintuitive if difficulty scaled with the number of alternatives; it may reflect that group questions often ask to identify a standout image, while pairs require finer relative judgment. A testable extension is to break down accuracy by question type and quality-level spacing.
  • Beyond the paper: The success of majority voting suggests individual multimodal models have complementary perceptual biases. An untested next step is to measure pairwise agreement among models and use disagreement to guide active data collection or more targeted preference optimization.
  • Beyond the paper: The same comparative protocol could transfer to video quality assessment or AI-generated content quality, where asking which frame or clip is better is more natural than assigning scalar scores.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This paper reports the organization and outcomes of the VQualA 2025 Visual Quality Comparison challenge. The track introduces a multi-image quality comparison benchmark (MICBench) with 4,000 human-annotated multiple-choice questions and uses it to rank five submitted LMM-based systems against three baseline LMMs. The paper describes the Co-Instruct-562K training corpus, the coarse-to-fine pairing protocol used to construct MICBench, and the methods of the top five teams. It claims that the results 'illustrate substantial advancements in visual quality comparison powered by multimodal reasoning and instruction-tuned models.'

Significance. The challenge fills a genuine gap: multi-image comparison is underexplored compared to single-image MOS regression, and the public release of MICBench and the detailed method descriptions are valuable to the community. The evaluation uses a held-out human-annotated test set, which is a real strength. However, the central quantitative claim is not supported by the reported statistics: the top ensemble (0.7570) is only 0.002 above the Qwen2.5-VL-72B baseline (0.7550) on 2,000 questions, and no confidence intervals or significance tests are provided. In addition, the training corpus and test benchmark share the same origin and construction protocol, so the claim of open-domain generalization requires additional evidence or explicit acknowledgment of this limitation. The paper is a useful challenge summary, but the interpretation of the leaderboard needs substantial revision.

major comments (3)
  1. [Section 4, Table 1] The differences among the top four teams and the Qwen2.5-VL-72B baseline are within 0.006, i.e., 12 correct answers out of 2,000. The winner-vs-baseline difference is 4 correct answers (0.002). With N=2000, the standard error of a proportion at p≈0.75 is about 0.0097, so the observed differences are less than one standard error. The paper reports no confidence intervals, bootstrap estimates, or paired significance tests. As written, the leaderboard does not demonstrate that any submitted method improves on the Qwen2.5-VL-72B baseline, and the term 'substantial advancements' is unjustified. The authors should either compute and report paired significance tests (e.g., McNemar) and confidence intervals, or temper the claim to note that the top methods are statistically indistinguishable from the strongest baseline at this test-set size.
  2. [Sections 3.1 and 3.2, Fig. 2] The evaluation set MICBench and the training set Co-Instruct-562K are built by the same organizers using the same source databases (Q-Instruct, etc.) and the same coarse-to-fine pairing protocol from Ref. [60] (see also Fig. 1). This raises a distribution-shift concern for the claim that the benchmark measures 'open-domain visual quality reasoning and comparison' (Sec. 1). If the test questions are drawn from the same pipeline as the training data, top-1 accuracy may reflect in-distribution familiarity rather than generalizable comparison ability. The paper should add an analysis of the overlap between training and test sources, or evaluate on an independent benchmark (e.g., a different IQA comparison set), or explicitly state this limitation and temper the generalization claim.
  3. [Section 3.2, benchmark construction] The paper states that MICBench MCQs are 'authored and verified by expert annotators' and that group-level questions are 'validated through controlled human annotation procedures,' but no inter-annotator agreement, number of annotators, or per-question filtering statistics are reported. Since the benchmark is the basis for all conclusions, the absence of label-reliability information is a significant omission. The authors should provide at least a summary of annotation agreement and the number of questions discarded or revised after validation.
minor comments (6)
  1. [Section 2.1] References [41] and [42] refer to the same SSIM paper; consolidate into a single citation.
  2. [Table 1 header] The 'Leader' column is ambiguous; indicate that it is the team leader or contact person.
  3. [Sections 3.1 and 5.5] Section 3.1 lists four subsets (202K/222K/77K/61K), while Section 5.5 names four different subsets (Q-Instruct-200K, Merge2Compare, Teach2Compare-general, Teach2Compare-QA); reconcile the naming.
  4. [Section 5.4] The Q-align-4bit model is not cited; add a citation and clarify whether it is the Q-Align model from Ref. [44].
  5. [Table 1 and Section 5.5] The 'Extra Data' column marks Labubu with ✓, but Section 5.5 says training is on Co-Instruct-562K; clarify what extra data is used and how it was incorporated.
  6. [General] The paper would benefit from a short 'limitations' paragraph acknowledging the statistical power and same-origin issues discussed in the major comments.

Circularity Check

0 steps flagged

No circularity found: the challenge result is an empirical, human-grounded measurement; self-citations support benchmark design but do not define the outcome.

full rationale

The paper's central conclusion ('These results illustrate substantial advancements...') is an interpretation of the leaderboard in Table 1, not a derivation. MICBench's ground truth is '4,000 human-annotated multiple-choice questions (MCQs)', and evaluation is 'comparing the model-predicted choice against the ground-truth answer key'. The top-1 accuracy is therefore an externally grounded measurement, not a quantity defined by the training data or by a fitted parameter. The in-distribution concern—teams train on Co-Instruct-562K and are tested on MICBench, whose pairwise protocol is 'following the coarse-to-fine pairing protocol proposed in [60]' and whose group format is 'inheriting the structural design and task format of MICBench in Co-Instruct [45]'—is a legitimate generalization-validity issue, but it does not make the reported accuracy equivalent to the training input by construction: the test labels come from human annotators, and no submission's score is computed from its own training objective. Similarly, the closeness of the top score (0.7570) to the Qwen2.5-VL-72B baseline (0.7550) without significance testing is a statistical-support concern, not a circularity. Self-citations to [45] and [60] are used as methodological building blocks for benchmark construction, but the evaluation itself is an independent empirical measurement. Hence no circular step can be exhibited under the required standard.

Axiom & Free-Parameter Ledger

1 free parameters · 3 axioms · 0 invented entities

No mathematical free parameters are fitted; the only hand-set threshold is the MOS interval [0,25). The central claim rests on the reliability of human MCQ annotations, the validity of MCQ accuracy as a proxy for open-ended reasoning, and the representativeness of the source image databases. No new physical or conceptual entities are postulated; MICBench is a dataset, not an entity in the sense of a new particle or force.

free parameters (1)
  • MOS interval [0,25) for realistic fine-grained pairs = None (hand-chosen)
    Used in pairing rule 3 (Fig. 1e-f) to define 'similarly rated' realistic distortion pairs; the bin boundary is chosen without stated justification.
axioms (3)
  • domain assumption Human expert annotations of MICBench are correct and consistent.
    Used as ground truth for all 4,000 MCQs; no inter-annotator agreement or adjudication statistics are reported (Sec. 3.2).
  • domain assumption Top-1 MCQ accuracy is a valid measure of open-ended visual quality comparison ability.
    The challenge motivates open-ended reasoning and natural language explanations, but the only scored metric is multiple-choice accuracy (Sec. 3.2).
  • domain assumption Source image databases with MOS labels provide trustworthy perceptual ground truth for constructing comparisons.
    Pairing rules rely on MOS values and synthetic distortion levels from existing databases (Sec. 3.2, Fig. 1).

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of VQualA 2025 Challenge on Visual Quality Comparison for Large Multimodal Models: Methods and Results." pith.science (2026). https://pith.science/paper/JSIFMBA3

@misc{pith2026250909190,
  author       = {Pith},
  title        = {Pith review of: VQualA 2025 Challenge on Visual Quality Comparison for Large Multimodal Models: Methods and Results},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JSIFMBA3}},
  note         = {Machine review of arXiv:2509.09190}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

This paper presents a summary of the VQualA 2025 Challenge on Visual Quality Comparison for Large Multimodal Models (LMMs), hosted as part of the ICCV 2025 Workshop on Visual Quality Assessment. The challenge aims to evaluate and enhance the ability of state-of-the-art LMMs to perform open-ended and detailed reasoning about visual quality differences across multiple images. To this end, the competition introduces a novel benchmark comprising thousands of coarse-to-fine grained visual quality comparison tasks, spanning single images, pairs, and multi-image groups. Each task requires models to provide accurate quality judgments. The competition emphasizes holistic evaluation protocols, including 2AFC-based binary preference and multi-choice questions (MCQs). Around 100 participants submitted entries, with five models demonstrating the emerging capabilities of instruction-tuned LMMs on quality assessment. This challenge marks a significant step toward open-domain visual quality reasoning and comparison and serves as a catalyst for future research on interpretable and human-aligned quality evaluation systems.

Figures

Figures reproduced from arXiv: 2509.09190 by Anh Dao, Chris Wei Zhou, Dandan Zhu, Feiran Sun, Guangtao Zhai, Hanwei Zhu, Haoning Wu, Hongyuan Yu, Jiaojiao Yi, Jing Liu, Lijuan Liao, Lingyu Zhu, Linhan Cao, Peilin Chen, Shiqi Wang, Shubo Xu, Song Jiang, Wei Sun, Weixia Zhang, Xiangyang Zhu, Xinyue Li, Xiongkuo Min, Yiding Tian, Yifan Li, Yixuan Li, Yucheng Zhu, Yupeng Wu, Zhichao Zhang, Zicheng Zhang.

Figure 1
Figure 1. Figure 1: Illustration of three pairing rules for fine-grained quality comparison. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Dataset Card of MIC-Bench, made up of (a) Which questions, (b) Yes-or-No questions, and (c) Other types of questions on three/four images. Image by courtesy of [45]. wise comparative reasoning across multiple distorted in￾stances. • 61K quadruplets: These involve four-image compar￾isons, focusing on nuanced ranking, identification of worst/best cases, or structured reasoning across complex quality differen… view at source ↗
Figure 3
Figure 3. Figure 3: The overall framework of Team ECNU-SJTU VQA. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: The overall framework of Team Digital Ocean. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 6
Figure 6. Figure 6: The overall framework of Team Labubu. These quality-aware and ranking embeddings are fused with standard visual and textual embeddings, enabling the model to deliver improved fine-grained quality comparison. Training Details: The proposed method extends DepictQA-Wild, leveraging a frozen CLIP-ViT-L/14 visual encoder and a Vicuna-7B large language model (LLM). The Quality Ranking Constraint (QRC) and Qual￾i… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

63 extracted references · 9 linked inside Pith

  1. [1]

    Claude 3.5 Sonnet: Faster, smarter, and more useful.https : / / www

    Anthropic. Claude 3.5 Sonnet: Faster, smarter, and more useful.https : / / www . anthropic . com / news / claude-3-5-sonnet, 2024. Accessed: 2025-05-15. 1

    2024

  2. [2]

    Qwen2.5-VL technical report.arXiv preprint arXiv:2502.13923, 2025

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2.5-VL technical report.arXiv preprint arXiv:2502.13923, 2025. 1

    Pith/arXiv arXiv 2025

  3. [3]

    Beyond keypoint coding: Temporal evo- lution inference with compact feature representation for talk- ing face video compression

    Bolin Chen, Zhao Wang, Binzhe Li, Rongqun Lin, Shiqi Wang, and Yan Ye. Beyond keypoint coding: Temporal evo- lution inference with compact feature representation for talk- ing face video compression. InData Compression Confer- ence, pages 13–22. IEEE, 2022. 1

    2022

  4. [4]

    Compact temporal trajectory representation for talking face video compression.IEEE Transactions on Circuits and Systems for Video Technology, 33(11):7009–7023, 2023

    Bolin Chen, Zhao Wang, Binzhe Li, Shiqi Wang, and Yan Ye. Compact temporal trajectory representation for talking face video compression.IEEE Transactions on Circuits and Systems for Video Technology, 33(11):7009–7023, 2023. 1

    2023

  5. [5]

    Gap-closing matters: Percep- tual quality evaluation and optimization of low-light image enhancement.IEEE Transactions on Multimedia, 26:3430– 3443, 2023

    Baoliang Chen, Lingyu Zhu, Hanwei Zhu, Wenhan Yang, Linqi Song, and Shiqi Wang. Gap-closing matters: Percep- tual quality evaluation and optimization of low-light image enhancement.IEEE Transactions on Multimedia, 26:3430– 3443, 2023. 2

    2023

  6. [6]

    Deep feature statistics mapping for generalized screen content image quality assessment.IEEE Transactions on Image Processing, 2024

    Baoliang Chen, Hanwei Zhu, Lingyu Zhu, Shiqi Wang, and Sam Kwong. Deep feature statistics mapping for generalized screen content image quality assessment.IEEE Transactions on Image Processing, 2024. 2

    2024

  7. [7]

    Pleno- generation: A scalable generative face video compression framework with bandwidth intelligence.arXiv preprint arXiv:2502.17085, 2025

    Bolin Chen, Hanwei Zhu, Shanzhi Yin, Lingyu Zhu, Jie Chen, Ru-Ling Liao, Shiqi Wang, and Yan Ye. Pleno- generation: A scalable generative face video compression framework with bandwidth intelligence.arXiv preprint arXiv:2502.17085, 2025. 2

    Pith/arXiv arXiv 2025

  8. [8]

    Debiased mapping for full- reference image quality assessment.IEEE Trans

    Baoliang Chen, Hanwei Zhu, Lingyu Zhu, Shanshe Wang, Jingshan Pan, and Shiqi Wang. Debiased mapping for full- reference image quality assessment.IEEE Trans. Multime- dia, pages 1–12, 2025. 1

    2025

  9. [9]

    Vquala 2025 chal- lenge on genai-bench aigc video quality assessment: Meth- ods and results

    Ying Chen, Huasheng Wang, Pengxiang Xiao, Yukang Ding, Enpeng Liu, Chris Wei Zhou, and et al. Vquala 2025 chal- lenge on genai-bench aigc video quality assessment: Meth- ods and results. InProceedings of the IEEE/CVF Conference on Computer Vision (ICCV) Workshops, pages 1–11, 2025. 2

    2025

  10. [10]

    Simoncelli

    Keyan Ding, Kede Ma, Shiqi Wang, and Eero P. Simoncelli. Image quality assessment: Unifying structure and texture similarity.IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, 44(5):2567–2581, 2022. 1, 2

    2022

  11. [11]

    Per- ceptual quality assessment of HDR deghosting algorithms

    Yuming Fang, Hanwei Zhu, Kede Ma, and Zhou Wang. Per- ceptual quality assessment of HDR deghosting algorithms. InIEEE International Conference on Image Processing, pages 3165–3169, 2017. 2

    2017

  12. [12]

    Perceptual quality assessment of smartphone photog- raphy

    Yuming Fang, Hanwei Zhu, Yan Zeng, Kede Ma, and Zhou Wang. Perceptual quality assessment of smartphone photog- raphy. InIEEE Conference on Computer Vision and Pattern Recognition, pages 3677–3686, 2020. 1

    2020

  13. [13]

    Superpixel-based quality assessment of multi- exposure image fusion for both static and dynamic scenes

    Yuming Fang, Yan Zeng, Wenhui Jiang, Hanwei Zhu, and Jiebin Yan. Superpixel-based quality assessment of multi- exposure image fusion for both static and dynamic scenes. IEEE Transactions on Image Processing, 30:2526–2537,

  14. [14]

    Vquala 2025 doc- ument image quality assessment challenge

    Fan Huang, Xiongkuo Min, Zhichao Ma, Xiaohong Liu, Chris Wei Zhou, Guangtao Zhai, and et al. Vquala 2025 doc- ument image quality assessment challenge. InProceedings of the IEEE/CVF Conference on Computer Vision (ICCV) Workshops, pages 1–8, 2025. 2

    2025

  15. [15]

    S 2-csnet: Scale-aware scalable sampling network for image compressive sensing

    Chen Hui, Haiqi Zhu, Shuya Yan, Shaohui Liu, Feng Jiang, and Debin Zhao. S 2-csnet: Scale-aware scalable sampling network for image compressive sensing. InACM Multime- dia, pages 1–10. 1

  16. [16]

    Rate-adaptive neural network for image compressive sensing.IEEE Transactions on Mul- timedia, 26:2515–2530, 2023

    Chen Hui, Shengping Zhang, Wenxue Cui, Shaohui Liu, Feng Jiang, and Debin Zhao. Rate-adaptive neural network for image compressive sensing.IEEE Transactions on Mul- timedia, 26:2515–2530, 2023. 1

    2023

  17. [17]

    GPT-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perel- man, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Weli- hinda, Alan Hayes, Alec Radford, et al. GPT-4o system card. arXiv preprint arXiv:2410.21276, 2024. 1

    Pith/arXiv arXiv 2024

  18. [18]

    Scope of va- lidity of psnr in image/video quality assessment.Electronics letters, 44(13):800–801, 2008

    Quan Huynh-Thu and Mohammed Ghanbari. Scope of va- lidity of psnr in image/video quality assessment.Electronics letters, 44(13):800–801, 2008. 2

    2008

  19. [19]

    Convolu- tional neural networks for no-reference image quality assess- ment

    Le Kang, Peng Ye, Yi Li, and David Doermann. Convolu- tional neural networks for no-reference image quality assess- ment. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 1733–1740, 2014. 2

    2014

  20. [20]

    MUSIQ: Multi-scale image quality trans- former

    Junjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar, and Feng Yang. MUSIQ: Multi-scale image quality trans- former. InIEEE International Conference on Computer Vi- sion, pages 5148–5157, 2021. 2

    2021

  21. [21]

    Detect and locate: Expos- ing face manipulation by semantic-and noise-level telltales

    Chenqi Kong, Baoliang Chen, Haoliang Li, Shiqi Wang, An- derson Rocha, and Sam Kwong. Detect and locate: Expos- ing face manipulation by semantic-and noise-level telltales. IEEE Transactions on Information Forensics and Security, 17:1741–1756, 2022. 1

    2022

  22. [22]

    Moe-ffd: Mixture of experts for generalized and parameter-efficient face forgery detection.arXiv preprint arXiv:2404.08452,

    Chenqi Kong, Anwei Luo, Peijun Bao, Yi Yu, Haoliang Li, Zengwei Zheng, Shiqi Wang, and Alex C Kot. Moe-ffd: Mixture of experts for generalized and parameter-efficient face forgery detection.arXiv preprint arXiv:2404.08452,

    Pith/arXiv arXiv

  23. [23]

    Pixel-inconsistency modeling for image manipulation localization.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025

    Chenqi Kong, Anwei Luo, Shiqi Wang, Haoliang Li, Ander- son Rocha, and Alex C Kot. Pixel-inconsistency modeling for image manipulation localization.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025. 1

    2025

  24. [24]

    LLaV A-Onevision: Easy visual task transfer

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Zi- wei Liu, et al. LLaV A-Onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326, 2024. 1

    Pith/arXiv arXiv 2024

  25. [25]

    Vquala 2025 challenge on engagement prediction for short videos: Methods and results

    Dasong Li, Sizhuo Ma, Hang Hua, Wenjie Li, Jian Wang, Chris Wei Zhou, Fengbin Guan, Xin Li, Zihao Yu, Yiting Lu, Ru-Ling Liao, Yan Ye, Zhibo Chen, Wei Sun, Linhan Cao, Yuqin Cao, Weixia Zhang, Wen Wen, Kaiwei Zhang, 9 Zijian Chen, Fangfang Lu, Xiongkuo Min, Guangtao Zhai, Erjia Xiao, Lingfeng Zhang, Zhenjie Su, Hao Cheng, Yu Liu, Renjing Xu, Long Chen, Xi...

    2025

  26. [26]

    Q-insight: Understanding im- age quality via visual reinforcement learning.arXiv preprint arXiv:2503.22679, 2025

    Weiqi Li, Xuanyu Zhang, Shijie Zhao, Yabin Zhang, Junlin Li, Li Zhang, and Jian Zhang. Q-insight: Understanding im- age quality via visual reinforcement learning.arXiv preprint arXiv:2503.22679, 2025. 3

    Pith/arXiv arXiv 2025

  27. [27]

    Boosting clip adaptation for image quality assessment via meta-prompt learning and gradient regularization.arXiv preprint arXiv:2409.05381, 2024

    Xudong Li, Zihao Huang, Runze Hu, Yan Zhang, Liujuan Cao, and Rongrong Ji. Boosting clip adaptation for image quality assessment via meta-prompt learning and gradient regularization.arXiv preprint arXiv:2409.05381, 2024. 3

    arXiv 2024

  28. [28]

    Vquala 2025 challenge on image super-resolution generated content qual- ity assessment: Methods and results

    Yixiao Li, Xin Li, Chris Wei Zhou, Shuo Xing, Hadi Amir- pour, Xiaoshuai Hao, Guanghui Yue, Baoquan Zhao, Weide Liu, Xiaoyuan Yang, Zhengzhong Tu, and et al. Vquala 2025 challenge on image super-resolution generated content qual- ity assessment: Methods and results. InProceedings of the IEEE/CVF Conference on Computer Vision (ICCV) Work- shops, pages 1–10...

    2025

  29. [29]

    DeepWSD: Projecting degradations in perceptual space to Wasserstein distance in deep feature space

    Xingran Liao, Baoliang Chen, Hanwei Zhu, Shiqi Wang, Mingliang Zhou, and Sam Kwong. DeepWSD: Projecting degradations in perceptual space to Wasserstein distance in deep feature space. InACM International Conference on Multimedia, pages 970–978, 2022. 2

    2022

  30. [30]

    Perceptual visual quality met- rics: A survey.Journal of Visual Communication and Image Representation, 22(4):297–312, 2011

    Weisi Lin and C-C Jay Kuo. Perceptual visual quality met- rics: A survey.Journal of Visual Communication and Image Representation, 22(4):297–312, 2011. 1

    2011

  31. [31]

    Rankiqa: Learning from rankings for no-reference image quality assessment

    Xialei Liu, Joost Van De Weijer, and Andrew D Bagdanov. Rankiqa: Learning from rankings for no-reference image quality assessment. InProceedings of the IEEE international conference on computer vision, pages 1040–1049, 2017. 2

    2017

  32. [32]

    A quality-of-experience database for adaptive omni- directional video streaming.IEEE Journal of Selected Topics in Signal Processing, 17(5):949–963, 2023

    Xuelin Liu, Jiebin Yan, Zheng Wan, Yuming Fang, and Zhou Wang. A quality-of-experience database for adaptive omni- directional video streaming.IEEE Journal of Selected Topics in Signal Processing, 17(5):949–963, 2023. 2

    2023

  33. [33]

    Vquala 2025 challenge on face image quality assessment: Methods and results

    Sizhuo Ma, Wei-Ting Chen, Qiang Gao, Jian Wang, Chris Wei Zhou, Wei Sun, Weixia Zhang, Linhan Cao, Jun Jia, Xiangyang Zhu, Dandan Zhu, Xiongkuo Min, Guang- tao Zhai, Baoying Chen, Xiongwei Xiao, Jishen Zeng, Wei Wu, Tiexuan Lou, Yuchen Tan, Chunyi Song, Zhiwei Xu, MohammadAli Hamidi, Hadi Amirpour, Mingyin Bai, Ji- awang Du, Zhenyu Jiang, Zilong Lu, Zigua...

    2025

  34. [34]

    Image quality assessment us- ing contrastive learning.IEEE Transactions on Image Pro- cessing, 31:4149–4161, 2022

    Pavan C Madhusudana, Neil Birkbeck, Yilin Wang, Balu Adsumilli, and Alan C Bovik. Image quality assessment us- ing contrastive learning.IEEE Transactions on Image Pro- cessing, 31:4149–4161, 2022. 2

    2022

  35. [35]

    PieAPP: Perceptual image-error assessment through pairwise preference

    Ekta Prashnani, Hong Cai, Yasamin Mostofi, and Pradeep Sen. PieAPP: Perceptual image-error assessment through pairwise preference. InIEEE Conference on Computer Vi- sion and Pattern Recognition, pages 1808–1817, 2018. 2

    2018

  36. [36]

    Hamid R Sheikh and Alan C. Bovik. Image information and visual quality.IEEE Transactions on Image Processing, 15 (2):430–444, 2006. 2

    2006

  37. [37]

    Blindly assess image qual- ity in the wild guided by a self-adaptive hyper network

    Shaolin Su, Qingsen Yan, Yu Zhu, Cheng Zhang, Xin Ge, Jinqiu Sun, and Yanning Zhang. Blindly assess image qual- ity in the wild guided by a self-adaptive hyper network. In IEEE Conference on Computer Vision and Pattern Recogni- tion, pages 3664–3673, 2020. 2

    2020

  38. [38]

    Perceptual quality assessment of 360 images based on generative scanpath representation.arXiv preprint arXiv:2309.03472, 2023

    Xiangjie Sui, Hanwei Zhu, Xuelin Liu, Yuming Fang, Shiqi Wang, and Zhou Wang. Perceptual quality assessment of 360 images based on generative scanpath representation.arXiv preprint arXiv:2309.03472, 2023. 2

    Pith/arXiv arXiv 2023

  39. [39]

    AI-generated image quality assess- ment in visual communication

    Yu Tian, Yixuan Li, Baoliang Chen, Hanwei Zhu, Shiqi Wang, and Sam Kwong. AI-generated image quality assess- ment in visual communication. InAAAI Conference on Arti- ficial Intelligence, pages 7392–7400, 2025. 1

    2025

  40. [40]

    Mstriq: No reference image quality assessment based on swin transformer with multi- stage fusion

    Jing Wang, Haotian Fan, Xiaoxia Hou, Yitian Xu, Tao Li, Xuechao Lu, and Lean Fu. Mstriq: No reference image quality assessment based on swin transformer with multi- stage fusion. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1269– 1278, 2022. 2

    2022

  41. [41]

    Image quality assessment: from error visibility to structural similarity.IEEE Transactions on Image Process- ing, 13(4):600–612, 2004

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Si- moncelli. Image quality assessment: from error visibility to structural similarity.IEEE Transactions on Image Process- ing, 13(4):600–612, 2004. 2

    2004

  42. [42]

    Bovik, Hamid R Sheikh, and Eero P

    Zhou Wang, Alan C. Bovik, Hamid R Sheikh, and Eero P. Simoncelli. Image quality assessment: From error visibility to structural similarity.IEEE Transactions on Image Pro- cessing, 13(4):600–612, 2004. 2

    2004

  43. [43]

    Q-Instruct: Improving low-level visual abil- ities for multi-modality foundation models

    Haoning Wu, Zicheng Zhang, Erli Zhang, Chaofeng Chen, Liang Liao, Annan Wang, Kaixin Xu, Chunyi Li, Jingwen Hou, Guangtao Zhai, Geng Xue, Wenxiu Sun, Qiong Yan, and Weisi Lin. Q-Instruct: Improving low-level visual abil- ities for multi-modality foundation models. InIEEE Con- ference on Computer Vision and Pattern Recognition, pages 25490–25500, 2024. 1, 2, 3

    2024

  44. [44]

    Q-Align: Teaching LMMs for visual scoring via discrete text-defined levels

    Haoning Wu, Zicheng Zhang, Weixia Zhang, Chaofeng Chen, Liang Liao, Chunyi Li, Yixuan Gao, Annan Wang, Erli Zhang, Wenxiu Sun, Qiong Yan, Xiongkuo Min, Guang- tao Zhai, and Weisi Lin. Q-Align: Teaching LMMs for visual scoring via discrete text-defined levels. InInternational Con- ference on Machine Learning, pages 54015–54029, 2024. 3

    2024

  45. [45]

    Towards open-ended visual quality comparison

    Haoning Wu, Hanwei Zhu, Zicheng Zhang, Erli Zhang, Chaofeng Chen, Liang Liao, Chunyi Li, Annan Wang, Wenxiu Sun, Qiong Yan, et al. Towards open-ended visual quality comparison. InEuropean Conference on Computer Vision, pages 360–377, 2024. 1, 2, 3, 4, 5

    2024

  46. [46]

    A comprehensive study of multimodal large language mod- 10 els for image quality assessment

    Tianhe Wu, Kede Ma, Jie Liang, Yujiu Yang, and Lei Zhang. A comprehensive study of multimodal large language mod- 10 els for image quality assessment. InEuropean Conference on Computer Vision, pages 143–160. Springer, 2024. 1

    2024

  47. [47]

    Visualquality-r1: Reasoning-induced image quality assess- ment via reinforcement learning to rank.arXiv preprint arXiv:2505.14460, 2025

    Tianhe Wu, Jian Zou, Jie Liang, Lei Zhang, and Kede Ma. Visualquality-r1: Reasoning-induced image quality assess- ment via reinforcement learning to rank.arXiv preprint arXiv:2505.14460, 2025. 3

    arXiv 2025

  48. [48]

    Subjective and objective quality of experi- ence of free viewpoint videos.IEEE Transactions on Image Processing, 31:3896–3907, 2022

    Jiebin Yan, Jing Li, Yuming Fang, Zhaohui Che, Xue Xia, and Yang Liu. Subjective and objective quality of experi- ence of free viewpoint videos.IEEE Transactions on Image Processing, 31:3896–3907, 2022. 2

    2022

  49. [49]

    From patches to pictures (paq-2-piq): Mapping the perceptual space of pic- ture quality

    Zhenqiang Ying, Haoran Niu, Praful Gupta, Dhruv Maha- jan, Deepti Ghadiyaram, and Alan Bovik. From patches to pictures (paq-2-piq): Mapping the perceptual space of pic- ture quality. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3575–3585,

  50. [50]

    Depicting beyond scores: Advanc- ing image quality assessment through multi-modal language models

    Zhiyuan You, Zheyuan Li, Jinjin Gu, Zhenfei Yin, Tianfan Xue, and Chao Dong. Depicting beyond scores: Advanc- ing image quality assessment through multi-modal language models. InEuropean Conference on Computer Vision, pages 259–276, 2024. 3

    2024

  51. [51]

    Teaching large language models to regress accurate image quality scores using score distribution

    Zhiyuan You, Xin Cai, Jinjin Gu, Tianfan Xue, and Chao Dong. Teaching large language models to regress accurate image quality scores using score distribution. InIEEE Con- ference on Computer Vision and Pattern Recognition, 2025. 3

    2025

  52. [52]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. InIEEE Conference on Computer Vision and Pattern Recognition, pages 586–595,

  53. [53]

    Blind image quality assessment using a deep bilinear convolutional neural network.IEEE Transactions on Cir- cuits and Systems for Video Technology, 30(1):36–47, 2020

    Weixia Zhang, Kede Ma, Jia Yan, Dexiang Deng, and Zhou Wang. Blind image quality assessment using a deep bilinear convolutional neural network.IEEE Transactions on Cir- cuits and Systems for Video Technology, 30(1):36–47, 2020. 2

    2020

  54. [54]

    Uncertainty-aware blind image quality assessment in the laboratory and wild.IEEE Transactions on Image Pro- cessing, 30:3474–3486, 2021

    Weixia Zhang, Kede Ma, Guangtao Zhai, and Xiaokang Yang. Uncertainty-aware blind image quality assessment in the laboratory and wild.IEEE Transactions on Image Pro- cessing, 30:3474–3486, 2021. 2

    2021

  55. [55]

    Blind image quality assessment via vision- language correspondence: A multitask learning perspective

    Weixia Zhang, Guangtao Zhai, Ying Wei, Xiaokang Yang, and Kede Ma. Blind image quality assessment via vision- language correspondence: A multitask learning perspective. InIEEE Conference on Computer Vision and Pattern Recog- nition, pages 14071–14081, 2023. 2, 3

    2023

  56. [56]

    Faster super-resolution imaging with auto-correlation two-step deconvolution.arXiv preprint arXiv:1809.07410, 2018

    Weisong Zhao, Jian Liu, Chenqi Kong, Yixuan Zhao, Changliang Guo, Chenguang Liu, Xiangyan Ding, Xumin Ding, Jiubin Tan, and Haoyu Li. Faster super-resolution imaging with auto-correlation two-step deconvolution.arXiv preprint arXiv:1809.07410, 2018. 2

    Pith/arXiv arXiv 2018

  57. [57]

    Learning spatiotemporal interactions for user- generated video quality assessment.IEEE Transactions on Circuits and Systems for Video Technology, 33(3):1031– 1042, 2022

    Hanwei Zhu, Baoliang Chen, Lingyu Zhu, and Shiqi Wang. Learning spatiotemporal interactions for user- generated video quality assessment.IEEE Transactions on Circuits and Systems for Video Technology, 33(3):1031– 1042, 2022. 2

    2022

  58. [58]

    DeepDC: Deep distance correlation as a perceptual image quality evaluator.arXiv preprint arXiv:2211.04927, 2022

    Hanwei Zhu, Baoliang Chen, Lingyu Zhu, Shiqi Wang, and Weisi Lin. DeepDC: Deep distance correlation as a perceptual image quality evaluator.arXiv preprint arXiv:2211.04927, 2022. 2

    Pith/arXiv arXiv 2022

  59. [59]

    Video quality assessment for spatio- temporal resolution adaptive coding.IEEE Transactions on Circuits and Systems for Video Technology, 2024

    Hanwei Zhu, Baoliang Chen, Lingyu Zhu, Peilin Chen, Linqi Song, and Shiqi Wang. Video quality assessment for spatio- temporal resolution adaptive coding.IEEE Transactions on Circuits and Systems for Video Technology, 2024. 2

    2024

  60. [60]

    2afc prompting of large multimodal models for image quality assessment

    Hanwei Zhu, Xiangjie Sui, Baoliang Chen, Xuelin Liu, Peilin Chen, Yuming Fang, and Shiqi Wang. 2afc prompting of large multimodal models for image quality assessment. IEEE Transactions on Circuits and Systems for Video Tech- nology, 34(12):12873–12878, 2024. 1, 2, 4

    2024

  61. [61]

    Adaptive image quality assess- ment via teaching large multimodal model to compare

    Hanwei Zhu, Haoning Wu, Yixuan Li, Zicheng Zhang, Bao- liang Chen, Lingyu Zhu, Yuming Fang, Guangtao Zhai, Weisi Lin, and Shiqi Wang. Adaptive image quality assess- ment via teaching large multimodal model to compare. In Advances in Neural Information Processing Systems, pages 32611–32629, 2024. 3

    2024

  62. [62]

    Temporally consistent enhancement of low-light videos via spatial-temporal com- patible learning.International Journal of Computer Vision, 132(10):4703–4723, 2024

    Lingyu Zhu, Wenhan Yang, Baoliang Chen, Hanwei Zhu, Xiandong Meng, and Shiqi Wang. Temporally consistent enhancement of low-light videos via spatial-temporal com- patible learning.International Journal of Computer Vision, 132(10):4703–4723, 2024. 2

    2024

  63. [63]

    Unrolled decom- posed unpaired learning for controllable low-light video en- hancement

    Lingyu Zhu, Wenhan Yang, Baoliang Chen, Hanwei Zhu, Zhangkai Ni, Qi Mao, and Shiqi Wang. Unrolled decom- posed unpaired learning for controllable low-light video en- hancement. InEuropean Conference on Computer Vision, pages 329–347. Springer, 2024. 2 Organizers Title:VQualA 2025 Challenge on Visual Quality Compar- ison for Large Multimodal Models: Method...

    2024

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.