REVIEW 4 major objections 5 minor 35 references
CAS-IQA: Teaching Vision-Language Models for Synthetic Angiography Quality Assessment
T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read A vision-language model that compares a synthetic angiogram against its non-contrast mask and real contrast reference can score its quality more accurately than existing single-image IQA methods, and the authors build the CAS-3K dataset…
desk verdict Real first dataset for angiography IQA, but the performance claims need better statistics and the ground truth needs clinician validation. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the MUST (Multi-path featUre fuSion and rouTing) module, a token-level fusion-and-routing block. It computes vessel tokens by deformably aligning Contrast tokens to Mask tokens and subtracting them, then creates three branch representations: VMC uses self-attention over vessel tokens as queries into Generated tokens; VBD uses a convolutional block to extract local bifurcation-level tokens before cross-attention; OQ combines the Generated self-attention map with the VMC and VBD attention maps using learnable weights $\alpha$, $\beta$, $\gamma$ summing to one. A vision abstractor and LLM, fine-tuned with low-rank adaptation, take the branch-specific tokens and prompts to emit five-level quality tokens, and a softmax-weighted sum of level midpoints converts the logits into 0–100 scores.
What would settle it
Ask a panel of board-certified interventional radiologists to independently score a stratified sample of CAS-3K images on the same three metrics, then compute the PLCC between their mean ratings and the student-derived MOS; if that correlation is below roughly 0.7, or if CAS-IQA's advantage over the second-best method disappears when evaluated against the radiologist ratings, the paper's claim of clinically meaningful quality scores is falsified.
Extended reading notes
Core claim
The central discovery the paper tries to establish is that vision-language models can produce fine-grained, reference-aware quality scores for synthetic angiographies that align with human ratings better than existing IQA methods, provided the model is given the generated image together with its non-contrast mask and its real contrast-filled counterpart. The authors claim CAS-3K is the first IQA dataset for medical AIGC, and that CAS-IQA, built on a VLM with the MUST module, achieves the best PLCC and SRCC on all three metrics, with the largest gain in overall quality (4.1% PLCC over the second-best method). The paper attributes this to the MUST module's ability to isolate vessel tokens via deformable subtraction of mask from contrast, apply global self-attention for morphology, local convolution for branch details, and weighted attention-map fusion for overall quality, then feed branch-specific tokens to the LLM through metric-specific prompts.
Load-bearing premise
The load-bearing premise is that the mean opinion scores from just three biomedical-engineering graduate students, who were trained briefly and admitted to annotation only after their pairwise correlations exceeded 0.7, are a valid proxy for what board-certified interventional radiologists would judge, since no radiologist actually rated the images.
Editorial extensions
If this is right
- Reference-aware scoring becomes a practical option for medical AIGC: generated angiograms can be audited against their mask and contrast reference instead of being judged as isolated images.
- The three task-specific metrics give generative-model developers concrete optimization targets: improving vessel morphology, branch detection, and overall realism should translate into higher quality scores.
- The MUST module's fusion-and-routing design is claimed to accelerate convergence, reaching optimal performance in about five epochs versus fifteen without it, which matters in small medical datasets.
- The VBD analysis suggests that branch-level artifacts are visually salient and may be detected with less contextual reference, so branch screening could potentially be handled by simpler single-image models.
Reading between the lines
- An implication the paper leaves implicit is that if the student-based MOS is validated against radiologist ratings, CAS-IQA's scores could serve as an automated acceptance gate for synthetic angiography before clinical use.
- The same mask–contrast–generated triple-input design should transfer to other paired medical synthesis tasks, such as contrast-enhanced CT or MRI, where reference-aware quality scoring is also missing.
- Because the ground truth comes from only three raters, the reported 0.5–4.1% PLCC margins may lie within the noise of subjective labelling; a multi-rater replication with expert adjudication would show whether the ranking is stable.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces CAS-IQA, a vision-language-model-based framework for assessing the quality of synthetic X-ray angiographies, and CAS-3K, a dataset of 3,565 synthetic angiographies generated from 713 Mask-Contrast pairs by five generative models. Three task-specific metrics are defined (VMC, VBD, OQ), and a Multi-path feature fusion and routing (MUST) module is proposed to combine Mask, Generated, and Contrast visual tokens and route them to metric-specific branches. On a single train/test split of CAS-3K, CAS-IQA reports the highest PLCC and SRCC among the listed baselines on all three metrics, and ablation results attribute part of the gain to MUST. The paper claims this is the first IQA dataset for medical AIGC.
Significance. If the results hold, the paper would fill a real gap: there is currently no standard benchmark for quality assessment of synthetic angiography, and the proposed VMC/VBD/OQ metrics are clinically motivated. The use of auxiliary Mask and Contrast images as references is a plausible and useful departure from single-image IQA, and the evaluation covers a broad set of handcrafted, DNN-based, and VLM-based baselines. However, the core empirical claims rest on a single split with no confidence intervals or significance tests, the ground-truth MOS comes from three student raters without expert validation, and the dataset/model are not yet publicly available. These issues currently limit the strength of the claims to conditional rather than established.
major comments (4)
- [§5.2, Table 1] The paper claims PLCC improvements of 0.5%, 2.0%, and 4.1% for VMC, VBD, and OQ, respectively, over the second-best method, MA-AGIQA. This is inconsistent with Table 1: for OQ, the second-best PLCC is LoDa (0.6899), not MA-AGIQA (0.6696), and the relative gain over LoDa is roughly 1.2%, not 4.1%. Moreover, the raw deltas are 0.0033 (VMC), 0.0132 (VBD), and 0.0289 (OQ vs MA-AGIQA), and no confidence intervals, significance tests, or repeated-seed results are provided. With n=715, the sampling uncertainty of a correlation is on the order of 0.04 in Pearson units, so the smaller gaps may be within noise. The central claim of significant superiority needs bootstrap or permutation evidence, and the baseline identification must be corrected.
- [§3.2] The ground-truth MOS are produced by three graduate students in biomedical engineering after a short training session, with board-certified radiologists involved only in defining the metrics. The 0.7 inter-rater correlation threshold measures agreement among the students, not agreement with expert clinical judgment. Since the value of the dataset and model is explicitly stated as clinical relevance, the paper should validate a subset of student ratings against ratings by interventional radiologists, or at least report the students' agreement with the radiologists who helped define the metrics. Without such validation, every model comparison inherits unvalidated labels.
- [§5.1] The train/test split is described only as 2,850 training images and 715 test images, with no statement about whether the split is disjoint at the level of the 713 source Mask-Contrast pairs. Because each source pair generates five synthetic images, an image-level random split can place near-duplicate content in both training and test, inflating measured performance, especially for a method that uses Mask and Contrast as auxiliary inputs. The authors should report a pair-disjoint (or at least generator-stratified) split and, if the current results were obtained with overlapping source pairs, re-run the comparisons under a clean split.
- [§5.3, Table 2] The ablation of MUST is confounded with the prompt/inference format. The w/o MUST condition uses a single prompt asking for all three quality levels at once, while the w/ MUST condition uses three separate metric-specific prompts and branch-specific tokens. The consistent improvements in Table 2 could therefore come from the multi-turn or multi-branch prompting scheme rather than from the fusion and routing operations attributed to MUST. An additional control, such as three separate prompts without the routing/fusion mechanism, is needed to isolate the effect of the MUST module.
minor comments (5)
- [Fig. 2 and §3.1] There are several typos in the text: 'sore estimation' should be 'score estimation' in Fig. 2(b), 'collaborat' should be 'collaborate' in §3.1, and 'imgae' should be 'image' in the prompt shown in §5.3.
- [§2.3] The claim that CAS-3K is 'the first IQA dataset for medical AIGC' should be softened to 'to the best of our knowledge' and supported by a more systematic literature search, since the current statement is difficult to verify from the cited references alone.
- [§3.1] The manual screening of the 713 high-quality Mask-Contrast pairs is not documented. The authors should state the inclusion/exclusion criteria and, ideally, report how many candidate pairs were rejected and why.
- [§5.1] The manuscript does not provide an availability statement for CAS-3K or the trained CAS-IQA model, nor does it report random seeds, the number of runs, or whether the split was stratified by generative model. Such details are needed for reproducibility of a dataset-plus-model paper.
- [§5.3] The statement that the w/o MUST baseline requires about 15 epochs to converge is not supported by learning curves. Either provide the curves or remove the claim.
Circularity Check
No significant circularity: CAS-IQA is evaluated on a held-out test split of the authors' own CAS-3K dataset with standard supervised training, and the sole author-overlapping citation (CAS-GAN) is motivational rather than load-bearing.
full rationale
CAS-IQA is an empirical modeling paper with no derivation chain whose conclusions could reduce to its inputs. The ground-truth MOS is computed from independent subject ratings (Eq. 2, Section 3.3), and the model is trained on a 2,850-image split and evaluated on a disjoint 715-image test split (Section 5.1), so the reported PLCC/SRCC values are genuine held-out predictions rather than fitted quantities renamed as predictions. The only author-overlapping citation is [6] (CAS-GAN, same D.-X. Huang and same institute), used in the Introduction to motivate that AIGC can generate photo-realistic angiographies from non-contrast X-ray images; this is background motivation, is not used as evidence for CAS-IQA's performance, and the CAS-3K dataset is generated with five external models (CycleGAN, AttentionGAN, CUT, ILVR, EGSDE), not with CAS-GAN, so the self-citation is not load-bearing. The five-level score mapping (Section 4.3) is a quantization/dequantization scheme applied to the same MOS scale and is evaluated on held-out data, so it does not force the result by construction. Quantitative concerns that fall outside circularity: the claimed 'significant' PLCC margins (0.003-0.029) lack confidence intervals or significance tests, and the reported 4.1% OQ improvement is computed against MA-AGIQA, which is not the second-best OQ baseline in Table 1 (LoDa is 0.6899); these are statistical-reporting and benchmarking-consistency issues, not circularity. Consistent with the reader's take, the non-circularity burden is low here.
Assumptions & free parameters
free parameters (4)
- Softmax score weights {10,30,50,70,90} =
10, 30, 50, 70, 90
- MUST fusion scalars alpha, beta, gamma =
learned on CAS-3K
- Inter-rater correlation threshold 0.7 =
0.7
- LoRA rank and alpha =
rank 128, alpha 256
assumptions (5)
- domain assumption Deformable convolution can align temporally mismatched Contrast and Mask tokens so that DConv(fC) - fM isolates vessel structure
- domain assumption Three biomedical-engineering graduate students provide valid clinical quality judgments after short training and a 0.7 correlation gate
- domain assumption The 713 manually screened Mask-Contrast pairs and the five chosen generative models represent the clinical deployment space of synthetic angiography
- standard math Cross-attention and token subtraction behave meaningfully in the CLIP visual-token space for X-ray images
- ad hoc to paper Five equal 20-point intervals map continuous quality to the levels bad, poor, fair, good, excellent
invented entities (2)
-
CAS-3K dataset (3,565 synthetic angiographies with VMC, VBD, and OQ MOS annotations)
-
Three task-specific quality metrics (VMC, VBD, OQ)
Cite this review
Pith. "Pith review of CAS-IQA: Teaching Vision-Language Models for Synthetic Angiography Quality Assessment." pith.science (2026). https://pith.science/paper/6NJFCCFR
@misc{pith2026250517619,
author = {Pith},
title = {Pith review of: CAS-IQA: Teaching Vision-Language Models for Synthetic Angiography Quality Assessment},
year = {2026},
howpublished = {\url{https://pith.science/paper/6NJFCCFR}},
note = {Machine review of arXiv:2505.17619}
}
read the original abstract
Synthetic X-ray angiographies generated by modern generative models hold great potential to reduce the use of contrast agents in vascular interventional procedures. However, low-quality synthetic angiographies can significantly increase procedural risk, underscoring the need for reliable image quality assessment (IQA) methods. Existing IQA models, however, fail to leverage auxiliary images as references during evaluation and lack fine-grained, task-specific metrics necessary for clinical relevance. To address these limitations, this paper proposes CAS-IQA, a vision-language model (VLM)-based framework that predicts fine-grained quality scores by effectively incorporating auxiliary information from related images. In the absence of angiography datasets, CAS-3K is constructed, comprising 3,565 synthetic angiographies along with score annotations. To ensure clinically meaningful assessment, three task-specific evaluation metrics are defined. Furthermore, a Multi-path featUre fuSion and rouTing (MUST) module is designed to enhance image representations by adaptively fusing and routing visual tokens to metric-specific branches. Extensive experiments on the CAS-3K dataset demonstrate that CAS-IQA significantly outperforms state-of-the-art IQA methods by a considerable margin.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
Lancet Discovery Science1, 51–61 (2018)
Clement, O., et al.: Immediate hypersensitivity to contrast agents: The French 5-year CIRTACI study. Lancet Discovery Science1, 51–61 (2018)
work page 2018
-
[2]
Nature Reviews Nephrology13(3), 169–180 (2017)
Fähling, M., Seeliger, E., Patzak, A., Persson, P.B.: Understanding and preventing contrast-induced acute kidney injury. Nature Reviews Nephrology13(3), 169–180 (2017)
work page 2017
- [3]
-
[4]
Hosu, V., Lin, H., Sziranyi, T., Saupe, D.: KonIQ-10K: An ecologically valid database for deep learning of blind image quality assessment. IEEE TIP29, 4041– 4056 (2020)
work page 2020
- [5]
- [6]
-
[7]
Expert Systems41(12), e13708 (2024)
Jiménez-Partinen, A., et al.: CADICA: A new dataset for coronary artery disease detection by using invasive coronary angiography. Expert Systems41(12), e13708 (2024)
work page 2024
- [8]
Show all 35 references
-
[9]
IEEE TCSVT34(8), 6833–6846 (2023)
Li, C., et al.: AGIQA-3K: An open database for AI-generated image quality as- sessment. IEEE TCSVT34(8), 6833–6846 (2023)
2023
-
[10]
In: Proc
Li, C., et al.: AIGIQA-20K: A large database for ai-generated image quality as- sessment. In: Proc. CVPR. pp. 6327–6336 (2024)
2024
-
[11]
In: Proc
Liu, H., Li, C., Wu, Q., Lee, Y.J.: Visual instruction tuning. In: Proc. NeurIPS. vol. 36, pp. 34892–34916 (2023)
2023
-
[12]
Liu, Y., et al.: MMBench: Is your multi-modal model an all-around player? In: Proc. ECCV. pp. 216–233 (2024)
2024
-
[13]
In: Proc
Lu, P., et al.: FINECAPTION: Compositional image captioning focusing on wher- ever you want at any granularity. In: Proc. CVPR (2025) CAS-IQA 15
2025
-
[14]
In: Proc
Ma, Y., et al.: Self-supervised vessel segmentation via adversarial learning. In: Proc. ICCV. pp. 7536–7545 (2021)
2021
-
[15]
IEEE TIP21(12), 4695–4708 (2012)
Mittal, A., Moorthy, A.K., Bovik, A.C.: No-reference image quality assessment in the spatial domain. IEEE TIP21(12), 4695–4708 (2012)
2012
-
[16]
completely blind
Mittal, A., Soundararajan, R., Bovik, A.C.: Making a “completely blind” image quality analyzer. IEEE SPL20(3), 209–212 (2012)
2012
-
[17]
In: Proc
Park, T., Efros, A.A., Zhang, R., Zhu, J.Y.: Contrastive learning for unpaired image-to-image translation. In: Proc. ECCV. pp. 319–345 (2020)
2020
-
[18]
In: Proc
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proc. CVPR. pp. 10684–10695 (2022)
2022
-
[19]
Recommendation ITU-R BT500(13) (2012)
Series, B.: Methodology for the subjective assessment of the quality of television pictures. Recommendation ITU-R BT500(13) (2012)
2012
-
[20]
In: Proc
Shin, N.H., Lee, S.H., Kim, C.S.: Blind image quality assessment based on geo- metric order learning. In: Proc. CVPR. pp. 12799–12808 (2024)
2024
-
[21]
In: Proc
Su, S., et al.: Blindly assess image quality in the wild guided by a self-adaptive hyper network. In: Proc. CVPR. pp. 3667–3676 (2020)
2020
-
[22]
IEEE TNNLS 34(4), 1972–1987 (2021)
Tang, H., Liu, H., Xu, D., Torr, P.H., Sebe, N.: AttentionGAN: Unpaired image- to-image translation using attention-guided generative adversarial networks. IEEE TNNLS 34(4), 1972–1987 (2021)
2021
-
[23]
Vaswani, A., et al.: Attention is all you need. Proc. NeurIPS (2017)
2017
-
[24]
In: Proc
Wang, P., et al.: Large multi-modality model assisted AI-generated image quality assessment. In: Proc. MM. pp. 7803–7812 (2024)
2024
-
[25]
In: Proc
Wu, H., et al.: Q-Align: Teaching LMMs for visual scoring via discrete text-defined levels. In: Proc. ICML. pp. 54015–54029 (2024)
2024
-
[26]
In: Proc
Wu, H., et al.: Q-Instruct: Improving low-level visual abilities for multi-modality foundation models. In: Proc. CVPR. pp. 25490–25500 (2024)
2024
-
[27]
In: Proc
Xu, K., et al.: Boosting image quality assessment through efficient transformer adaptation with local feature enhancement. In: Proc. CVPR. pp. 2662–2672 (2024)
2024
-
[28]
In: Proc
Yang, S., et al.: MANIQA: Multi-dimension attention network for no-reference image quality assessment. In: Proc. CVPR. pp. 1191–1200 (2022)
2022
-
[29]
In: Proc
Ye, Q., et al.: mPLUG-Owl2: Revolutionizing multi-modal large language model with modality collaboration. In: Proc. CVPR. pp. 13040–13051 (2024)
2024
-
[30]
In: Proc
You, Z., Cai, X., Gu, J., Xue, T., Dong, C.: Teaching large language models to regress accurate image quality scores using score distribution. In: Proc. CVPR (2025)
2025
-
[31]
IEEE TCSVT30(1), 36–47 (2020)
Zhang, W., Ma, K., Yan, J., Deng, D., Wang, Z.: Blind image quality assessment using a deep bilinear convolutional neural network. IEEE TCSVT30(1), 36–47 (2020)
2020
-
[32]
Med (2024), DOI: 10.1016/j.medj.2024.07.025
Zhao, H., et al.: Large-scale pretrained frame generative model enables real-time low-dose DSA imaging: An AI system development and multi-center validation study. Med (2024), DOI: 10.1016/j.medj.2024.07.025
2024 doi
-
[33]
In: Proc
Zhao, M., Bao, F., Li, C., Zhu, J.: ILVR: Conditioning method for denoising dif- fusion probabilistic models. In: Proc. ICCV. pp. 14367–14376 (2021)
2021
-
[34]
In: Proc
Zhao, M., Bao, F., Li, C., Zhu, J.: EGSDE: Unpaired image-to-image translation via energy-guided stochastic differential equations. In: Proc. NeurIPS. vol. 35, pp. 3609–3623 (2022)
2022
-
[35]
In: Proc
Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proc. ICCV. pp. 2223–2232 (2017)
2017
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.