Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

Vision-language models fail systematically on rare anatomical variants, dropping from about 74% to 29% accuracy, because learned priors about typical anatomy override clear visual evidence.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 18:37 UTC pith:YAE3NZK2

load-bearing objection Real benchmark, real effect, but missing human baseline and inconsistent headline numbers keep the natural-adversarial claim from being fully closed. the 3 major comments →

arxiv 2512.04238 v2 pith:YAE3NZK2 submitted 2025-12-03 cs.CV

6 Fingers, 1 Kidney: Natural Adversarial Medical Images Reveal Critical Weaknesses of Vision-Language Models

Leon Mayer , Piotr Kalinowski , Caroline Ebersbach , Marcel Knopp , Tim R\"adsch , Evangelia Christodoulou , Annika Reinke , Fiona R. Kolbinger
show 1 more author
Lena Maier-Hein
This is my paper
classification cs.CV
keywords vision-language modelsanatomical biasrare anatomical variantsmedical imaging benchmarknatural adversarial examplesbias ratemultimodal medical AIbenchmark evaluation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that current vision-language models (VLMs) cannot be trusted to recognize rare anatomical variants in medical images, even when the images are visually clear to human clinicians. To test this, the authors built AdversarialAnatomyBench, a benchmark of 200 paired typical/atypical images spanning seven imaging modalities, and evaluated 22 state-of-the-art models. Across all models, mean accuracy fell from roughly 74% on typical anatomy to 29% on atypical anatomy, while 65–95% of errors matched the answer expected for typical anatomy. The authors further show that neither scaling up model parameters, prompting models about rare conditions, nor increasing test-time reasoning effort closes the gap. If correct, the finding implies that current VLMs rely on anatomical priors from internet-scale training data rather than on the visual evidence in the image.

Core claim

The core discovery is that vision-language models have a strong, systematic anatomical bias: when shown naturally occurring rare anatomical variants, they default to answers consistent with typical anatomy despite clear visual evidence. This is quantified by AdversarialAnatomyBench, whose 200 paired image–question items show a mean accuracy drop from 74% on typical to 29% on atypical anatomy across 22 models, with per-model bias rates of 41–76%. The authors also find that model scaling (from 2B to 235B parameters), bias-aware prompting, and test-time reasoning budgets (low to high) produce no meaningful improvement on atypical cases. They interpret the persistent gap as evidence that learned

What carries the argument

The central object is AdversarialAnatomyBench: 200 medical image–question pairs, organized as 100 atypical variants each matched with a typical reference image across seven modalities (X-ray, CT, MRI, MRA, ultrasound, fluoroscopy, photography). The benchmark's paired design ensures that a model cannot score well by relying on language priors alone, because typical and atypical versions of the same question have different correct answers. The key metric is the bias rate, defined as the fraction of answers on atypical images that match the expected typical-anatomy prior; this directly measures the extent to which errors stem from learned priors rather than random failures.

Load-bearing premise

The benchmark's core assumption is that the atypical images are visually unambiguous to clinicians and that any human reader would give the ground-truth answer; if humans also disagreed on a substantial fraction of items, the accuracy gap would overstate the role of anatomical bias.

What would settle it

Measure clinician or expert-human accuracy on the same 200 image–question pairs. If human accuracy on atypical images is well below ceiling or inter-rater agreement is low, the premise of visual unambiguousness fails and the performance gap cannot be attributed solely to anatomical priors. Alternatively, if a model fine-tuned on rare-anatomy examples still shows a large typical/atypical gap, the claim that priors override visual evidence would need revision.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the central claim holds, current VLMs cannot be safely deployed for basic visual perception tasks in clinical workflows without explicit evaluation on rare anatomical presentations.
  • Medical AI evaluation standards should incorporate rare-anatomy cases as a mandatory robustness check, not just common presentations.
  • Model scaling alone will not fix anatomical bias; the paper's scaling experiment shows flat atypical accuracy from 2B to 235B parameters.
  • Bias-aware prompting and test-time reasoning are insufficient mitigations, so future work must target pretraining data or architecture.
  • AdversarialAnatomyBench provides a reusable stress test for measuring anatomical bias in future multimodal medical models.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because the benchmark is drawn from publicly available images, some items may have appeared in model training; if so, the measured gap could actually understate the bias on truly unseen rare cases.
  • The paired typical/atypical design could be extended to other under-represented visual categories beyond anatomy, such as rare congenital conditions or non-medical long-tail objects.
  • The failure pattern suggests that VLMs may also exhibit similar prior-driven errors on demographic or cultural minorities whose presentations are rare in training data.
  • A direct next experiment implied by the paper is to test whether fine-tuning on a small set of atypical examples reduces the bias; the paper does not perform this intervention.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces AdversarialAnatomyBench, a 200-image benchmark of rare anatomical variants paired with typical references across seven imaging modalities, and evaluates 22 vision-language models on basic visual perception questions. The central claim is that VLMs exhibit a strong anatomical prior bias: accuracy drops from ~74% on typical to ~29% on atypical anatomy, model errors align with expected typical-anatomy answers 65–95% of the time, and neither parameter scaling, bias-aware prompting, nor increased test-time reasoning closes the gap. The authors argue these are 'natural adversarial' examples in the Hendrycks et al. sense: visually unambiguous to clinicians yet systematically misclassified by models.

Significance. If the main result holds, this is an important and timely quantification of a real clinical AI risk: current VLMs fail systematically on rare anatomical variants, and the failure mode is robust across model families and interventions. The benchmark design is thoughtful: paired typical/atypical questions, standardized prompts, stratified bootstrap CIs, and a broad model set including medical-specific models. The negative results on scaling, prompting, and reasoning are valuable for the community. The paper is clearly written and the central finding will likely influence medical VLM evaluation. However, the 'natural adversarial' framing and the mechanistic claim of prior-driven bias require a human baseline to separate model bias from image-level difficulty; this is currently the main weakness.

major comments (3)
  1. [Introduction; Methods (Verifiability); Discussion] The paper's central claim is that VLMs fail 'despite clear visual evidence' and that the benchmark consists of natural adversarial examples that are easy for humans (Hendrycks et al. [12]). This premise is load-bearing, but no human or clinician accuracy baseline is reported for the 200 items. The paired typical/atypical design (Fig. 2) controls for question type but not for image-level difficulty: the two sets use different images, often from different modalities and cases. The 74%→29% gap and 65–95% bias rates could therefore partly reflect lower image quality, cropping, annotation ambiguity, or harder counting tasks in the atypical set, rather than learned anatomical priors. I request a human/clinician baseline on the same item set (or a representative subset) and agreement on ground-truth labels. If this is not feasible, the 'clear visual evidence' / 'natural adversarial' claims shou
  2. [Methods (Bootstrap); Table 1] The stratified bootstrap resamples images within question type, but some question types contain very few images; the Methods give the example of only 4 atypical images for one question. With such small pools, the point estimates for individual question types are unstable and the reported 95% CIs (Table 1) likely understate uncertainty. Moreover, the benchmark data and code are not yet released ('will be available soon'), so readers cannot audit individual items or reproduce the bootstrap. As a benchmark paper, this limits the immediate utility of AdversarialAnatomyBench. Please release the data/code, report per-question sample sizes, and consider aggregating only question types with sufficient image counts for the headline accuracy numbers.
  3. [Results (Errors predominantly match expected anatomical bias); Methods (Bias rate)] The bias rate is defined as the percentage of model answers matching the expected typical-anatomy answer on atypical images. On ambiguous or poorly annotated images, a model may produce the typical answer for reasons other than prior-driven bias, e.g., generic counting errors that happen to land on the typical count. The claim that errors 'closely mirrored expected anatomical biases' would be strengthened by a control: report the bias rate on typical images (showing that errors are not symmetrically aligned with the atypical answer) or condition the bias rate on items that pass a human-verifiability check. Without such a control, the bias rate may overstate the role of anatomical priors.
minor comments (4)
  1. [Abstract; Table 1] The abstract states that 25 models were benchmarked, but the full text and Table 1 list 22. Please harmonize the count.
  2. [Abstract; Results] The first abstract in the submission says mean accuracy dropped from 71% to 28%, while the full-text abstract says 74% to 29%; Fig. 3 reports 73% to 34% for the 10-model subset. Clarify which subset each number refers to and ensure consistency.
  3. [Discussion] The Discussion says 'accuracy drops by 16-79%' while the Results say 'performance gaps ranged up to 69.0 percentage points.' Specify whether these are absolute or relative drops and reconcile the values.
  4. [Limitations; Data availability] The Limitations section acknowledges lack of clinically relevant task evaluation and single-image assessment, but does not mention the missing human baseline for the 'visually unambiguous' premise. Please address this. Also, the data availability statement ('will be available soon') should include a timeline or a repository link if possible.

Circularity Check

0 steps flagged

No circularity: this is an empirical benchmark study with no fitted derivation, no self-citation chain, and no prediction that reduces to its own inputs by construction.

full rationale

The paper contains no derivation chain in the sense of fitted equations or first-principles predictions. Its results are measured model outputs on a constructed benchmark; accuracy, bias rate, and intervention comparisons are empirical statistics, not quantities derived from the benchmark's construction. The paired atypical/typical design intentionally ensures that atypical images have answers different from the typical answer ('Questions are formulated in a way that ensures images of atypical cases produce answers different from those observed in the majority of the population'), but this is a standard adversarial-benchmark convention that makes the test sensitive to prior bias; it does not force the observed accuracies, the 65-95% ratio of errors matching typical answers, or the failure of scaling/prompting/reasoning interventions. The bias-rate metric is defined as the percentage of model answers matching the expected anatomical prior on atypical images, and for some binary questions the non-correct answer is indeed the typical answer, so high alignment is partly structured by item design; however, the paper's central claims still rest on measured model behavior rather than on a tautology. There are no load-bearing self-citations: reference [12] (Hendrycks et al.) is used only as the inspiration for the term 'natural adversarial examples', and the other cited work is external. The absence of a human/clinician baseline for the 'visually unambiguous' premise is a validity and interpretability limitation, correctly located as a design assumption, but it is not circular reasoning under the criteria here.

Axiom & Free-Parameter Ledger

0 free parameters · 4 axioms · 0 invented entities

The benchmark is an empirical instrument; no fitted parameters or ad hoc entities are introduced. The concept 'natural adversarial anatomy' is a label, not a postulated mechanism. The central assumptions concern annotation correctness, image ambiguity, output parsing, and training-data contamination.

axioms (4)
  • domain assumption The 200 atypical images are visually unambiguous to clinicians, so failures cannot be attributed to image ambiguity.
    Asserted in the Introduction ('visually unambiguous to clinicians') and in the Methods 'Verifiability' design decision. No human accuracy baseline is reported.
  • domain assumption Clinician-authored ground-truth answers and typical/atypical labels are correct.
    Methods state questions were developed by a team of clinicians and data scientists, but no independent verification or inter-rater agreement is reported.
  • domain assumption Model answers can be unambiguously parsed from the standardized output format.
    Methods instruct models to output answers in a standardized format for comparison, but no parsing or rejection procedure is described.
  • domain assumption Publicly sourced images were not memorized during pretraining in a way that invalidates generalization measurement.
    The Limitations section acknowledges images may have appeared in model training data, which could inflate or deflate measured gaps.

pith-pipeline@v1.3.0-alltime-deepseek · 14607 in / 11993 out tokens · 109466 ms · 2026-08-03T18:37:12.779334+00:00 · methodology

0 comments
read the original abstract

Vision-language models (VLMs) are increasingly integrated into clinical workflows. However, existing benchmarks primarily assess performance on common anatomical presentations and fail to capture the challenges posed by rare variants. To address this gap, we introduce AdversarialAnatomyBench, the first benchmark comprising naturally occurring rare anatomical variants across diverse imaging modalities and anatomical regions. We call such variants that violate learned priors about "typical" human anatomy natural adversarial anatomy. Benchmarking 25 state-of-the-art VLMs with AdversarialAnatomyBench yielded three key insights. First, when queried with basic medical perception tasks, mean accuracy dropped from 71% on typical to 28% on atypical anatomy. Even the best-performing models, GPT-5, Gemini 2.5 Pro, and Llama 4 Maverick, showed performance drops of 41-51%. Second, model errors closely mirrored expected anatomical biases. Third, neither model scaling nor interventions, including bias-aware prompting and test-time reasoning, resolved these issues. These findings highlight a critical limitation in current VLMs: their poor generalization to rare anatomical presentations. AdversarialAnatomyBench provides a foundation for systematically measuring and mitigating anatomical bias in multimodal medical artificial intelligence (AI) systems.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Attention Without Grounding: Causal Evaluation of Visual Explanations in Medical VLMs

    cs.CV 2026-07 accept novelty 6.0

    Medical VLM attention and saliency heatmaps are not causally faithful: they miss radiologist-annotated regions and anti-correlate with patch-occlusion importance, unlike CXR classifier baselines.

Reference graph

Works this paper leans on

46 extracted references · 18 canonical work pages · cited by 1 Pith paper

  1. [1]

    Bubshait, D. K. A review of polydactyly and its inheritance: Connecting the dots. Medicine101, e32060 (2022)

    2022

  2. [2]

    & Mirjalili, S

    Taghavi, K., Kirkpatrick, J. & Mirjalili, S. The horseshoe kidney: surgical anatomy and embryology.Journal of pediatric urology12, 275–280 (2016)

    2016

  3. [3]

    & Telkes, G

    Eitler, K., Bibok, A. & Telkes, G. Situs inversus totalis: a clinical review. International journal of general medicine2437–2449 (2022)

    2022

  4. [4]

    & Sekuboyina, A

    Royer, C., Menze, B. & Sekuboyina, A. Multimedeval: A benchmark and a toolkit for evaluating medical vision-language models.arXiv preprint arXiv:2402.09262 (2024)

    arXiv 2024

  5. [5]

    Zhou, T.et al.Drvd-bench: Do vision-language models reason like human doctors in medical image diagnosis?arXiv preprint arXiv:2505.24173(2025)

    Pith/arXiv arXiv 2025

  6. [6]

    Vo, A.et al.Vision language models are biased.arXiv preprint arXiv:2505.23941 (2025)

    Pith/arXiv arXiv 2025

  7. [7]

    A., Rajpurkar, P., Rodman, A

    Buckley, T., Diao, J. A., Rajpurkar, P., Rodman, A. & Manrai, A. K. Multimodal foundation models exploit text to make medical image predictions.arXiv e-prints arXiv–2311 (2023)

    2023

  8. [8]

    Gu, Y.et al.The illusion of readiness: Stress testing large frontier models on multimodal medical benchmarks.arXiv preprint arXiv:2509.18234(2025)

    arXiv 2025

  9. [9]

    & Behzadan, V

    Sadanandan, B. & Behzadan, V. Vsf-med: A vulnerability scoring framework for medical vision-language models.arXiv preprint arXiv:2507.00052(2025). 21

    Pith/arXiv arXiv 2025

  10. [10]

    Yang, Y.et al.Demographic bias of expert-level vision-language foundation models in medical imaging.Science Advances11, eadq0305 (2025)

    2025

  11. [11]

    D., Jiang, Y

    Xu, S., Janizek, J. D., Jiang, Y. & Daneshjou, R.Biasicl: In-context learning and demographic biases of vision language models, 88–97 (Springer, 2025)

    2025

  12. [12]

    & Song, D.Natural adversarial examples, 15262–15271 (2021)

    Hendrycks, D., Zhao, K., Basart, S., Steinhardt, J. & Song, D.Natural adversarial examples, 15262–15271 (2021)

    2021

  13. [13]

    Guo, D.et al.Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948(2025)

    Pith/arXiv arXiv 2025

  14. [14]

    Li, B.et al.Naturalbench: Evaluating vision-language models on natural adversarial samples.Advances in Neural Information Processing Systems37, 17044–17068 (2024)

    2024

  15. [15]

    Shrestha, P., Amgain, S., Khanal, B., Linte, C. A. & Bhattarai, B. Medical vision language pretraining: A survey.arXiv preprint arXiv:2312.06224(2023)

    Pith/arXiv arXiv 2023

  16. [16]

    & Qin, J

    Lin, H., Xu, C. & Qin, J. Taming vision-language models for medical image analysis: A comprehensive review.arXiv preprint arXiv:2506.18378(2025)

    Pith/arXiv arXiv 2025

  17. [17]

    & Hernandez-Boussard, T

    Handler, R., Sharma, S. & Hernandez-Boussard, T. The fragile intelligence of gpt-5 in medicine.Nature Medicine1–3 (2025)

    2025

  18. [18]

    Chen, K.et al.Think twice to see more: Iterative visual reasoning in medical vlms.arXiv preprint arXiv:2510.10052(2025)

    arXiv 2025

  19. [19]

    Medvlm-r1: Incentivizing medical reasoning capability of vision- language models (vlms) via reinforcement learning, 337–347 (Springer, 2025)

    Pan, J.et al. Medvlm-r1: Incentivizing medical reasoning capability of vision- language models (vlms) via reinforcement learning, 337–347 (Springer, 2025). 22

    2025

  20. [20]

    & Parikh, D.Making the v in vqa matter: Elevating the role of image understanding in visual question answering, 6904–6913 (2017)

    Goyal, Y., Khot, T., Summers-Stay, D., Batra, D. & Parikh, D.Making the v in vqa matter: Elevating the role of image understanding in visual question answering, 6904–6913 (2017)

    2017

  21. [21]

    [Accessed 27-11-2025]

    https://radiopaedia.org/. [Accessed 27-11-2025]

    2025

  22. [22]

    Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison, Vol

    Irvin, J.et al. Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison, Vol. 33, 590–597 (2019)

    2019

  23. [23]

    Rajpurkar, P.et al.Mura: Large dataset for abnormality detection in muscu- loskeletal radiographs.arXiv preprint arXiv:1712.06957(2017)

    Pith/arXiv arXiv 2017

  24. [24]

    & Jdey, I

    Brahmi, W. & Jdey, I. Automatic tooth instance segmentation and identification from panoramic x-ray images using deep cnn.Multimedia Tools and Applications 83, 55565–55585 (2024)

    2024

  25. [25]

    & Drira, F

    Brahmi, W., Jdey, I. & Drira, F. Exploring the role of convolutional neural networks (cnn) in dental radiography segmentation: A comprehensive systematic literature review.Engineering Applications of Artificial Intelligence133, 108510 (2024)

    2024

  26. [26]

    https://ai.google.dev/gemini-api/docs/openai

    Gemini API documentation. https://ai.google.dev/gemini-api/docs/openai. Accessed: 2025-11-02

    2025

  27. [27]

    Accessory premolar teeth.Case study, Radiopaedia.org(2013)

    Jones, J. Accessory premolar teeth.Case study, Radiopaedia.org(2013). URL http://doi.org/10.53347/rID-22225

    work page doi:10.53347/rid-22225 2013

  28. [28]

    Normal orthopantomography (opg).Case study, Radiopaedia.org (2016)

    Uribe, S. Normal orthopantomography (opg).Case study, Radiopaedia.org (2016). URL http://doi.org/10.53347/rID-46431

    work page doi:10.53347/rid-46431 2016

  29. [29]

    Situs inversus with pneumothorax.Case study, Radiopaedia.org(2010)

    Singh, G. Situs inversus with pneumothorax.Case study, Radiopaedia.org(2010). URL http://doi.org/10.53347/rID-8495. 23

    work page doi:10.53347/rid-8495 2010

  30. [30]

    Normal chest radiograph - female.Case study, Radiopaedia.org(2015)

    Bickle, I. Normal chest radiograph - female.Case study, Radiopaedia.org(2015). URL http://doi.org/10.53347/rID-33225

    work page doi:10.53347/rid-33225 2015

  31. [31]

    Aberrant right subclavian artery - mr angiography.Case study, Radiopaedia.org(2011)

    Schubert, R. Aberrant right subclavian artery - mr angiography.Case study, Radiopaedia.org(2011). URL http://doi.org/10.53347/rID-13708

    work page doi:10.53347/rid-13708 2011

  32. [32]

    Normal mra of the aortic arch.Case study, Radiopaedia.org(2015)

    Gaillard, F. Normal mra of the aortic arch.Case study, Radiopaedia.org(2015). URL http://doi.org/10.53347/rID-35881

    work page doi:10.53347/rid-35881 2015

  33. [33]

    Beaver tail liver (2015)

    Hacking, C. Beaver tail liver (2015). URL http://doi.org/10.53347/rID-39317

    work page doi:10.53347/rid-39317 2015

  34. [34]

    Amiodarone hepatotoxicity.Case study, Radiopaedia.org(2012)

    Di Muzio, B. Amiodarone hepatotoxicity.Case study, Radiopaedia.org(2012). URL http://doi.org/10.53347/rID-19223

    work page doi:10.53347/rid-19223 2012

  35. [35]

    Santizo Castillo, J. C. Supernumerary kidney.Case study, Radiopaedia.org (2022). URL http://doi.org/10.53347/rID-97733

    work page doi:10.53347/rid-97733 2022

  36. [36]

    Normal ct abdomen (2015)

    Dixon, A. Normal ct abdomen (2015). URL http://doi.org/10.53347/rID-36677

    work page doi:10.53347/rid-36677 2015

  37. [37]

    Qureshi, P. A. A. A. Macrodystrophia lipomatosa.Case study, Radiopaedia.org (2018). URL http://doi.org/10.53347/rID-60654

    work page doi:10.53347/rid-60654 2018

  38. [38]

    Macrodystrophia lipomatosa (2009)

    Al Salam, H. Macrodystrophia lipomatosa (2009). URL http://doi.org/10.53347/ rID-7609

    2009

  39. [39]

    Incomplete double ureter.Case study, Radiopaedia.org(2014)

    Pai, V. Incomplete double ureter.Case study, Radiopaedia.org(2014). URL http://doi.org/10.53347/rID-26900

    work page doi:10.53347/rid-26900 2014

  40. [40]

    Niknejad, M. T. Normal intravenous urography.Case study, Radiopaedia.org (2020). URL http://doi.org/10.53347/rID-85286

    work page doi:10.53347/rid-85286 2020

  41. [41]

    Dichorionic tri-amniotic (dcta) triplets.Case study, Radiopaedia.org (2015)

    Knipe, H. Dichorionic tri-amniotic (dcta) triplets.Case study, Radiopaedia.org (2015). URL http://doi.org/10.53347/rID-36543. 24

    work page doi:10.53347/rid-36543 2015

  42. [42]

    Normal first trimester pregnancy (2023)

    Lukies, M. Normal first trimester pregnancy (2023). URL http://doi.org/10. 53347/rID-76936

    2023

  43. [43]

    Tibial hemimelia.Case study, Radiopaedia.org(2021)

    Khurfan, B. Tibial hemimelia.Case study, Radiopaedia.org(2021). URL http: //doi.org/10.53347/rID-86822

    work page doi:10.53347/rid-86822 2021

  44. [44]

    Normal tibia fibula x-rays - 10-year-old.Case study, Radiopaedia.org (2023)

    Liew, G. Normal tibia fibula x-rays - 10-year-old.Case study, Radiopaedia.org (2023). URL http://doi.org/10.53347/rID-163734

    work page doi:10.53347/rid-163734 2023

  45. [45]

    Elthokapy, M. M. Accessory navicular bones bilaterally.Case study, Radiopae- dia.org(2021). URL https://doi.org/10.53347/rID-93839

    work page doi:10.53347/rid-93839 2021

  46. [46]

    Model Size

    Ranchod, A. I. Splay feet.Case study, Radiopaedia.org(2023). URL https: //doi.org/10.53347/rID-170048. 25 Supplementary material Organization Name Medical Access Model Size Alibaba Qwen3-VL 235B No Open 235B (A22B) Qwen3-VL 30B No Open 30B (A3B) Qwen3-VL 8B No Open 8B Qwen3-VL 4B No Open 4B Qwen3-VL 2B No Open 2B Anthropic Claude 4.5 Sonnet (R) No Closed ...

    work page doi:10.53347/rid-170048 2023