Pith. sign in

REVIEW 5 major objections 6 minor 55 references

Open-Linguistic Concept Unified Learning for Cross-Site Interpretable Dermatology Image Diagnosis

T0 review · 5 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read UniCon claims that dermoscopic and clinical skin-image concept vocabularies can be unified into one shared codebook, enabling clinician corrections that transfer across imaging modalities and sites without dataset-specific retraining.

desk verdict Competent integration of known techniques, but the headline cross-site/cross-modal claim is not actually tested: the only held-out site is dermoscopic. read the letter →

arxiv 2608.03225 v1 pith:O7GU3267 submitted 2026-08-04 cs.CV

Chengyu Wu , Junpeng Tan , Wanxiang Luo , Yaqi Wang , Yandong Wen , Yefeng Zheng This is my paper
classification cs.CV
keywords conceptbottleneckmodelsinterpretablediagnosisdermatologyvision-languageunifiedcodebookcross-sitegeneralizationtest-timeinterventiondermoscopicandclinicalimages
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Concept-based diagnosis models let clinicians see and correct the reasoning behind an AI skin-cancer readout, but their concept lists are usually tied to one dataset: dermoscopic images use microscopic features like pigment network, while clinical photos use surface morphology like plaque. UniCon is an attempt to remove that constraint. It builds a unified concept-prototype codebook by averaging language-model-generated definitions, synonyms, edge cases, and counter-examples across datasets, then routes both image types through this shared space. The paper claims this yields diagnostic accuracy competitive with the best concept models while also making test-time interventions transferable across imaging modalities and sites. The practical stakes: a clinician could correct a diagnosis using a concept visible in one imaging modality and have the correction apply consistently to the other.

What carries the argument

The Unified Concept Prototype Codebook (UCPC): for each concept $c$ and state $v$, the text embeddings of the four generated specification slots are averaged across all datasets that annotate $(c,v)$, $e^x_{c,v} = \frac{1}{|\mathcal{S}_{c,v}|}\sum_{d\in\mathcal{S}_{c,v}} \text{Norm}(P_T(F_T(\tilde{T}^{x}_{c,v,d})))$, marginalizing dataset-specific phrasing. Learnable concept anchors $\{a_c\}$ probe the visual encoder by cross-attention to produce queries $\hat{q}_c$, and the reliability-gated bottleneck $z'' = z \odot \text{expand}(r,\{V_c\})$ filters concept scores before the frozen diagnosis head. The codebook does the unification; the gate does the plasticity; the piecewise one-hot overri

What would settle it

Take a held-out set of dermoscopic and clinical concept names with no shared words and measure whether their codebook prototypes are nearest neighbors to linguistically related concepts (e.g., 'pigment network' near 'reticular mesh') rather than random; then, on a held-out clinical dataset and a held-out dermoscopic dataset, override a clinical concept state on dermoscopic images and a dermoscopic concept state on clinical images and check that the diagnosis changes in the medically expected direction. If either fails, the unified-space and cross-modal intervention claims are falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that disparate, site-specific concept taxonomies can be coordinated without dataset-specific adaptation by a unified concept prototype codebook. For each concept state, the codebook stores the averaged text embedding of four open-linguistic specifications — definition, synonym, edge case, and counter-example — generated by a vision-language model for each cohort. Learnable concept anchors extract visual queries from an image; the queries are scored against the codebook through positive evidence aggregation with a negative-concept penalty; and an image-conditioned reliability gate filters which concepts apply before the diagnosis head. At inference, clinicians int

Load-bearing premise

The load-bearing premise is that the frozen text encoder places dermoscopic concepts like 'pigment network' and clinical concepts like 'plaque' into a shared semantic space where averaging their language-model-generated descriptions yields comparable prototypes; if that alignment fails, the codebook is just two disjoint concept sets and the cross-modal intervention claim collapses, and the paper's external test on an unseen dermoscopic dataset does not exercise the cross-moda

Editorial extensions

If this is right

  • If the codebook is truly shared, a single trained system can serve dermoscopic and clinical inputs, and adding a new site's concepts requires only generating text specifications and averaging them into existing prototypes, not retraining.
  • Clinician corrections can cross modalities: a concept judged on a clinical photograph (e.g. scale) can be overridden on a dermoscopic image, and vice versa (e.g. arborizing vessels), because the bottleneck contains both concept sets.
  • Correct concept states monotonically improve accuracy and incorrect ones monotonically degrade it, implying the final decision is mediated by the concept bottleneck rather than shortcut features.
  • Cross-modal retrieval improves over the base vision-language encoder, indicating the multi-faceted text slots tighten image-concept alignment beyond single fixed templates.
  • The reliability gate's learned applicability scores provide an interpretable, image-conditioned signal for which concepts are meaningful for a given modality and sample.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: if the shared text-embedding space genuinely aligns dermoscopic and clinical semantics, the same codebook recipe should transfer to other specialties where modalities use disjoint vocabularies (e.g., radiology CT vs. X-ray reports), and this transfer would be a strong external test the paper has not run.
  • Editorial inference: the reliability gate could double as an out-of-distribution detector — inputs whose gated scores are all low should be treated as unseen acquisition types — a use the paper does not explore.
  • Editorial inference: because prototypes are averaged text embeddings, one could add a new concept post hoc by generating its four specifications and appending a row to the codebook; whether the frozen head can exploit it without retraining is an open, testable question.
  • Editorial inference: the paper's only unseen-site evaluation is dermoscopic-to-dermoscopic; the stronger claim of cross-modal transfer would need an unseen clinical dataset or a clinician study intervening across modalities to be fully tested.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes UniCon, a concept-bottleneck framework for dermatology that aims to unify heterogeneous concept taxonomies across dermoscopic and clinical images. It constructs a Unified Concept Prototype Codebook (UCPC) by averaging text embeddings of LVLM-generated multi-faceted semantic specifications (Eq. 4), aligns images to these specifications through a multi-margin contrastive objective (Eqs. 1–3), and uses Learnable Evidence Aggregation plus Image-Conditioned Reliability Gating to produce a gated concept bottleneck for diagnosis and test-time intervention. Experiments are reported on Derm7pt, SkinCon, and PH2, with baselines, ablations, retrieval, visualization, and intervention analyses. The central claim is that UniCon 'seamlessly coordinates heterogeneous concept systems across modalities' and enables 'cross-site intervention' without dataset-specific retraining.

Significance. If the cross-site and cross-modal unification claims were supported, the paper would make a useful contribution to interpretable medical image diagnosis: it combines concept bottlenecks with modern vision-language models, releases code, compares against twelve baselines, and includes ablations and a zero-shot external test. The framework is well structured and the three-step training scheme is clearly described. However, the headline contribution is currently under-tested: the only unseen cohort is PH2, which is dermoscopic only, and the cross-modal intervention capability is demonstrated only qualitatively. The load-bearing assumption that averaging text embeddings in Eq. (4) yields genuinely unified prototypes is not directly validated. The significance of the paper depends on whether these gaps can be closed with additional experiments.

major comments (5)
  1. [§4, Table 2 and Table 3] The cross-site claim is not tested. SkinCon and Derm7pt are combined and randomly split 8:1:1, so the results in Table 3 are in-distribution performances on the training cohorts, not cross-site transfer results. The only held-out dataset is PH2, which is dermoscopic and therefore tests only within-modality transfer. To support the central claim of 'cross-site' generalization, the authors should evaluate with a leave-one-dataset-out protocol or add an unseen clinical/dermoscopic dataset from a different institution, and report site-holdout metrics.
  2. [Eq. (4), §4.6.2 and Fig. 7(b)] The UCPC construction in Eq. (4) averages text embeddings of dataset-specific LVLM descriptions and assumes the frozen BiomedCLIP text space aligns dermoscopic and clinical taxonomies. If this assumption fails, the codebook is a concatenation of modality-specific clusters and the observed modality separation in Fig. 4(b–c) can be explained by the ICRG learning to suppress the irrelevant cluster, not by a unified semantic space. The t-SNE in Fig. 7(b) uses test features from the same cohorts, and Table 7 reports retrieval only on SkinCon; neither directly tests cross-modal alignment. A direct experiment is needed, e.g., measuring whether dermoscopic visual queries retrieve the correct clinical prototypes and vice versa, or evaluating cross-modal zero-shot concept prediction.
  3. [§4.6, Table A1 and Fig. 6] Quantitative test-time intervention is reported only on PH2, a dermoscopic dataset. The claim of 'cross-modal corrections'—applying dermoscopic concepts to clinical photographs and vice versa—is supported only by the illustrative Fig. 6, with no quantitative result. The authors should report intervention accuracy/RMSE for cross-modal interventions, or explicitly scope the claim to within-modality intervention.
  4. [§4, SkinCon cleaning rule] The manuscript states that SkinCon is cleaned by removing concepts with >99% missing annotations, retaining 18 of 48 concepts. This post-hoc selection changes the label space and may affect both the concept prediction numbers in Table 4 and the unified-codebook claim. The retained concept list and the effect of this filtering on the results should be reported (e.g., sensitivity to the threshold). As written, the 48-concept claim in Table 2 and the 18-concept evaluation are inconsistent.
  5. [§4.4, Table 7] The cross-modal retrieval result is in-domain (SkinCon) and does not include retrieval between dermoscopic and clinical concept spaces. The 'robustness' result with 10% shuffled pairs (UniCon†) is interesting but not a substitute for an alignment test. Please report retrieval accuracy separately for dermoscopic and clinical concepts, and, if possible, for cross-modal queries.
minor comments (6)
  1. [Throughout] There are multiple typos: Table 1 'Demoscopic images' should be 'Dermoscopic'; Fig. 7 caption 'effective of test-time intervention' should be 'effect of'; Fig. 5 caption 'dot and globals' should be 'globules'.
  2. [Eq. (4)] The symbol d is used both as the embedding dimension and as the dataset index. Please disambiguate, e.g., use D for the dataset index or d_e for the dimension.
  3. [§4.5.2 and Fig. 4] The quantity Δ is used to describe modality differences in reliability and bottleneck output, but its precise definition is not given in the text. Please define it explicitly.
  4. [Table 6] Unlike Tables 3 and 4, the zero-shot PH2 results do not include standard deviations or confidence intervals. Given the small external test set, reporting variability would help assess the reliability of the comparison.
  5. [§7.2, Table A2] The multi-margin hyperparameters (τ_close, τ_spread, τ_in, τ_away, τ_inter) are fixed without sensitivity analysis. Since the margin constraints are essential to the alignment loss, a brief sensitivity study or at least a statement of stability would strengthen the paper.
  6. [§1 Contribution (3)] The phrase 'unprecedented cross-site intervention capabilities' overstates the evidence; the intervention experiments are limited to within-modality and in-distribution settings. Please temper the claim.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: UCPC is a text-embedding average, objectives are supervised, and interventions are test-time overrides; the cross-modal claim is under-tested but not circular.

full rationale

UniCon's derivation chain does not reduce to its own inputs. The UCPC (Eq. 4) is an explicit averaging of frozen text-encoder embeddings of LVLM-generated concept descriptions; it is a design choice rather than a fitted quantity, and it is not later used as a predicted target. Stage 1 (Eqs. 1-3) and Stage 2 (Eqs. 5-8) are standard contrastive, BCE, and CE objectives optimized against ground-truth concept and diagnosis labels, so the reported concept and diagnosis numbers are empirical benchmarks, not constructions. The test-time intervention (Eq. 9) is a deliberate one-hot override whose effect on the already-trained classifier is the quantity being measured; it is a causal sanity check, not a self-prediction. The paper's self-citations ([40], [48]) support only motivational claims about dataset adaptation overhead and human-in-the-loop needs; neither is load-bearing for the central derivation. The main weakness is evidentiary rather than circular: the claimed cross-modal unification is only validated on dermoscopic PH2, with no unseen clinical-cohort transfer test, and the assumption that the frozen text encoder aligns dermoscopic and clinical taxonomies is untested. That is a generalization/correctness risk, which belongs outside the circularity score.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The paper's central claims rest on four assumptions about the semantic behavior of frozen text encoders and LVLM generation, plus hand-chosen margins and loss weights. No new physical entities are introduced. The free parameters are standard training hyperparameters, but they are hand-chosen rather than derived.

free parameters (6)
  • tau_close = 0.1
    Margin for intra-textual anchoring (Eq. 1), hand-chosen.
  • tau_spread = 0.9
    Margin for anchor spread (Eq. 1), hand-chosen.
  • tau_in = 0.2
    Margin for image-to-centroid distance (Eq. 2), hand-chosen.
  • tau_away = 0.8
    Margin for distance from counter-example (Eq. 2), hand-chosen.
  • tau_inter = 0.5
    Margin for inter-class separation (Eq. 3), hand-chosen.
  • lambda_gate = 0.5
    Weight for the gating loss in Steps 1 and 3 (Eqs. 6-8), hand-chosen.
assumptions (4)
  • domain assumption BiomedCLIP text embeddings live in a shared semantic space that makes open-ended clinical text comparable across cohorts.
    Stage 1 and Eq. 4 rely on this to average text embeddings into a unified codebook.
  • domain assumption Qwen3-VL-generated MSS descriptions are faithful, image-grounded, and correctly describe concept presence or absence.
    Section 3.1 and Figure A1; if descriptions are wrong, the anchors and prototypes are wrong.
  • domain assumption Averaging text embeddings across datasets (Eq. 4) yields a cohort-agnostic prototype without losing discriminative semantics.
    Section 3.2; this is the core construction of UCPC and is not externally validated.
  • domain assumption The reliability gate trained with cohort-specific applicability masks transfers to inference when masks are discarded.
    ICRG in Section 3.2; the paper tests this only implicitly via zero-shot PH2.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Open-Linguistic Concept Unified Learning for Cross-Site Interpretable Dermatology Image Diagnosis." pith.science (2026). https://pith.science/paper/O7GU3267

@misc{pith2026260803225,
  author       = {Pith},
  title        = {Pith review of: Open-Linguistic Concept Unified Learning for Cross-Site Interpretable Dermatology Image Diagnosis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/O7GU3267}},
  note         = {Machine review of arXiv:2608.03225}
}
read the original abstract

Human-interpretable computer-aided diagnosis is crucial for clinical decision making. Concept-based models excel by providing transparent reasoning and enabling post-hoc, clinician-in-the-loop interventions. However, their rigid dataset-specific adaptation inherently restricts cross-site generalization. Applying them across diverse modalities, such as dermoscopic and clinical photographs, is challenging due to heterogeneous concept taxonomies varying in availability, granularity, and semantics across cohorts. Consequently, adapting Foundation Vision-Language Models (FVLMs) demands costly label engineering and repeated post-training. Existing intervention mechanisms remain rigidly tied to predefined concepts, lacking adaptability and hindering scalable dermatology CAD deployment. To address these bottlenecks, we propose UniCon, an open-linguistic unified concept learning framework for multimodal interpretable vision-language diagnosis. UniCon resolves these challenges through three contributions: (1) A shared semantic representation space via a unified concept prototype codebook, seamlessly coordinating heterogeneous concept systems across modalities without dataset-specific retraining. (2) Open-linguistic based multi-faceted semantic specifications to overcome sparse textual label limitations, improving boundary sensitivity in uncertain clinical contexts. (3) A robust, cross-site adjustable intervention interface powered by reliability-gated bottleneck aggregation, enabling consistent reasoning and transferable clinician corrections. Extensive experiments demonstrate that beyond securing top-tier diagnostic accuracy, UniCon successfully bridges disparate clinical taxonomies, unlocking unprecedented cross-site intervention capabilities. Code is available at https://github.com/wuchengyu123/UniCon.

Figures

Figures reproduced from arXiv: 2608.03225 by the authors.

Figure 1
Figure 1. Overview of existing approaches versus ours. (a) [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Illustration of the proposed Multi-Faceted Semantic [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Overview of UniCon. (a) Stage 1 aligns images and MSS in a shared space using the multi-margin objective [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Concept bottleneck and reliability gate visualization. (a) Sankey flow from image evidence through gated concepts to [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 6
Figure 6. Figure 6: Test-time positive and negative intervention visual [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 5
Figure 5. Figure 5: Grad-CAM visualization of dermoscopic and clinical [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 7
Figure 7. Figure 7: Test-time intervention results of our model. (a) [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

55 extracted references · 44 canonical work pages

  1. [1]

    Diego Ardila, Atilla P Kiraly, Sujeeth Bharadwaj, Bokyung Choi, Joshua J Reicher, Lily Peng, Daniel Tse, Mozziyar Etemadi, Wenxing Ye, Greg Corrado, et al. 2019. End-to-end lung cancer screening with three-dimensional deep learning on low- dose chest computed tomography.Nature Medicine25, 6 (2019), 954–961

    work page 2019

  2. [2]

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, et al . 2025. Qwen3-VL technical report. arXiv preprint arXiv:2511.21631(2025)

    arXiv 2025

  3. [3]

    Yequan Bie, Luyang Luo, and Hao Chen. 2024. MICA: Towards explainable skin lesion diagnosis via multi-level image-concept alignment. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 837–845

    work page 2024

  4. [4]

    Yuwen Chen, Ju Zhang, and Xiaolin Qin. 2022. Interpretable instance disease prediction based on causal feature selection and effect analysis.BMC Medical Informatics and Decision Making22, 1 (2022), 51

    work page 2022

  5. [5]

    Yu-Hxiang Chen, Ting-Ting Chang, Yao-Zhi Xue, Wei-Hsiang Sung, and Chia-Yu Lin. 2025. Evaluating the feasibility of vision-language models in skin cancer detection: a comparative study with CNNs and vision transformers. InIEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology. IEEE, 1–7

    work page 2025

  6. [6]

    Roxana Daneshjou, Kailas Vodrahalli, Roberto A Novoa, et al. 2022. Disparities in dermatology AI performance on a diverse, curated clinical image set.Science Advances8, 31 (2022), eabq6147

    work page 2022

  7. [7]

    Roxana Daneshjou, Mert Yuksekgonul, Zhuo Ran Cai, Roberto Novoa, and James Y Zou. 2022. SkinCon: A skin disease dataset densely annotated by domain ex- perts for fine-grained debugging and analysis.Advances in Neural Information Processing Systems35 (2022), 18157–18167

    work page 2022

  8. [8]

    Yiheng Dong, Yi Lin, and Xin Yang. 2025. CoPA: hierarchical concept prompting and aggregating network for explainable diagnosis. InInternational Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 67–76

    work page 2025

Show all 55 references
  1. [9]

    Yingjun Du, Wenfang Sun, and Cees G Snoek. 2024. IPO: Interpretable prompt op- timization for vision-language models.Advances in Neural Information Processing Systems37 (2024), 126725–126766

    2024

  2. [10]

    Niharika Dutta, Kartik Bose, Emir Syailendra, Linda Chu, and Pankaj Gupta. 2025. Vision-language models in diagnostic imaging: review of technical advances, clinical validation, and practical deployment.International Journal of Medical Informatics(2025), 106227

    2025

  3. [11]

    Andre Esteva, Brett Kuprel, Roberto A Novoa, Justin Ko, Susan M Swetter, He- len M Blau, and Sebastian Thrun. 2017. Dermatologist-level classification of skin cancer with deep neural networks.Nature542, 7639 (2017), 115–118

    2017

  4. [12]

    Samuel G Finlayson, Adarsh Subbaswamy, Karandeep Singh, et al . 2021. The clinician and dataset shift in artificial intelligence.New England Journal of Medicine385, 3 (2021), 283–286

    2021

  5. [13]

    Marta Fonseca, Paula Broeiro-Gonçalves, Mariana Barosa, Pedro Marvão, Marta Carreira, Sofia Azeredo-Lopes, Joana Pires, António Rendas, Patrícia Rosado- Pinto, and Bruno Heleno. 2024. Concept mapping to promote clinical reasoning in multimorbidity: a mixed methods study in und...

    2024

  6. [14]

    Yibo Gao, Zheyao Gao, Xin Gao, et al . 2024. Evidential concept embedding models: Towards reliable concept explanations for skin disease diagnosis. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 308–317

    2024

  7. [15]

    Yunhe Gao, Difei Gu, Mu Zhou, et al. 2024. Aligning human knowledge with visual concepts towards explainable medical image classification. InInternational Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 46–56

    2024

  8. [16]

    Yunpeng Gong, Yongjie Hou, Jiangming Shi, KL DIEP, and Min Jiang. 2026. A Theory-Inspired Framework for Few-Shot Cross-Modal Sketch Person Re- Identification. (2026)

    2026

  9. [17]

    Yunpeng Gong, Liqing Huang, and Lifei Chen. 2022. Person re-identification method based on color attack and joint defence. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 4313–4322

    2022

  10. [18]

    Yunpeng Gong, Zhun Zhong, Yansong Qu, Zhiming Luo, Rongrong Ji, and Min Jiang. 2024. Cross-modality perturbation synergy attack for person re- identification.Advances in Neural Information Processing Systems37 (2024), 23352–23377

    2024

  11. [19]

    Matthew Groh, Caleb Harris, Luis Soenksen, Felix Lau, Rachel Han, Aerin Kim, Arash Koochek, and Omar Badri. 2021. Evaluating deep neural networks trained on clinical images in dermatology with the Fitzpatrick 17k dataset. InProceedings of the IEEE/CVF Conference on Computer Vi...

    2021

  12. [20]

    Varun Gulshan, Lily Peng, Marc Coram, Martin C Stumpe, Derek Wu, Arunacha- lam Narayanaswamy, Subhashini Venugopalan, Kasumi Widner, Tom Madams, Jorge Cuadros, et al. 2016. Development and validation of a deep learning algo- rithm for detection of diabetic retinopathy in retin...

    2016

  13. [21]

    Kai Han, Yunhe Wang, Hanting Chen, Xinghao Chen, Jianyuan Guo, Zhenhua Liu, Yehui Tang, An Xiao, Chunjing Xu, Yixing Xu, et al. 2022. A survey on vision transformer.IEEE Transactions on Pattern Analysis and Machine Intelligence45, 1 (2022), 87–110

    2022

  14. [22]

    Hangzhou He, Jiachen Tang, Lei Zhu, Kaiwen Li, and Yanye Lu. 2025. Training- free test-time improvement for explainable medical image classification. InInter- national Conference on Medical Image Computing and Computer-Assisted Inter- vention. Springer, 638–648

    2025

  15. [23]

    Jeremy Kawahara, Sara Daneshvar, et al. 2018. Seven-point checklist and skin lesion classification using multitask multimodal neural nets.IEEE Journal of Biomedical and Health Informatics23, 2 (2018), 538–546

    2018

  16. [24]

    Pang Wei Koh, Thao Nguyen, Yew Siang Tang, et al. 2020. Concept bottleneck models. InInternational Conference on Machine Learning. PMLR, 5338–5348

    2020

  17. [25]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. BLIP-2: Bootstrap- ping language-image pre-training with frozen image encoders and large language models. InInternational Conference on Machine Learning. PMLR, 19730–19742

    2023

  18. [26]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual in- struction tuning.Advances in Neural Information Processing Systems36 (2023), 34892–34916

    2023

  19. [27]

    Sarah Matta, Mathieu Lamard, Philippe Zhang, et al. 2024. A systematic review of generalization research in medical image classification.Computers in Biology and Medicine183 (2024), 109256

    2024

  20. [28]

    Teresa Mendonça, Pedro M Ferreira, Jorge S Marques, André RS Marcal, and Jorge Rozeira. 2013. PH2: A dermoscopic image database for research and benchmark- ing. In35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE, 5437–5440

    2013

  21. [29]

    Michael Moor, Oishi Banerjee, Zahra Shakeri Hossein Abad, et al. 2023. Founda- tion models for generalist medical artificial intelligence.Nature616, 7956 (2023), 259–265

    2023

  22. [30]

    Keshav Shree Mudgal and Neelanjan Das. 2020. The ethical adoption of artificial intelligence in radiology.BJR|Open2, 1 (2020), 20190020

    2020

  23. [31]

    Luke Oakden-Rayner, Jared Dunnmon, Gustavo Carneiro, et al. 2020. Hidden stratification causes clinically meaningful failures in machine learning for medical imaging. InProceedings of the ACM Conference on Health, Inference, and Learning. 151–159

    2020

  24. [32]

    Tuomas Oikarinen, Subhro Das, Lam M Nguyen, et al. 2023. Label-free concept bottleneck models. InInternational Conference on Learning Representations

    2023

  25. [33]

    Winnie Pang, Xueyi Ke, Satoshi Tsutsui, and Bihan Wen. 2024. Integrating clinical knowledge into concept bottleneck models. InInternational Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 243–253

    2024

  26. [34]

    Cristiano Patrício, Luis F Teixeira, and João C Neves. 2024. Towards concept- based interpretability of skin lesion diagnosis using vision-language models. In IEEE International Symposium on Biomedical Imaging. IEEE, 1–5

    2024

  27. [35]

    Cristiano Patrício, Luís F Teixeira, and João C Neves. 2025. A two-step concept- based approach for enhanced interpretability and trust in skin lesion diagnosis. Computational and Structural Biotechnology Journal28 (2025), 71–79

    2025

  28. [36]

    Alec Radford, Jong Wook Kim, Chris Hallacy, et al. 2021. Learning transferable visual models from natural language supervision. InInternational Conference on Machine Learning. PMLR, 8748–8763

    2021

  29. [37]

    Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedan- tam, Devi Parikh, and Dhruv Batra. 2017. Grad-CAM: Visual explanations from deep networks via gradient-based localization. InProceedings of the IEEE Interna- tional Conference on Computer Vision. 618–626

    2017

  30. [38]

    Matthew Shen, Aliyah Hsu, Abhineet Agarwal, and Bin Yu. 2025. Adaptive test- time intervention for concept bottleneck models.arXiv preprint arXiv:2503.06730 (2025)

    2025 arXiv

  31. [39]

    Sungbin Shin, Yohan Jo, Sungsoo Ahn, and Namhoon Lee. 2023. A closer look at the intervention procedure of concept bottleneck models. InInternational Conference on Machine Learning. PMLR, 31504–31520

    2023

  32. [40]

    Wenfang Sun, Yingjun Du, Gaowen Liu, Yefeng Zheng, and Cees GM Snoek. 2026. QUOTA: Quantifying objects with text-to-image models for any domain. In2026 IEEE/CVF Winter Conference on Applications of Computer Vision (W ACV). IEEE, 6381–6390

    2026

  33. [41]

    Wenfang Sun, Yingjun Du, Xiantong Zhen, Fan Wang, Ling Wang, and Cees GM Snoek. 2023. MetaModulation: Learning variational feature hierarchies for few- shot learning with fewer tasks. InInternational Conference on Machine Learning. PMLR, 32847–32858

    2023

  34. [42]

    Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research9, 11 (2008)

    2008

  35. [43]

    Yuheng Wang, Tianze Yu, Jiayue Cai, Sunil Kalia, Harvey Lui, Z Jane Wang, and Tim K Lee. 2025. Integrating clinical knowledge graphs and gradient-based neural systems for enhanced melanoma diagnosis via the seven-point checklist. IEEE Transactions on Neural Networks and Learni...

    2025

  36. [44]

    David Wen, Saad M Khan, Antonio Ji Xu, et al. 2022. Characteristics of publicly available skin cancer image datasets: a systematic review.The Lancet Digital Health4, 1 (2022), e64–e74

    2022

  37. [45]

    Walter F Wiggins and Ali S Tejani. 2022. On the opportunities and risks of foundation models for natural language processing in radiology.Radiology: Artificial Intelligence4, 4 (2022), e220119

    2022

  38. [46]

    Martin J Willemink, Wojciech A Koszek, Cailin Hardell, et al. 2020. Preparing medical imaging data for machine learning.Radiology295, 1 (2020), 4–15

    2020

  39. [47]

    Daniel Wolf, Heiko Hillenhagen, Billurvan Taskin, et al. 2025. Your other left! vision-language models fail to identify relative positions in medical images. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 691–701

    2025

  40. [48]

    Hongtao Wu, Yifeng Wu, Jiaxuan Jiang, Chengyu Wu, Hong Wang, and Yefeng Zheng. 2025. SAMVSR: Leveraging Semantic Priors to Zone-Focused Mamba for Video Snow Removal. InProceedings of the 33rd ACM International Conference on Multimedia. 7376–7385

    2025

  41. [49]

    Hongtao Wu, Yijun Yang, Angelica I Aviles-Rivero, Jingjing Ren, Sixiang Chen, Haoyu Chen, and Lei Zhu. 2024. Semi-supervised video desnowing network via temporal decoupling experts and distribution-driven contrastive regularization. InEuropean Conference on Computer Vision. Sp...

    2024

  42. [50]

    Xinyue Xu, Yueying Hu, Hui Tang, Yi Qin, Lu Mi, Hao Wang, and Xiaomeng Li. 2025. Concept-based unsupervised domain adaptation.arXiv preprint arXiv:2505.05195(2025)

    2025 arXiv

  43. [51]

    Siyuan Yan, Xieji Li, Ming Hu, Yiwen Jiang, Zhen Yu, and Zongyuan Ge. 2025. MAKE: Multi-aspect knowledge-enhanced vision-language pretraining for zero- shot dermatological assessment. InInternational Conference on Medical Image Computing and Computer-Assisted Intervention. Spr...

    2025

  44. [52]

    Siyuan Yan, Zhen Yu, Clare Primiero, Cristina Vico-Alonso, Zhonghua Wang, Litao Yang, Philipp Tschandl, Ming Hu, Lie Ju, Gin Tan, et al. 2025. A multimodal vision foundation model for clinical dermatology.Nature Medicine31, 8 (2025), 2691–2702

    2025

  45. [53]

    Yue Yang, Artemis Panagopoulou, Shenghao Zhou, et al . 2023. Language in a bottle: Language model guided concept bottlenecks for interpretable image classification. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 19187–19197

    2023

  46. [54]

    Mert Yuksekgonul, Maggie Wang, and James Zou. 2022. Post-hoc concept bottle- neck models.arXiv preprint arXiv:2205.15480(2022)

    2022 arXiv

  47. [55]

    indistinct but lacks any directional linear ele- ments

    Sheng Zhang, Yanbo Xu, Naoto Usuyama, et al. 2023. BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs.arXiv preprint arXiv:2303.00915(2023). 7 Appendix 7.1 Test-time Intervention Details To evaluate the interpretabi...

    2023 arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.