Pith. sign in

REVIEW 5 major objections 5 minor 25 references

A three-stage pipeline that aligns risk metadata with images beats prior medical diagnosis models on two benchmarks.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

PRIMA aligns images and patient metadata using four contrastive-style losses plus an LLM-generated clinical knowledge corpus, reporting higher average F1 than SOTA baselines on PAD-UFES-20 and the private AQUA dataset.

T0 review reviewed 2026-08-02 challenge →

load-bearing objection PRIMA is a plausible engineering contribution with real but combinatorial novelty; its central 'alignment-driven >5%' claim is confounded by encoder and LLM differences, and the authors openly admit this in the conclusion. the 5 major comments →

arxiv 2602.23297 v2 pith:4LJOAGWQ submitted 2026-02-26 cs.CV

PRIMA: Pre-training with Risk-integrated Image-Metadata Alignment for Medical Diagnosis via LLM

Yiqing Wang , Chunming He , Ming-Chen Lu , Mercy Pawar , Leslie Niziol , Maria Woodward , Sina Farsiu This is my paper
classification cs.CV
keywords risk-disease correlationretrieval-augmented generationclinical knowledge injectionvision-language alignmentmedical image classificationmetadata fusionskin lesion diagnosiskeratitis
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes PRIMA, a framework for medical image diagnosis that treats clinical metadata as rich semantic knowledge rather than isolated tags. The central claim is that injecting risk–disease priors into the text encoder via a RAG-curated, physician-vetted corpus, then aligning image and text features with four complementary losses, improves classification beyond current state-of-the-art. On PAD-UFES-20 (skin lesions) and AQUA (keratitis), PRIMA reports F1/accuracy of 73.75/78.27 and 85.22/86.04 respectively, consistently beating baselines by more than five accuracy points. The authors argue this is achieved without massive paired datasets or extensive compute, because the knowledge is distilled from literature rather than collected as data.

Core claim

PRIMA's core discovery is that explicit risk–disease knowledge can be made part of the representation learning process itself. A dual-encoder stage aligns DINOv3's image features with a Clinical ModernBERT text encoder that has been fine-tuned on a RAG-generated corpus of risk factor–diagnosis relationships. Four losses capture different granularities: image consistency across scans, global semantic alignment, local token-to-patch grounding, and soft alignment that relaxes one-to-one matching to account for shared clinical attributes. A final stage uses a large language model (Qwen3) to fuse the aligned features, with vocabulary-restricted logits to avoid hallucination. The paper reports sta

What carries the argument

The key machinery is the suite of four alignment objectives — image consistency (intra-patient visual invariance), global semantic (image–text class-token contrast), local semantic (attention-guided grounding of text tokens to image patches), and soft semantic (metadata-derived soft labels) — plus the RAG-refined text encoder that embeds diagnostic priors. Together they map heterogeneous scans and structured metadata into a shared feature space that an LLM can reason over.

Load-bearing premise

The load-bearing premise is that the RAG-generated risk–disease corpus is an accurate, unbiased, and sufficiently validated distillation of clinical literature; if it contains hallucinated correlations or the metadata filter was chosen with validation performance, the reported gains may come from aligning images to self-generated prose rather than from real clinical priors.

What would settle it

Take the pipeline and replace the RAG-generated corpus with a corpus of random risk–diagnosis pairings (keeping the same procedures); if the model's accuracy on PAD-UFES-20 and AQUA stays at the same level, the clinical content is not the source of the gain. Alternatively, have clinicians independently score the risk–disease statements in the corpus and check whether classification accuracy on each disease correlates with the quality of its corpus entries.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the claims hold, clinical metadata such as age, sun exposure, and family history can be exploited systematically rather than concatenated as tags, improving diagnosis in low-data settings.
  • The RAG-based corpus curation suggests that expert clinical literature can substitute for large paired image-text datasets in pretraining for medical vision-language models.
  • The four-loss design indicates that multi-granular alignment—pixels, patches, and global semantics—is necessary to capture the loose correlations between images and clinical risk factors.
  • Private-dataset gains (AQUA) argue that the improvement is not from memorizing publicly available images/metadata, but from the alignment strategy itself.
  • The vocabulary-restricted LLM fusion offers a way to use generative models for classification without hallucinated free-form outputs.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A testable extension: apply the same pipeline to other specialties where structured risk factors are routine (e.g., diabetic retinopathy with HbA1c and diabetes duration); the soft-loss formulation may already accommodate continuous metadata if converted to similarity vectors.
  • A caution: because the RAG corpus is synthesized by LLMs and the paper does not fully specify the physician vetting or the metadata attribute filter, a careful re-run with a corpus built from manual extractions could reveal whether the gains come from the clinical content or from additional training signal.
  • If the soft semantic loss's label sharpening is effective on these two datasets, it might transfer to multi-label or co-morbid conditions where strict one-to-one mapping is known to hurt.
  • The paper states it needs only LoRA-tuned backbones and two RTX 4090 GPUs; if that efficiency reproduces, the approach could be practical in clinics with limited compute, but the 2500-epoch stage 1 suggests a hidden compute cost.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. PRIMA proposes a three-stage framework for medical image classification. Stage 1 uses RAG on PubMed with GPT-5.1/Gemini-2.5 to generate a physician-vetted risk-disease corpus and fine-tunes Clinical ModernBERT via masked language modeling. Stage 2 aligns DINOv3 image features with the refined text encoder through four losses: image consistency (Eq. 2), global semantic consistency (Eq. 3), local semantic consistency (Eq. 5), and soft semantic consistency (Eq. 6). Stage 3 fuses the aligned features with a frozen Qwen3-1.7B under LoRA and vocabulary-restricted classification. The paper reports state-of-the-art results on PAD-UFES-20 and the private AQUA dataset, and attributes >5% accuracy gains to multi-granular alignment. The loss equations are internally consistent and the experimental protocol includes five-fold cross-validation with standard deviations, but the attribution of gains to the alignment mechanism is not isolated and several reporting details are missing.

Significance. If the reported results are reproducible, PRIMA would be a practically useful contribution: it combines knowledge-injected text encoding with feature alignment and LLM fusion to improve diagnostic classification, and the AQUA evaluation is valuable because that dataset is private and inaccessible to pretrained foundation models. The framework is described in enough detail to be re-implemented, and reporting means with standard deviations is good practice. The paper's main weakness is that its central claim—that multi-granular alignment itself yields the >5% improvement—is currently under-evidenced. The comparisons conflate encoder strength, text branch, RAG corpus, and LLM fusion, and the conclusion explicitly acknowledges the need for backbone-controlled experiments. The significance of the work therefore depends on closing this gap.

major comments (5)
  1. [§3.1, Tables 1–2, Conclusion] The claim that 'PRIMA consistently yields a >5% accuracy boost via multi-granular alignment' is not supported by the presented comparisons. Against the +Metadata-MLP baseline, the PAD-UFES-20 accuracy gain is 78.27 − 73.93 = 4.34 points, not >5; against DINOv3 alone the gain is >5, but that baseline lacks the text branch, the RAG corpus, and the Qwen3 fusion stage. Table 3 toggles the four losses but does not state whether Stage-3 Qwen3 fusion and the supervised fine-tuning are held fixed in every row. The conclusion explicitly acknowledges the need for backbone-controlled experiments; without such an experiment, the measured gains cannot be attributed to Limg, Lglo, Lloc, and Lsoft. Please add a controlled experiment in which the only change is the addition or removal of the four alignment losses.
  2. [Table 3] The ablation table is not legible as printed: the checkmarks and numeric entries do not align cleanly with the column headers, and rows with what appear to be identical checkmark patterns show different results (e.g., the rows with five checkmarks show F1 values 70.02, 71.42, and 73.75). The columns Lloc_dir and Lsup_con are not clearly toggled in any row, and there is no key stating which components each row includes. Since the per-component attribution is a central claim, please provide a correctly formatted table with explicit row labels, and report paired significance tests or per-fold differences across the five cross-validation folds.
  3. [§3.1, Table 1] The phrase 'significantly outperforming' is not supported by statistical tests. Five-fold means with standard deviations overlap in several key comparisons. For example, on PAD-UFES-20, PRIMA's average F1 is 73.75±3.40 versus MedBLIP's 72.01±2.36, and on the SCC class PRIMA is lower (37.33±7.86 vs. 38.37±5.00). Please provide paired tests (e.g., paired t-test or Wilcoxon over folds, or bootstrap over patients) for F1, accuracy, and balanced accuracy, and temper the 'consistently' claim if per-class results are not uniformly better.
  4. [§2.2, Eq. (6)] The soft semantic loss needs clarification on what enters the vector y. If the metadata sequence includes the disease label or a one-hot diagnosis, then Lsoft is at least partly supervised by the label, and the statement that it captures 'clinical ambiguity' should be revised. If it does not, specify exactly which metadata fields are used, how the disease-related dimensions are identified and upweighted, and whether the metadata filter was chosen before or after inspecting validation performance. This distinction is load-bearing because Lsoft is presented as a metadata-driven novelty and because the later comparison with Lsup_con (basic class labels) in Table 3 implies a contrast that the current text does not define.
  5. [§2.1] Stage 1 is the foundation of the claimed knowledge injection, but its implementation is underspecified. Please report the size and composition of the retrieved PubMed corpus, the exact prompts (or state that the prompt shown in Figure 2 was adapted for AQUA, which concerns keratitis rather than skin lesions), the number of generated risk-disease descriptions, the physician vetting protocol, and how disagreements or hallucinations were handled. Without these details, the RAG corpus cannot be audited for spurious correlations and the stage cannot be reproduced by other groups.
minor comments (5)
  1. [Abstract, §3] The abstract says 'Our code will is available at https://github.com/yqwang01/PRIMA' while the body says 'Our code will be public upon acceptance.' Please make the availability statement consistent and include the link if the repository is accessible.
  2. [Figure 2] The font size in Figure 2 is very small, particularly for the prompt text and the loss labels. Please enlarge or vectorize the figure so the components are readable.
  3. [§2.2] The phrase 'we filter for visually relevant attributes' is not operationalized. Please provide the list of metadata fields retained for PAD-UFES-20 and for AQUA, and state the filtering criterion.
  4. [§3.2] The row labels in Table 3 are ambiguous. The header includes 'Knowledge Pretraining' and five loss/modification columns, but no row clearly isolates 'with vs. without knowledge pretraining' as a single change. Please label each row explicitly, e.g., 'A: no pretraining, no alignment'.
  5. [§2.2] For patients with only a single image, the paper says a 'duplicated single scan' is used as the positive pair. Please clarify whether independent augmentations are applied to the two copies and whether this is equivalent to standard multi-crop augmentations in the denominator of Eq. (2).

Circularity Check

0 steps flagged

No significant circularity: PRIMA's losses and benchmarks are externally grounded; the acknowledged alignment-vs-encoder confound is a validity issue, not a circular derivation.

full rationale

I find no circularity in PRIMA's derivation chain. The central empirical claims are benchmark results on PAD-UFES-20 and AQUA, evaluated with external metrics (F1, accuracy, BAcc). The four alignment losses in Section 2.2 (L_img, L_glo, L_loc, L_soft) are standard contrastive and soft-target objectives; L_soft builds soft targets from metadata similarity (Eq. 6: s_{i,j} = Softmax(<y_i,y_j>/tau_label)), not from test labels or from the final classifier's predictions. Supervised fine-tuning with ground-truth labels is ordinary training, not a fitted-input-called-prediction step. Stage 1's corpus is generated from public literature via RAG with physician vetting, and Stage 3 uses Qwen3 with vocabulary-restricted classification; neither is defined in terms of PRIMA's own outputs. The only overlapping-author citation, [10], is a background reference for large-scale medical datasets and is not load-bearing for PRIMA's novelty or results. The skeptic's concern that the >5% boost is not isolated from encoder strength and LLM fusion is a genuine internal-validity confound, and the paper itself acknowledges it in the Conclusion ('the need for backbone-controlled experiments to isolate alignment gains from encoder strength'). But an unresolved confound is not circularity: it does not make the reported result equivalent to an input by construction, and no equation or citation chain reduces the claimed prediction to a fitted parameter or to the authors' own prior claims. Therefore the appropriate circularity score is 0.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The paper contributes an assembly of existing models and hand-set hyperparameters rather than a derivation. The ledger is therefore dominated by design choices (loss weights, temperatures, upweighting factor, LoRA ranks) and by trust in third-party components (DINOv3, Clinical ModernBERT, Qwen3, the RAG corpus). No new physical or conceptual entities are introduced.

free parameters (5)
  • Alignment loss weights beta_1..beta_4 = 0.2, 0.3, 0.2, 0.3
    Set in Eq. 7 without sensitivity analysis or stated selection criterion; they balance all four alignment terms and directly control the pre-training objective.
  • Similarity temperature tau = unspecified
    Used in Eq. 1 for every contrastive term; the value is not reported, yet it strongly affects gradient behavior in all four losses.
  • Soft-label temperature tau_label = unspecified
    Controls label sharpening in Lsoft; the value is not reported, so the effective supervision signal is not reproducible.
  • Disease-dimension upweighting factor = 3
    Metadata one-hot vectors are multiplied by 3 on disease-related dimensions in Lsoft, chosen by hand without justification.
  • LoRA ranks / learning rates / epochs = r=8/32/16; LR 5e-5/3e-5/1e-5; 2500/150/80 epochs
    Reported but not ablated; the central performance claims depend on these choices, and there is no evidence they transfer to other datasets.
axioms (5)
  • domain assumption The GPT-5.1/Gemini-2.5 RAG-generated risk-disease corpus is accurate and physician-vetted
    Section 2.1 states LLMs generate structured descriptions 'vetted by senior physicians,' but vetting details, corpus size, and disagreement handling are omitted. If the corpus is wrong, the text encoder injects false priors.
  • domain assumption Masked-language-modeling refinement on the corpus transfers usable diagnostic priors to Clinical ModernBERT
    Section 2.1 uses MLM for knowledge injection; no evaluation shows the refined text embeddings actually encode the intended risk-disease relationships beyond a generic language-model objective.
  • domain assumption DINOv3 features are suitable medical image representations for skin and cornea images without domain-specific pretraining
    DINOv3 is used as the vision backbone with LoRA. If its representations are poor for dermoscopy or slit-lamp images, the alignment gains would vanish.
  • domain assumption PAD-UFES-20 and AQUA labels, metadata, and split protocols are correct
    All metrics depend on these data. AQUA is private and cannot be audited, and the metadata filtering in Section 2.2 may have been chosen with validation-set knowledge.
  • domain assumption Qwen3 with LoRA and vocabulary-restricted softmax performs reliable classification from projected features
    Section 2.3 assumes the LLM fuses aligned features without overfitting or hallucination; no separate ablation isolates the LLM fusion benefit from the alignment stage.

reviewed 2026-08-02 · how reviews work

0 comments
Cite this review

Pith. "Pith review of PRIMA: Pre-training with Risk-integrated Image-Metadata Alignment for Medical Diagnosis via LLM." pith.science (2026). https://pith.science/paper/4LJOAGWQ

@misc{pith2026260223297,
  author       = {Pith},
  title        = {Pith review of: PRIMA: Pre-training with Risk-integrated Image-Metadata Alignment for Medical Diagnosis via LLM},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4LJOAGWQ}},
  note         = {Machine review of arXiv:2602.23297}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Medical diagnosis requires the effective synthesis of visual manifestations and clinical metadata. However, existing methods often treat metadata as isolated tags, failing to exploit the rich semantic knowledge embedded in clinical descriptions. We propose PRIMA (Pre-training with Risk-integrated Image-Metadata Alignment), a framework that integrates domain-specific knowledge into multi-modal representation learning. We first curate an expert corpus of risk--disease correlations via Retrieval-Augmented Generation (RAG) to refine Clinical ModernBERT, embedding diagnostic priors into the text encoder. To bridge the modality gap, we introduce a dual-encoder pre-training strategy utilizing DINOv3 and our refined Clinical ModernBERT, optimized by a suite of four complementary loss functions. These losses are designed to capture multi-granular semantic alignment and handle the ambiguity of clinical correlations through soft labels. Finally, we leverage Qwen3 to fuse these aligned features for precise disease classification. Extensive experiments demonstrate that PRIMA effectively harmonizes pixel-level features with abstract clinical expertise, consistently outperforming other state-of-the-art methods. Notably, our framework achieves strong performance without requiring massive data collection or exhaustive computational resources. Our code will is available at https://github.com/yqwang01/PRIMA.

Figures

Figures reproduced from arXiv: 2602.23297 by Chunming He, Leslie Niziol, Maria Woodward, Mercy Pawar, Ming-Chen Lu, Sina Farsiu, Yiqing Wang.

Figure 1
Figure 1. Figure 1: Traditional vs. Our PRIMA Approach. Furthermore, data scarcity remains a persistent barrier; despite recent efforts to curate large-scale medical datasets [21,10,19], relying on massive data is often in￾feasible for specialized tasks or rare diseases where patient cohorts are inherently limited. Previous studies [13,17] have explored metadata fusion, yet these often rely on ad-hoc designs that impede gener… view at source ↗
Figure 2
Figure 2. Figure 2: illustrates the overall architecture of PRIMA, which comprises three pro￾gressive training stages. First, we curate a domain-specific knowledge bank via Retrieval-Augmented Generation (RAG) using GPT [15] and Gemini [3] based on public literature. This corpus serves as the foundation for fine-tuning our text encoder, ClinicalBERT [7], to inject medical priors (Section 2.1). Next, we introduce a set of four… view at source ↗
Figure 3
Figure 3. Figure 3: Training Objectives of Our PRIMA. product (Eq. 1) with hyperparameter τ . sim(x1, x2) = x ⊤ 1 x2 ∥x1∥∥x2∥ /τ (1) For a batch of N patients, let p cls i,1 and p cls i,2 be global visual tokens from two views (distinct scans or augmentations) of patient i. Defined in Eq. 2, Limg promotes patient-invariant representation learning by aligning latent features belonging to the same subject, where ⊮ is the indica… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

25 extracted references · 4 linked inside Pith

  1. [1]

    PLOS Digital Health4(6), e0000877 (2025)

    Amugongo, L.M., Mascheroni, P., Brooks, S., Doering, S., Seidel, J.: Retrieval aug- mented generation for large language models in healthcare: A systematic review. PLOS Digital Health4(6), e0000877 (2025)

    2025

  2. [2]

    In: Proceedings of the Asian conference on computer vision

    Chen, Q., Hong, Y.: Medblip: Bootstrapping language-image pre-training from 3d medical images and texts. In: Proceedings of the Asian conference on computer vision. pp. 2404–2420 (2024)

    2024

  3. [3]

    arXiv preprint arXiv:2507.06261 (2025)

    Comanici, G., Bieber, E., Schaekermann, M., Pasupat, I., Sachdeva, N., Dhillon, I., Blistein, M., Ram, O., Zhang, D., Rosen, E., et al.: Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261 (2025)

    Pith/arXiv arXiv 2025

  4. [4]

    In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers)

    Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidi- rectional transformers for language understanding. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers). pp. 4171–4186 (2019)

    2019

  5. [5]

    In: Interna- tional conference on medical image computing and computer-assisted intervention

    Du, J., Guo, J., Zhang, W., Yang, S., Liu, H., Li, H., Wang, N.: Ret-clip: A retinal image foundation model pre-trained with clinical diagnostic reports. In: Interna- tional conference on medical image computing and computer-assisted intervention. pp. 709–719. Springer (2024)

    2024

  6. [6]

    Iclr1(2), 3 (2022)

    Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., et al.: Lora: Low-rank adaptation of large language models. Iclr1(2), 3 (2022)

    2022

  7. [7]

    arXiv preprint arXiv:2504.03964 (2025)

    Lee, S.A., Wu, A., Chiang, J.N.: Clinical modernbert: An efficient and long context encoder for biomedical text. arXiv preprint arXiv:2504.03964 (2025)

    Pith/arXiv arXiv 2025

  8. [8]

    Advances in Neural Information Processing Systems36, 28541–28564 (2023)

    Li, C., Wong, C., Zhang, S., Usuyama, N., Liu, H., Yang, J., Naumann, T., Poon, H., Gao, J.: Llava-med: Training a large language-and-vision assistant for biomedicine in one day. Advances in Neural Information Processing Systems36, 28541–28564 (2023)

    2023

  9. [9]

    NPJ Digital Medicine7(1), 181 (2024)

    Li, Z., Xie, H., Wang, Z., Li, D., Chen, K., Zong, X., Qiang, W., Wen, F., Deng, Z., Chen, L., et al.: Deep learning for multi-type infectious keratitis diagnosis: a nationwide, cross-sectional, multicenter study. NPJ Digital Medicine7(1), 181 (2024)

    2024

  10. [10]

    arXiv preprint arXiv:2511.15943 (2025)

    Li, Z., Wang, Y., Farsiu, S., Kinahan, P.: Boosting medical visual understanding from multi-granular language learning. arXiv preprint arXiv:2511.15943 (2025)

    arXiv 2025

  11. [11]

    Effective clinical practice3(5) (2000) 10 Y

    Lindberg, D.: Internet access to the national library of medicine. Effective clinical practice3(5) (2000) 10 Y. Wang et al

    2000

  12. [12]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Liu, K., Ma, Z., Kang, X., Li, Y., Xie, K., Jiao, Z., Miao, Q.: Enhanced con- trastive learning with multi-view longitudinal data for chest x-ray report genera- tion. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 10348–10359 (2025)

    2025

  13. [13]

    IEEE Transactions on Medical Imaging 42(12), 3590–3601 (2023)

    Lu, S., Liu, J., Wang, X., Zhou, Y.: Collaborative multi-metadata fusion to improve the classification of lumbar disc herniation. IEEE Transactions on Medical Imaging 42(12), 3590–3601 (2023)

    2023

  14. [14]

    In: International conference on Machine learning

    Mao, A., Mohri, M., Zhong, Y.: Cross-entropy loss functions: Theoretical analysis and applications. In: International conference on Machine learning. pp. 23803– 23828. pmlr (2023)

    2023

  15. [15]

    OpenAI: Openai 5.1 system card.https://cdn.openai.com/pdf/ 4173ec8d-1229-47db-96de-06d87147e07e/5_1_system_card.pdf(2026), ac- cessed: 2026-02-23

    2026

  16. [16]

    Data in Brief32(2020)

    Pacheco, A.G.C., Lima, G.R., da Silva Salomão, A., Krohling, B., Biral, I.P., de Angelo, G.G., Jr, F.C.A., Esgario, J.G.M., Simora, A.C., Castro, P.B.C., Ro- drigues, F.B., Frasson, P.H.L., Krohling, R.A., Knidel, H., Santos, M.C.S., do Es- pírito Santo, R.B., Macedo, T.L., Canuto, T.R.P., de Barros, L.F.: Pad-ufes-20: A skin lesion dataset composed of pa...

    2020

  17. [17]

    IEEE journal of biomedical and health informatics25(9), 3554–3563 (2021)

    Pacheco, A.G., Krohling, R.A.: An attention-based mechanism to combine images and metadata in deep learning models applied to skin cancer classification. IEEE journal of biomedical and health informatics25(9), 3554–3563 (2021)

    2021

  18. [18]

    In: International conference on machine learning

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International conference on machine learning. pp. 8748–8763. PmLR (2021)

    2021

  19. [19]

    Medical Image Analysis99, 103357 (2025)

    Silva-Rodriguez, J., Chakor, H., Kobbi, R., Dolz, J., Ayed, I.B.: A foundation language-image model of the retina (flair): Encoding expert knowledge in text supervision. Medical Image Analysis99, 103357 (2025)

    2025

  20. [20]

    arXiv preprint arXiv:2508.10104 (2025)

    Siméoni, O., Vo, H.V., Seitzer, M., Baldassarre, F., Oquab, M., Jose, C., Khali- dov, V., Szafraniec, M., Yi, S., Ramamonjisoa, M., et al.: Dinov3. arXiv preprint arXiv:2508.10104 (2025)

    Pith/arXiv arXiv 2025

  21. [21]

    IEEE Transactions on Neural Networks and Learning Systems (2025)

    Wang, H., Guo, S., Ye, J., Deng, Z., Cheng, J., Li, T., Chen, J., Su, Y., Huang, Z., Shen, Y., et al.: Sam-med3d: a vision foundation model for general-purpose seg- mentation on volumetric medical images. IEEE Transactions on Neural Networks and Learning Systems (2025)

    2025

  22. [22]

    In: Proceedings of the 2022 Conference on Empir- ical Methods in Natural Language Processing

    Wang, Z., Wu, Z., Agarwal, D., Sun, J.: Medclip: Contrastive learning from un- paired medical images and text. In: Proceedings of the 2022 Conference on Empir- ical Methods in Natural Language Processing. pp. 3876–3887 (2022)

    2022

  23. [23]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Wu, C., Zhang, X., Zhang, Y., Wang, Y., Xie, W.: Medklip: Medical knowledge enhanced language-image pre-training for x-ray diagnosis. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 21372–21383 (2023)

    2023

  24. [24]

    arXiv preprint arXiv:2505.09388 (2025) PRIMA 11

    Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., Zheng, C., Liu, D., Zhou, F., Huang, F., Hu, F., Ge, H., Wei, H., Lin, H., Tang, J., Yang, J., Tu, J., Zhang, J., Yang, J., Yang, J., Zhou, J., Zhou, J., Lin, J., Dang, K., Bao, K., Yang, K., Yu, L., Deng, L., Li, M., Xue, M., Li, M., Zhang, P., Wang, P., Zhu, Q...

    Pith/arXiv arXiv 2025

  25. [25]

    Advances in neural information processing systems37, 90683– 90713 (2024)

    Yang, Y., Gandhi, M., Wang, Y., Wu, Y., Yao, M., Callison-Burch, C., Gee, J., Yatskar, M.: A textbook remedy for domain shifts: Knowledge priors for medical image analysis. Advances in neural information processing systems37, 90683– 90713 (2024)

    2024

This paper was first reviewed by deepseek-v4-flash on August 2, 2026.