{"work":{"id":"6fcf8750-00b8-4f3e-9f0b-965a879a5dff","openalex_id":null,"doi":null,"arxiv_id":"2303.00915","raw_key":null,"title":"BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs","authors":null,"authors_text":"Sheng Zhang, Yanbo Xu, Naoto Usuyama, Hanwen Xu, Jaspreet Bagga, Robert Tinn","year":2023,"venue":"cs.CV","abstract":"Biomedical data is inherently multimodal, comprising physical measurements and natural language narratives. A generalist biomedical AI model needs to simultaneously process different modalities of data, including text and images. Therefore, training an effective generalist biomedical model requires high-quality multimodal data, such as parallel image-text pairs. Here, we present PMC-15M, a novel dataset that is two orders of magnitude larger than existing biomedical multimodal datasets such as MIMIC-CXR, and spans a diverse range of biomedical image types. PMC-15M contains 15 million biomedical image-text pairs collected from 4.4 million scientific articles. Based on PMC-15M, we have pretrained BiomedCLIP, a multimodal foundation model, with domain-specific adaptations tailored to biomedical vision-language processing. We conducted extensive experiments and ablation studies on standard biomedical imaging tasks from retrieval to classification to visual question-answering (VQA). BiomedCLIP achieved new state-of-the-art results in a wide range of standard datasets, substantially outperforming prior approaches. Intriguingly, by large-scale pretraining on diverse biomedical image types, BiomedCLIP even outperforms state-of-the-art radiology-specific models such as BioViL in radiology-specific tasks such as RSNA pneumonia detection. In summary, BiomedCLIP is a fully open-access foundation model that achieves state-of-the-art performance on various biomedical tasks, paving the way for transformative multimodal biomedical discovery and applications. We release our models at https://aka.ms/biomedclip to facilitate future research in multimodal biomedical AI.","external_url":"https://arxiv.org/abs/2303.00915","cited_by_count":null,"metadata_source":"pith","metadata_fetched_at":"2026-05-25T06:20:24.426722+00:00","pith_arxiv_id":"2303.00915","created_at":"2026-05-10T00:29:46.976519+00:00","updated_at":"2026-06-05T21:23:00.469572+00:00","title_quality_ok":true,"display_title":"BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs","render_title":"BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs"},"hub":{"state":{"work_id":"6fcf8750-00b8-4f3e-9f0b-965a879a5dff","tier":"hub","tier_reason":"10+ Pith inbound or 1,000+ external citations","pith_inbound_count":47,"external_cited_by_count":null,"distinct_field_count":5,"first_pith_cited_at":"2023-05-17T17:50:16+00:00","last_pith_cited_at":"2026-05-21T15:02:44+00:00","author_build_status":"not_needed","summary_status":"needed","contexts_status":"needed","graph_status":"needed","ask_index_status":"not_needed","reader_status":"not_needed","recognition_status":"not_needed","updated_at":"2026-06-10T16:16:44.026614+00:00","tier_text":"hub"},"tier":"hub","role_counts":[{"context_role":"background","n":7},{"context_role":"method","n":2},{"context_role":"baseline","n":1}],"polarity_counts":[{"context_polarity":"background","n":7},{"context_polarity":"use_method","n":2},{"context_polarity":"baseline","n":1}],"runs":{},"summary":{},"graph":{},"authors":[]}}