Pith. sign in

REVIEW 11 cited by

BiomedParse: a biomedical foundation model for image parsing of everything everywhere all at once

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2405.12971 v3 pith:TGST6MZG submitted 2024-05-21 cs.CV

Theodore Zhao , Yu Gu , Jianwei Yang , Naoto Usuyama , Ho Hin Lee , Tristan Naumann , Jianfeng Gao , Angela Crabtree
show 7 more authors
Jacob Abel Christine Moung-Wen Brian Piening Carlo Bifulco Mu Wei Hoifung Poon Sheng Wang
This is my paper
classification cs.CV
keywords biomedicalimagebiomedparseobjectobjectssegmentationdetectionrecognition
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Biomedical image analysis is fundamental for biomedical discovery in cell biology, pathology, radiology, and many other biomedical domains. Holistic image analysis comprises interdependent subtasks such as segmentation, detection, and recognition of relevant objects. Here, we propose BiomedParse, a biomedical foundation model for imaging parsing that can jointly conduct segmentation, detection, and recognition for 82 object types across 9 imaging modalities. Through joint learning, we can improve accuracy for individual tasks and enable novel applications such as segmenting all relevant objects in an image through a text prompt, rather than requiring users to laboriously specify the bounding box for each object. We leveraged readily available natural-language labels or descriptions accompanying those datasets and use GPT-4 to harmonize the noisy, unstructured text information with established biomedical object ontologies. We created a large dataset comprising over six million triples of image, segmentation mask, and textual description. On image segmentation, we showed that BiomedParse is broadly applicable, outperforming state-of-the-art methods on 102,855 test image-mask-label triples across 9 imaging modalities (everything). On object detection, which aims to locate a specific object of interest, BiomedParse again attained state-of-the-art performance, especially on objects with irregular shapes (everywhere). On object recognition, which aims to identify all objects in a given image along with their semantic types, we showed that BiomedParse can simultaneously segment and label all biomedical objects in an image (all at once). In summary, BiomedParse is an all-in-one tool for biomedical image analysis by jointly solving segmentation, detection, and recognition for all major biomedical image modalities, paving the path for efficient and accurate image-based biomedical discovery.

Discussion (0). Sign in to comment.

Forward citations

Cited by 11 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. VERITAS: A Multi-Agent Co-Scientist for Verifiable Image-Derived Hypothesis Testing

    cs.MA 2026-04 unverdicted novelty 7.0 of 10

    VERITAS is a multi-agent system for verifiable hypothesis testing on multimodal clinical MRI datasets that achieves 81.4% verdict accuracy with frontier models and introduces an epistemic evidence labeling framework.

  2. IBISAgent: Reinforcing Pixel-Level Visual Reasoning in MLLMs for Universal Biomedical Object Referring and Segmentation

    cs.CV 2026-01 conditional novelty 7.0 of 10

    IBISAgent enables MLLMs to perform iterative pixel-level visual reasoning for biomedical object referring and segmentation via text-based clicks and agentic RL, outperforming prior SOTA methods without model modifications.

  3. Open-Ended CT Volume Segmentation with Weak Supervision from Language

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A text-promptable CT segmenter trained with weak slice labels mined from radiology reports beats strong-only training by 8–22% relative dice, with larger gains when expert masks are scarce.

  4. VERITAS: A Multi-Agent Co-Scientist for Verifiable Image-Derived Hypothesis Testing

    cs.MA 2026-04 conditional novelty 6.0 of 10

    A four-phase multi-agent co-scientist tests natural-language hypotheses on cardiac and glioma MRI and labels outcomes Supported, Refuted, Underpowered, or Invalid with an executable evidence trail.

  5. SciVid: Cross-Domain Evaluation of Video Models in Scientific Applications

    cs.CV 2025-07 conditional novelty 6.0 of 10

    General-purpose video foundation models, adapted with lightweight readout heads, reach state-of-the-art performance on three of five scientific video benchmarks.

  6. UltraSAM3: A Concept-Driven Foundation Model for Universal Ultrasound Image Segmentation

    cs.CV 2026-07 conditional novelty 5.0 of 10

    Adapting SAM3 to ultrasound with 171k image–mask–concept pairs gives a text-promptable multi-organ segmenter that beats general medical concept-segmentation baselines on average, though not on every organ.

  7. APRIL-MedSeg: A Modular Medical Image Segmentation Toolbox Embracing Modern Paradigms

    cs.CV 2026-06 unverdicted novelty 5.0 of 10

    APRIL-MedSeg is a new open-source modular toolbox that uses YAML configuration and component registries to unify multiple advanced paradigms for medical image segmentation.

  8. ReportMedSAM: Guiding Segmentation Through Radiology Reports

    cs.CL 2026-05 conditional novelty 5.0 of 10

    ReportMedSAM learns a bank of organ concepts in frozen BiomedCLIP space and uses report-to-concept similarity to route segmentation experts for four abdominal organs.

  9. LesiOnTime -- Joint Temporal and Clinical Modeling for Small Breast Lesion Segmentation in Longitudinal DCE-MRI

    cs.CV 2025-08 unverdicted novelty 5.0 of 10

    LesiOnTime segments small breast lesions in longitudinal DCE-MRI using temporal attention and BI-RADS consistency regularization, reporting a 5% Dice gain over baselines.

  10. APRIL-MedSeg: A Modular Medical Image Segmentation Toolbox Embracing Modern Paradigms

    cs.CV 2026-06 unverdicted novelty 4.0 of 10

    Presents APRIL-MedSeg, a modular YAML-configurable toolbox for 2D medical image segmentation integrating semi-supervised, domain adaptation, distillation, weakly supervised, text-guided, and foundation model paradigms...

  11. UNICON: UNIfied CONtinual Learning for Medical Foundational Models

    eess.IV 2025-08 unverdicted novelty 4.0 of 10

    UNICON attaches task-specific adapters (LoRA, MLP, decoder, fusion) to a frozen CT foundation model, enabling continual extension to prognosis, segmentation, and PET scans.

Pith tools