Pith. sign in

REVIEW 5 major objections 6 minor 40 references

Reinforced Correlation Between Vision and Language for Precise Medical AI Assistant

T0 review · 5 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read One model, prompted by plain text, outperforms prior medical segmentation assistants in 165 of 177 tasks across nine imaging modalities.

desk verdict Big empirical bet on text-promptable medical segmentation; the CRD dataset trick is the real contribution, but the central 38.93-point claim is unauditable until code and weights ship. read the letter →

arxiv 2505.03380 v1 pith:BRKQOA43 submitted 2025-05-06 cs.CV cs.AIeess.IV

Haonan Wang , Jiaji Mao , Lehan Wang , Qixiang Zhang , Marawan Elbatel , Yi Qin , Huijun Hu , Baoxun Li
show 6 more authors
Wenhui Deng Weifeng Qin Hongrui Li Jialin Liang Jun Shen Xiaomeng Li
This is my paper · ORCID
classification cs.CVcs.AIeess.IV
keywords medicalimagesegmentationvision-languagemodeltext-promptablefoundationself-reinforcingcorrelationcolorregiondescriptioncancermultimodalAI
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a single text-promptable model can handle medical segmentation, localization, and classification across nine imaging modalities, because it is trained on 20 million image-mask-description triplets that teach it the shape and position of each structure rather than just its name. If the claim holds, clinicians and non-specialists could obtain precise lesion boundaries with natural-language prompts, without drawing bounding boxes or possessing radiology expertise. The reported evidence is a held-out evaluation of 835,081 samples across 177 tasks, where the model outperforms BiomedParse by an average of 38.93 Dice points and beats MedSAM's prompt-free mode by 70.86 points, plus external and in-house cancer datasets that were allegedly unseen during training.

What carries the argument

The load-bearing mechanism is a self-reinforcing vision-language correlation loop, aided by a Color Region Description (CRD) annotation strategy. CRD turns each segmentation mask into a set of colored regions and asks an off-the-shelf vision-language model to describe their shape and relative position, so the training text encodes spatial morphology instead of a bare class name. In the network, a vision-to-language projection carries image features into the Vicuna language model, while a language-to-vision projection carries conditioned embeddings back into the mask decoder; the <SEG> token triggers mask generation. This loop is what converts a text prompt like 'liver tumor' into a pixel-precise mask, and it is also what the paper credits for generalization to unseen diseases.

What would settle it

Inspect the component datasets inside SA-Med2D-20M and test whether AbdomenAtlas, DDTI, or the uwaterloo dermoscopy images (or their patient-level volumes) appear in the training corpus; if any are found, the external-validation Dice gains would no longer demonstrate generalization to unseen data.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that the weakness of previous medical AI assistants is a weak vision-language correlation, and that a closed-loop architecture trained on richly described masks fixes it. RCMed runs visual features through a language model and then feeds language-conditioned features back into a SAM-style mask decoder, so text semantics can steer pixel-level attention while image details sharpen the text. The Color Region Description strategy generates the needed supervision by converting masks into colored patches and asking a vision-language model to describe their shapes and relative positions, creating the 20-million-triplet RCMedData. This combination reportedly yields state-of-the-art Dice scores on 165 of 177 held-out tasks, a 23.5 percent relative gain on microscopy cell segmentation, and competitive results on external cancer segmentation, including classes the model never saw, which the authors attribute to learned knowledge of normal anatomy.

Load-bearing premise

The load-bearing premise is that the external public datasets used to prove generalization truly were absent from the 20-million-image training corpus; the paper asserts this but does not report any overlap analysis.

Editorial extensions

If this is right

  • Text-only prompting could replace box-and-click interaction for routine organ and lesion segmentation, lowering the expertise barrier for using medical AI.
  • If the external numbers hold, the model's learned normality model lets it flag anomalies it was never named, such as acoustic neuroma or ovarian cancer.
  • The CRD annotation pipeline converts any existing image-mask dataset into language-driven training data, so scaling to new modalities or tasks requires no manual caption writing.
  • One-shot training-free adaptation offers a route to new classes without retraining, which the paper identifies as still limited and subject to catastrophic forgetting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The large 38.93-point gap over BiomedParse may partly reflect the weakness of the BiomedParse baseline itself; against MedSAM with a tight ground-truth box, RCMed wins on prompt-free usability but often not on raw Dice, so the practical claim is best read as 'text prompting can approach box-guided accuracy with far less input effort.'
  • Because CRD descriptions are generated by an off-the-shelf vision-language model from synthetic colored masks, the RCMedData supervision inherits whatever shape-description errors that generator makes; measuring segmentation performance against description quality would reveal the ceiling of the whole pipeline.
  • A straightforward testable extension is to train the same architecture on the same 20 million triplets but with class-name-only prompts, which would isolate how much of the gain is CRD text versus the closed-loop network design.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. This manuscript proposes RCMed, a vision-language assistant for medical image analysis that performs text-prompted segmentation, classification, and localization. The authors construct a large training corpus (RCMedData) of 20 million image-mask-description triplets by applying a Color Region Description (CRD) strategy that uses the VLM InternVL-1.5 to describe the shapes and relative positions of colored anatomical masks. RCMed is built on a LLaVA-style architecture with a SAM encoder and Vicuna-7B LLM, trained end-to-end with a V-L and L-V projection loop. The authors report a held-out evaluation on 835,081 samples across 177 tasks in 9 modalities, claiming an average DSC improvement of 38.93 points over BiomedParse and superiority over MedSAM in prompt-free and loosely-prompted modes. They also report external validation on 33 datasets, including an in-house multi-cancer set from Chinese and Egyptian hospitals, with 20 cancer types, several of which are claimed to be unseen during training, plus a radiologist user study.

Significance. If the reported results are reproducible, RCMed would be a substantial advance: it presents a scalable way to generate language-driven medical segmentation datasets and demonstrates that a single text-promptable model can handle a wide range of clinical tasks across modalities. The CRD strategy is a practical contribution, and the in-house multinational cancer evaluation is a genuine attempt to assess generalization in a clinical setting. The held-out evaluation is large (835k samples) and includes per-task breakdowns. However, the lack of released code or weights, the absence of an overlap analysis between RCMedData and the public external datasets, and the absence of strong per-task supervised baselines currently prevent verification of the state-of-the-art claims.

major comments (5)
  1. [§4.3, §2.3, Fig. 2] The assertion in §4.3 that 'all datasets were held out and did not appear during model training' is unsubstantiated for the public external datasets because RCMedData is built directly from SA-Med2D-20M, which is itself an aggregation of public medical segmentation datasets. The manuscript does not list the component datasets of SA-Med2D-20M or perform an overlap analysis against AbdomenAtlas, DDTI, and uwaterloo. If any of these public external sets are contained in the training corpus, the external validation results in Fig. 2a do not demonstrate generalization. The authors must provide an explicit overlap analysis or a complete list of constituent datasets, and remove from the external evaluation any dataset that overlaps with training.
  2. [Data/Code Availability] The central quantitative claims—the 38.93-point average DSC improvement over BiomedParse and the 70.86-point improvement over MedSAM with no prompt—cannot be independently verified because the code, model weights, and a complete inference specification (exact text prompts for all 177 tasks, preprocessing steps, and the CRD description generation protocol) are not provided. The paper states that code and weights will be released 'upon publication,' but no demo or repository link is active for review. Without an executable implementation, the reported numbers are not auditable. The authors should release the code and weights, or provide a sufficiently detailed experimental protocol to allow replication, before final acceptance.
  3. [§2.2, Table 1] The comparison with BiomedParse may not be fair because the prompt format for BiomedParse is not specified. The paper states that category names are used as text prompts for both RCMed and BiomedParse, but BiomedParse is a foundation model trained with specific prompt templates (e.g., 'Segment <class> in the image'). In Table 1, BiomedParse achieves exactly 0.00 DSC on numerous tasks (e.g., adrenal gland left, brainstem, gluteus maximus left, all rib left/right entries). A zero DSC across many tasks suggests that BiomedParse may be failing to parse the prompt or return an empty mask, rather than producing an incorrect but nonempty segmentation. The authors should report the exact prompt format for each baseline and discuss whether zero-DSC cases arise from prompt mismatch; if so, those tasks should be excluded or the baseline should be re-run with its recommended prompt.
  4. [Abstract, §2.2] The claim of 'state-of-the-art' performance is not supported by comparison with strong per-task supervised baselines such as nnU-Net (ref. [18]), which is the standard benchmark for medical image segmentation. The paper compares only with foundation models (BiomedParse, MedSAM) and a few classification/localization models. If the claim is intended to mean 'state-of-the-art among text-promptable foundation models,' that should be stated explicitly. Otherwise, the authors should add per-task comparisons with nnU-Net or equivalent supervised methods on a representative subset of the 177 tasks to justify the unqualified 'state-of-the-art' wording.
  5. [§2.1 vs §4.3] The reported evaluation split is internally inconsistent. Section 2.1 states 'We held out 20% of the RCMedData data to comprehensively evaluate the model's performance,' while Section 4.3 states the data was 'randomly split into 80%, 10%, and 10% as training, tuning, and validation.' Neither percentage matches the reported test size of 835,081 samples, which is about 4.2% of 20 million. The authors must clarify the exact split procedure, the number of samples in each split, and why the validation set used for comparisons differs from the stated split. This inconsistency undermines the precision of the central held-out evaluation.
minor comments (6)
  1. [§4.4] The sentence 'The model undergoes end-to-end training for 5 iterations' is almost certainly a typo and should read '5 epochs'; otherwise the model would not converge on 20 million samples.
  2. [Table 1] The row 'clavicula right' appears twice with identical values, and the table header says 'Dice Coefficient Similarity' instead of 'Dice Similarity Coefficient.' Please clean the table and correct the metric name.
  3. [§2.1] The caveat 'some of our held-out data is also involved in training MedSAM' is important and should appear earlier, ideally in the Results overview or a dedicated note, so that readers understand the MedSAM comparison on the held-out set is partially confounded.
  4. [Data Availability] The Data Availability section mentions 'in-house datasets from Guangdong Provincial People’s Hospital (GDPH),' but Section 4.1 states the data were collected from Sun Yat-sen Memorial Hospital, Sun Yat-sen University. Please clarify the source institution and the relationship between the two names.
  5. [Abstract] The abstract claims 'a 23.5% relative improvement in cell segmentation from microscopy images over prior art,' but the prior art is never named and no dataset or metric is specified. Please provide a precise comparison with the baseline and cite the source.
  6. [References] References [7] and [28] appear to refer to the same paper (BiomedParse by Zhao et al.). Please merge or differentiate them.

Circularity Check

1 steps flagged · score 4.0 of 10

External 'unseen-class' generalization is one-shot-adapted with a test-set mask, but the main held-out benchmark is independent.

  1. fitted input called prediction [Section 4.2, 'One-shot Training-free New Class Adaptation'; used in Section 2.3 external validation]
    "To enhance performance on unseen classes, we developed a one-shot, training-free adaptation strategy, illustrated in Fig. 4c. This approach operates during inference and consists of two key stages: one-shot information registration and adaptation. In the first stage, the model processes a sample—in this case, image-mask pairs from the unseen class—to register semantic and spatial information."

    Section 2.3 presents the 20-cancer external results as evidence of generalization to 'unseen classes' ('most of the cancers are unseen classes during training ... our RCMed surprisingly demonstrated significant improvements'). But the model's inference pipeline for new classes explicitly takes an image-mask pair from that unseen class and registers its semantics and spatial prior before segmenting the test images. The reported 'unseen-class' scores therefore are not zero-shot predictions; they are one-shot adapted outputs conditioned on a ground-truth label drawn from the evaluation distribution. The external generalization claim reduces, by the paper's own architecture, to few-shot in-context conditioning on a test-set mask.

full rationale

The paper's central held-out claim is empirical and not mathematically circular: RCMed is evaluated on a random 20% split of RCMedData, and the 38.93 DSC improvement over BiomedParse is computed from masks and images not used to fit the model. The CRD descriptions are generated from training masks and used as training targets/conditioning; they are not test-time predictions, so that pipeline is not circular. The principal circularity-adjacent issue is the one-shot training-free adaptation: for the external unseen-cancer evaluation, the method supplies a ground-truth mask of the target class at inference, then reports 'unseen class' generalization. That specific prediction is partly constructed from a label in the evaluation set, which is the 'fitted input called prediction' pattern. The paper does disclose the one-shot mechanism in the methods, but the abstract and Section 2.3 frame the results as unqualified external generalization. The reader's additional concern about public external datasets (AbdomenAtlas, DDTI, uwaterloo) being contained in SA-Med2D-20M is a plausible verification risk: Section 4.3 asserts 'all datasets were held out' without an overlap analysis, and the training corpus is built from SA-Med2D-20M. However, the manuscript itself does not exhibit that these specific public sets are in the training corpus, so under the no-speculation rule this is noted as a reproducibility/audit concern rather than a demonstrated circular step. No load-bearing self-citation chain or imported uniqueness theorem is present. Overall score 4 reflects one partial circularity in the external unseen-class evaluation while the core held-out benchmark retains independent content.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim depends on a small number of modeling choices and one critical dataset-disjointness assumption. The architecture is a standard composition of Vicuna, SAM, and MLP projections; the closed loop is implemented by two trainable projection layers rather than a formal mechanism. The CRD strategy assumes VLM-generated color descriptions are reliable supervision. The most load-bearing unverified premise is that public external benchmarks are absent from the SA-Med2D-20M training set.

free parameters (4)
  • Training epochs = 5
    Chosen by hand for end-to-end training; could affect convergence but is not central to the conceptual mechanism.
  • Initial learning rate = 1e-2
    Standard choice with polynomial decay; tuned on the validation split.
  • LoRA alpha = 8
    Set for the LLM fine-tuning; a standard value that does not change the core claim.
  • CRD color palette size = 7 predefined colors
    The number of distinct colors used to convert masks into color patches; this choice is arbitrary and could affect description quality.
assumptions (5)
  • domain assumption Pretrained Vicuna-7B and SAM-H provide sufficient inductive bias for medical vision-language alignment
    The model is initialized from natural-image pretrained weights and fine-tuned on 20M medical triplets; there is no evidence that this initialization is optimal or that domain shift is fully overcome.
  • ad hoc to paper Off-the-shelf VLMs can reliably describe colored mask patches as shape and relative-position text
    The CRD strategy assumes InternVL-1.5 outputs on synthetic colored masks are accurate enough to serve as supervision; the paper shows examples but no systematic quality evaluation.
  • ad hoc to paper The public external datasets are disjoint from the SA-Med2D-20M training corpus
    External validation at Sec 2.3 depends on this disjointness; it is asserted but never verified with an overlap analysis.
  • domain assumption Anatomical consistency of the human body allows Gaussian-prior centroids in one-shot adaptation to transfer across patients
    Location registration in the one-shot module uses a Gaussian centered at the example mask's centroid, assuming cross-patient anatomical alignment.
  • domain assumption DSC as the sole primary metric is an adequate measure of clinical segmentation quality
    No clinical acceptability thresholds, volume errors, or surface distances are used in the evaluation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Reinforced Correlation Between Vision and Language for Precise Medical AI Assistant." pith.science (2026). https://pith.science/paper/BRKQOA43

@misc{pith2026250503380,
  author       = {Pith},
  title        = {Pith review of: Reinforced Correlation Between Vision and Language for Precise Medical AI Assistant},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BRKQOA43}},
  note         = {Machine review of arXiv:2505.03380}
}
read the original abstract

Medical AI assistants support doctors in disease diagnosis, medical image analysis, and report generation. However, they still face significant challenges in clinical use, including limited accuracy with multimodal content and insufficient validation in real-world settings. We propose RCMed, a full-stack AI assistant that improves multimodal alignment in both input and output, enabling precise anatomical delineation, accurate localization, and reliable diagnosis through hierarchical vision-language grounding. A self-reinforcing correlation mechanism allows visual features to inform language context, while language semantics guide pixel-wise attention, forming a closed loop that refines both modalities. This correlation is enhanced by a color region description strategy, translating anatomical structures into semantically rich text to learn shape-location-text relationships across scales. Trained on 20 million image-mask-description triplets, RCMed achieves state-of-the-art precision in contextualizing irregular lesions and subtle anatomical boundaries, excelling in 165 clinical tasks across 9 modalities. It achieved a 23.5% relative improvement in cell segmentation from microscopy images over prior methods. RCMed's strong vision-language alignment enables exceptional generalization, with state-of-the-art performance in external validation across 20 clinically significant cancer types, including novel tasks. This work demonstrates how integrated multimodal models capture fine-grained patterns, enabling human-level interpretation in complex scenarios and advancing human-centric AI healthcare.

Figures

Figures reproduced from arXiv: 2505.03380 by the authors.

Figure 3
Figure 3. Human-centric evaluation. a, In a traditional clinician’s workflow, different diseases re￾quire various image modalities for analysis, leading to a need for specialized expertise in different diseases or modalities. Additionally, the time required to perform segmentation is quite significant. b, In contrast, an AI-assisted workflow using our language-driven segmentation foundation model, RCMed, can perform segmentat… view at source ↗
Figure 4
Figure 4. a, The pipeline of Color Region Describing (CRD) strategy, which can theoretically con￾vert any segmentation dataset into a language-driven format. b, technical detail of RCMed. c, pipeline of the one-shot training-free new class adaptation module. 4 Methodologies 4.1 Dataset Curation Color Region Description Annotating Strategy. A large number of medical image segmentation datasets exist with image-mask pairs. Howe… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 35 canonical work pages

  1. [18]

    nnu-net: a self-configuring method for deep learning-based biomedical image segmentation,

    F. Isensee, P. F. Jaeger, S. A. Kohl, J. Petersen, and K. H. Maier-Hein, “nnu-net: a self-configuring method for deep learning-based biomedical image segmentation,” Nature methods, vol. 18, no. 2, pp. 203–211, 2021

    2021

  2. [1]

    A visual–language foundation model for pathology image analysis using medical twitter,

    Z. Huang, F. Bianchi, M. Yuksekgonul, T. J. Montine, and J. Zou, “A visual–language foundation model for pathology image analysis using medical twitter,” Nature medicine, vol. 29, no. 9, pp. 2307–2316, 2023

    2023

  3. [2]

    A visual-language foundation model for computational pathology,

    M. Y . Lu, B. Chen, D. F. Williamson, R. J. Chen, I. Liang, T. Ding, G. Jaume, I. Odintsov, L. P. Le, G. Gerber, et al., “A visual-language foundation model for computational pathology,” Nature Medicine, vol. 30, no. 3, pp. 863–874, 2024

    work page 2024

  4. [3]

    Vision–language foundation model for echocardiogram interpretation,

    M. Christensen, M. Vukadinovic, N. Yuan, and D. Ouyang, “Vision–language foundation model for echocardiogram interpretation,” Nature Medicine, pp. 1–8, 2024

    work page 2024

  5. [4]

    A foundation model of transcription across human cell types,

    X. Fu, S. Mo, A. Buendia, A. P. Laurent, A. Shao, M. d. M. Alvarez-Torres, T. Yu, J. Tan, J. Su, R. Sagatelian, et al., “A foundation model of transcription across human cell types,” Nature, pp. 1–9, 2025

    work page 2025

  6. [5]

    Accurate predictions on small data with a tabular foundation model,

    N. Hollmann, S. Müller, L. Purucker, A. Krishnakumar, M. Körfer, S. B. Hoo, R. T. Schirrmeister, and F. Hutter, “Accurate predictions on small data with a tabular foundation model,” Nature, vol. 637, no. 8045, pp. 319–326, 2025

    work page 2025

  7. [6]

    A vision–language foundation model for precision oncology,

    J. Xiang, X. Wang, X. Zhang, Y . Xi, F. Eweje, Y . Chen, Y . Li, C. Bergstrom, M. Gopaulchan, T. Kim, et al., “A vision–language foundation model for precision oncology,”Nature, pp. 1–10, 2025

    work page 2025

  8. [7]

    A foundation model for joint segmentation, detection and recognition of biomedical objects across nine modalities,

    T. Zhao, Y . Gu, J. Yang, N. Usuyama, H. H. Lee, S. Kiblawi, T. Naumann, J. Gao, A. Crabtree, J. Abel, et al., “A foundation model for joint segmentation, detection and recognition of biomedical objects across nine modalities,” Nature Methods, pp. 1–11, 2024

    work page 2024

Show all 40 references
  1. [8]

    A vision–language foundation model for the generation of realistic chest x-ray images,

    C. Bluethgen, P. Chambon, J.-B. Delbrouck, R. van der Sluijs, M. Połacin, J. M. Zambrano Chaves, T. M. Abraham, S. Purohit, C. P. Langlotz, and A. S. Chaudhari, “A vision–language foundation model for the generation of realistic chest x-ray images,” Nature Biomedical Engineeri...

    2024

  2. [9]

    A foundation model for enhancing magnetic resonance images and downstream segmentation, registration and diagnostic tasks,

    Y . Sun, L. Wang, G. Li, W. Lin, and L. Wang, “A foundation model for enhancing magnetic resonance images and downstream segmentation, registration and diagnostic tasks,”Nature Biomedical Engineering, pp. 1–18, 2024

    2024

  3. [10]

    A generalist vision–language foundation model for diverse biomedical tasks,

    K. Zhang, R. Zhou, E. Adhikarla, Z. Yan, Y . Liu, J. Yu, Z. Liu, X. Chen, B. D. Davison, H. Ren,et al., “A generalist vision–language foundation model for diverse biomedical tasks,” Nature Medicine, pp. 1–13, 2024

    2024

  4. [11]

    scgpt: toward building a foundation model for single-cell multi-omics using generative ai,

    H. Cui, C. Wang, H. Maan, K. Pang, F. Luo, N. Duan, and B. Wang, “scgpt: toward building a foundation model for single-cell multi-omics using generative ai,” Nature Methods, vol. 21, no. 8, pp. 1470–1480, 2024

    2024

  5. [12]

    The multimodality cell segmentation challenge: toward universal solutions,

    J. Ma, R. Xie, S. Ayyadhury, C. Ge, A. Gupta, R. Gupta, S. Gu, Y . Zhang, G. Lee, J. Kim, et al., “The multimodality cell segmentation challenge: toward universal solutions,” Nature methods, vol. 21, no. 6, pp. 1103–1113, 2024

    2024

  6. [13]

    Towards foundation models of biological image segmentation,

    J. Ma and B. Wang, “Towards foundation models of biological image segmentation,” Nature Methods, vol. 20, no. 7, pp. 953–955, 2023

    2023

  7. [14]

    Foundation model for cancer imaging biomarkers,

    S. Pai, D. Bontempi, I. Hadzic, V . Prudente, M. Sokaˇc, T. L. Chaunzwa, S. Bernatz, A. Hosny, R. H. Mak, N. J. Birkbak, et al., “Foundation model for cancer imaging biomarkers,” Nature Machine Intelligence, vol. 6, no. 3, pp. 354–367, 2024

    2024

  8. [15]

    Exploring scalable medical image encoders beyond text supervision,

    F. Pérez-García, H. Sharma, S. Bond-Taylor, K. Bouzid, V . Salvatelli, M. Ilse, S. Bannur, D. C. Cas- tro, A. Schwaighofer, M. P. Lungren, et al., “Exploring scalable medical image encoders beyond text supervision,” Nature Machine Intelligence, pp. 1–12, 2025

    2025

  9. [16]

    U-net: Convolutional networks for biomedical image seg- mentation,

    O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image seg- mentation,” in International conference on medical image computing and computer-assisted intervention, pp. 234–241, Springer, 2015

    2015

  10. [17]

    H-denseunet: hybrid densely connected unet for liver and tumor segmentation from ct volumes,

    X. Li, H. Chen, X. Qi, Q. Dou, C.-W. Fu, and P.-A. Heng, “H-denseunet: hybrid densely connected unet for liver and tumor segmentation from ct volumes,” IEEE transactions on medical imaging, vol. 37, no. 12, pp. 2663–2674, 2018

    2018

  11. [19]

    Swin-unet: Unet-like pure transformer for medical image segmentation,

    H. Cao, Y . Wang, J. Chen, D. Jiang, X. Zhang, Q. Tian, and M. Wang, “Swin-unet: Unet-like pure transformer for medical image segmentation,” in European conference on computer vision, pp. 205–218, Springer, 2022. 18

    2022

  12. [20]

    Uctransnet: rethinking the skip connections in u-net from a channel-wise perspective with transformer,

    H. Wang, P. Cao, J. Wang, and O. R. Zaiane, “Uctransnet: rethinking the skip connections in u-net from a channel-wise perspective with transformer,” in Proceedings of the AAAI conference on artificial intelligence, vol. 36, pp. 2441–2449, 2022

    2022

  13. [21]

    Towards generic semi-supervised framework for volumetric medical image segmen- tation,

    H. Wang and X. Li, “Towards generic semi-supervised framework for volumetric medical image segmen- tation,” in Advances in Neural Information Processing Systems, vol. 36, 2024

    2024

  14. [22]

    Dhc: Dual-debiased heterogeneous co-training framework for class-imbalanced semi-supervised medical image segmentation,

    H. Wang and X. Li, “Dhc: Dual-debiased heterogeneous co-training framework for class-imbalanced semi-supervised medical image segmentation,” in International conference on medical image computing and computer-assisted intervention, pp. 582–591, Springer, 2023

    2023

  15. [23]

    Scs: cell segmentation for high-resolution spatial transcriptomics,

    H. Chen, D. Li, and Z. Bar-Joseph, “Scs: cell segmentation for high-resolution spatial transcriptomics,” Nature methods, vol. 20, no. 8, pp. 1237–1243, 2023

    2023

  16. [24]

    Omnipose: a high-precision morphology-independent solution for bacterial cell segmenta- tion,

    K. J. Cutler, C. Stringer, T. W. Lo, L. Rappez, N. Stroustrup, S. Brook Peterson, P. A. Wiggins, and J. D. Mougous, “Omnipose: a high-precision morphology-independent solution for bacterial cell segmenta- tion,” Nature methods, vol. 19, no. 11, pp. 1438–1448, 2022

    2022

  17. [25]

    Cellotype: a unified model for segmentation and classification of tissue images,

    M. Pang, T. K. Roy, X. Wu, and K. Tan, “Cellotype: a unified model for segmentation and classification of tissue images,” Nature methods, pp. 1–10, 2024

    2024

  18. [26]

    Uncertainty-guided dual-views for semi- supervised volumetric medical image segmentation,

    H. Peiris, M. Hayat, Z. Chen, G. Egan, and M. Harandi, “Uncertainty-guided dual-views for semi- supervised volumetric medical image segmentation,”Nature Machine Intelligence, vol. 5, no. 7, pp. 724– 738, 2023

    2023

  19. [27]

    Physics-based machine learning for subcellular segmentation in living cells,

    A. A. Sekh, I. S. Opstad, G. Godtliebsen, Å. B. Birgisdottir, B. S. Ahluwalia, K. Agarwal, and D. K. Prasad, “Physics-based machine learning for subcellular segmentation in living cells,” Nature Machine Intelligence, vol. 3, no. 12, pp. 1071–1080, 2021

    2021

  20. [28]

    Biomedparse: a biomedical foundation model for image parsing of everything everywhere all at once,

    T. Zhao, Y . Gu, J. Yang, N. Usuyama, H. H. Lee, T. Naumann, J. Gao, A. Crabtree, J. Abel, C. Moung- Wen, et al., “Biomedparse: a biomedical foundation model for image parsing of everything everywhere all at once,” Nature Methods, 2024

    2024

  21. [29]

    Visual instruction tuning,

    H. Liu, C. Li, Q. Wu, and Y . J. Lee, “Visual instruction tuning,”Advances in neural information processing systems, vol. 36, pp. 34892–34916, 2023

    2023

  22. [30]

    Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality,

    W.-L. Chiang, Z. Li, Z. Lin, Y . Sheng, Z. Wu, H. Zhang, L. Zheng, S. Zhuang, Y . Zhuang, J. E. Gonzalez, et al., “Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality,”See https://vicuna. lmsys. org (accessed 14 April 2023), vol. 2, no. 3, p. 6, 2023

    2023

  23. [31]

    Segment anything,

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.- Y . Lo,et al., “Segment anything,” inProceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4015–4026, 2023

    2023

  24. [32]

    Sam-med2d,

    J. Cheng, J. Ye, Z. Deng, J. Chen, T. Li, H. Wang, Y . Su, Z. Huang, J. Chen, L. Jiang,et al., “Sam-med2d,” arXiv preprint arXiv:2308.16184, 2023

    2023 arXiv

  25. [33]

    Llava- med: Training a large language-and-vision assistant for biomedicine in one day,

    C. Li, C. Wong, S. Zhang, N. Usuyama, H. Liu, J. Yang, T. Naumann, H. Poon, and J. Gao, “Llava- med: Training a large language-and-vision assistant for biomedicine in one day,” Advances in Neural Information Processing Systems, vol. 36, 2024

    2024

  26. [34]

    Interpretable bilingual multimodal large language model for diverse biomedical tasks,

    L. Wang, H. Wang, H. Yang, J. Mao, Z. Yang, J. Shen, and X. Li, “Interpretable bilingual multimodal large language model for diverse biomedical tasks,”The Thirteenth International Conference on Learning Representations (ICLR), 2025

    2025

  27. [35]

    Medical phrase grounding with region-phrase context contrastive alignment,

    Z. Chen, Y . Zhou, A. Tran, J. Zhao, L. Wan, G. S. K. Ooi, L. T.-E. Cheng, C. H. Thng, X. Xu, Y . Liu, et al. , “Medical phrase grounding with region-phrase context contrastive alignment,” in International Conference on Medical Image Computing and Computer-Assisted Interventio...

    2023

  28. [36]

    Segment anything in medical images,

    J. Ma, Y . He, F. Li, L. Han, C. You, and B. Wang, “Segment anything in medical images,” Nature Com- munications, vol. 15, no. 1, p. 654, 2024

    2024

  29. [37]

    The medical segmentation decathlon,

    M. Antonelli, A. Reinke, S. Bakas, K. Farahani, A. Kopp-Schneider, B. A. Landman, G. Litjens, B. Menze, O. Ronneberger, R. M. Summers, et al., “The medical segmentation decathlon,” Nature com- munications, vol. 13, no. 1, p. 4128, 2022

    2022

  30. [38]

    Multimodal generative ai for medical image interpretation,

    V . M. Rao, M. Hla, M. Moor, S. Adithan, S. Kwak, E. J. Topol, and P. Rajpurkar, “Multimodal generative ai for medical image interpretation,” Nature, vol. 639, no. 8056, pp. 888–896, 2025. 19

    2025

  31. [39]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al., “Learning transferable visual models from natural language supervision,” inInternational conference on machine learning, pp. 8748–8763, PMLR, 2021

    2021

  32. [40]

    Glamm: Pixel grounding large multimodal model,

    H. Rasheed, M. Maaz, S. Shaji, A. Shaker, S. Khan, H. Cholakkal, R. M. Anwer, E. Xing, M.-H. Yang, and F. S. Khan, “Glamm: Pixel grounding large multimodal model,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13009–13018, 2024. 20 B...

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.