Pith. sign in

REVIEW 4 major objections 6 minor 51 references

ViLa-MIL: Dual-scale Vision-Language Multiple Instance Learning for Whole Slide Image Classification

T0 review · 4 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LLM-written dual-scale pathology descriptions let a frozen vision-language model beat few-shot whole-slide classifiers.

desk verdict A credible few-shot WSI method with a neat dual-scale LLM-prompt idea, but the missing TOP baseline keeps the state-of-the-art claim unsubstantiated. read the letter →

arxiv 2502.08391 v1 pith:K32DELWV submitted 2025-02-12 cs.CV

Jiangbo Shi , Chen Li , Tieliang Gong , Yefeng Zheng , Huazhu Fu This is my paper · ORCID
classification cs.CV
keywords wholeslideimageclassificationmultipleinstancelearningvision-languagemodelCLIPfew-shottextpromptprototype-guidedaggregationdigitalpathology
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ViLa-MIL proposes that a vision-language model can classify whole-slide images under extreme data scarcity if the text prompt carries the diagnostic reasoning pathologists actually use. The paper constructs dual-scale visual descriptive prompts that describe each cancer subtype at low resolution in terms of tissue architecture and at high resolution in terms of cellular detail, generated by a frozen large language model and prefixed with learnable vectors. These prompts guide a frozen CLIP backbone, while a prototype-guided patch decoder aggregates gigapixel patches into slide-level features and a context-guided text decoder refines the text features with image context. On three multi-cancer, multi-center subtyping datasets with only 16 labeled slides per class, the paper reports gains of 1.7-7.2% in AUC and 2.1-7.3% in F1 over state-of-the-art MIL baselines. If correct, this would mean pathology language priors can be injected without collecting millions of image-text pairs or retraining a foundation model.

What carries the argument

The mechanism has three interacting pieces. First, the dual-scale visual descriptive text prompt: a frozen large language model is asked to describe each class at low and high resolution, yielding two text strings per class, one about architecture-level morphology for low magnification and one about cytologic detail for high magnification, with $M$ learnable context vectors prepended to each string. Second, the prototype-guided patch decoder: a set of learnable prototype vectors serves as queries in a cross-attention layer over patch features, $\mathrm{Softmax}(QK^\top/\sqrt{d})V$ with a residual connection and layer normalization, so patches sharing semantics are grouped into the same prototype, and an attention pooling over the prototypes produces the slide-level feature. Third, the context-guided text decoder: the text features are used as queries in another cross-attention layer whose keys and values are the concatenation of local patch features and global prototype features, pulling image context into the language features. Training uses cross-entropy on the sum of per-scale similarity logits, and inference is the same forward pass. The decoders are lightweight and trainable while the vision-language encoders and the language model stay frozen, which is what makes the transfer parameter-efficient.

What would settle it

Retrain ViLa-MIL with the generated descriptive prompts replaced by class-name-replacement templates while keeping all decoders, data, and training identical; if the AUC margin over the best MIL baseline does not shrink substantially, the descriptive prompt is not the operative component. A complementary check is to reuse the same frozen prompts on slides from a hospital not in the training set and watch whether the cross-center AUC advantage persists.

Watch

Extended reading notes

Core claim

The paper's central claim is that the weak link in vision-language whole-slide classification is the text prompt, not the encoder: replacing class-name templates with dual-scale descriptive texts that encode routine pathological diagnostic reasoning unlocks the few-shot regime. On three multi-cancer, multi-center subtyping tasks with 16 labeled slides per class, the proposed framework reports state-of-the-art results, improving AUC by 1.7-7.2% and F1 by 2.1-7.3% over existing MIL methods, and holding a 5.5% AUC advantage in cross-center transfer between two renal cell carcinoma datasets. The discovery, stated on the paper's own terms, is that a frozen LLM can write the pathology knowledge into the prompt, and two lightweight decoders can adapt a frozen vision-language model to gigapixel slides without any image-text pre-training on pathology data.

Load-bearing premise

The entire improvement depends on the generated text descriptions being accurate, class-discriminative, and transferable across institutions, a premise the paper checks with only three pathologists reviewing two cancer types.

Editorial extensions

If this is right

  • With 16 labeled slides per class, the method outperforms the best MIL baselines by 1.7-7.2% in AUC and 2.1-7.3% in F1 across three multi-cancer, multi-center subtyping datasets.
  • In cross-center transfer between two renal cell carcinoma datasets, the model keeps a 5.5% AUC advantage over the best baseline, indicating that the language prior transfers to unseen data distributions.
  • Ablations show each component contributes: the prototype-guided decoder improves over attention pooling, the dual-scale prompt adds over either single scale, and the context-guided text decoder adds further gains.
  • The prompt generation is robust to the choice of LLM: several popular large language models all produce prompts that beat class-name-replacement templates.
  • Swapping the frozen CLIP backbone for pathology-specific vision-language encoders raises AUC by about one point, so the method improves further as better vision-language encoders become available.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the descriptive prompts are the main driver, then prompt quality should be directly tunable: editing the generated texts with a pathologist or using a stronger LLM should yield further few-shot gains without touching the decoders, a cheap experiment on the same three datasets.
  • The same dual-scale design could transfer to other gigapixel-image domains, such as satellite or whole-brain imaging, whenever a domain expert can specify what morphology appears at low versus high resolution, since no image-text pre-training pairs are needed.
  • Because the method computes per-scale similarities and sums their logits, the scale weights could be made learnable per task, so a task whose diagnostic features live at one magnification would automatically down-weight the other scale.
  • A failure mode worth testing externally is the rare-disease case: if the LLM has never seen the cancer subtype, its generated descriptions may be generic, and a test on a rare subtype would reveal how much of the gain depends on the LLM's prior knowledge.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper proposes ViLa-MIL, a multiple instance learning framework for few-shot whole slide image classification that combines a frozen CLIP backbone with (i) dual-scale visual descriptive text prompts generated by a frozen LLM (GPT-3.5), (ii) a prototype-guided patch decoder that aggregates large sets of patch features into learnable prototypes, and (iii) a context-guided text decoder that refines text features using patch and prototype features. The model is trained end-to-end with a cross-entropy loss on slide labels. Experiments on three WSI subtyping datasets (TIHD-RCC, TCGA-RCC, TCGA-Lung) under a 16-shot setting report AUC/F1/ACC improvements over several MIL baselines, plus ablation studies, cross-dataset generalization, and interpretability visualizations.

Significance. If the reported gains hold, the paper makes a useful contribution by showing that LLM-generated, scale-conditioned descriptive prompts can transfer CLIP to pathology in a parameter-efficient way, reducing the need for large collections of pathology image-text pairs. The prototype-guided aggregation is a sensible way to adapt CLIP to gigapixel images, and the cross-center evaluation is a strength. The paper ships code and reports multiple runs with standard deviations. However, the central claim of state-of-the-art performance is presently supported only against MIL baselines that do not use language priors, and the most directly related VLM-MIL method (TOP) is not evaluated, which tempers the significance until that comparison is added.

major comments (4)
  1. [Section 4.2 and Table 1; Section 2 (TOP [30])] The main comparison omits TOP, a two-level prompt learning MIL framework for few-shot weakly-supervised WSI classification that is the most closely related VLM-MIL method. Without a head-to-head comparison under the same protocol, the claim that ViLa-MIL outperforms state-of-the-art by 1.7–7.2% AUC is not substantiated for VLM-based methods. Please add TOP (and, if feasible, MI-Zero [24]) to Table 1 and the cross-dataset table, or revise the claim to refer only to MIL baselines.
  2. [Supplementary K and Section 4.1] Hyperparameters Np and M are selected on the TIHD-RCC dataset, which is also one of the three datasets used for the main evaluation. The manuscript does not state whether this selection used the validation split only. To avoid optimistic bias on TIHD-RCC, please clarify the model-selection protocol and, if the test split was used, report results retrained with hyperparameters fixed on a validation set for all datasets.
  3. [Table 3] The gain from adding the context-guided text decoder to the dual-scale model is 83.6±2.7 to 84.3±4.6 AUC on TIHD-RCC; with five runs and overlapping standard deviations, this difference is not established as significant. Please report paired significance tests (or a confidence interval) for the ablation steps, and temper the corresponding sentence in Section 4.4.
  4. [Table 5 and Section 4.4] The 'Large Language Model' baseline uses a single-scale GPT-3.5 prompt, while ViLa-MIL uses dual-scale prompts plus two new decoders. This confounds the contribution of the dual-scale text prompt with the architectural changes. Please add an ablation that uses the dual-scale prompt with ABMIL (no prototype or text decoders) to isolate the effect of the text prompt design.
minor comments (6)
  1. [Section 3.3, Eq. (4)] Please define Ql, Kl, Vl explicitly before the equation; currently the notation is introduced only in the surrounding prose.
  2. [Section 3.5, Eq. (9)] Please clarify whether α_l and α_h are learned or fixed (the implementation says both are 1); if fixed, note it in the text.
  3. [Table 1 caption] The superscript * is described as denoting comparable performance based on a paired t-test, but it is not stated to which reference the asterisked methods are compared; please clarify.
  4. [Section 4.1] Please specify the exact GPT-3.5 model version used for reproducibility (e.g., text-davinci-003 or gpt-3.5-turbo).
  5. [Supplementary F] The phrase 'to obtain the prediction result' should read 'to obtain the visualization result' to match the context of threshold-based visualization.
  6. [Table 6] The claim that GPT-4 achieves superior performances over GPT-3.5 is not true for F1 (68.0 vs 68.7); please report precisely.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: ViLa-MIL is an end-to-end supervised pipeline evaluated on held-out test splits, with LLM-generated prompts independent of test labels.

full rationale

ViLa-MIL's derivation chain is not circular. The dual-scale descriptive text prompts are produced by a frozen LLM (GPT-3.5) from class names and a fixed question template, before training and without access to the 16-shot training labels or test labels; they are an externally generated language prior, not a fitted parameter. The learnable context vectors, prototypes, and decoder weights are trained only on the 16-shot training subset and evaluated on held-out test splits over five random splits, so the reported AUC/F1 numbers are genuine out-of-sample measurements rather than reconstructions of the training objective. Each architectural component (prototype-guided patch decoder, context-guided text decoder) is ablated against alternatives (Tables 3-5, S2), and the text-prompt contribution is compared against class-name-replacement and diagnostic-guideline prompts; these ablations show incremental gains that are not forced by construction. The paper cites prior works by the same authors ([35,36]) only as examples of MIL applications in cancer subtyping and staging; these citations are not load-bearing for the central claim. The omission of the TOP [30] baseline from Table 1 is a legitimate comparative-evaluation concern, but it is a baseline-completeness issue, not circularity: no claim in the paper reduces to its own inputs by definition or by self-citation. Supplementary A's note that only three pathologists validated the GPT-3.5 prompts is an external-validity limitation, not a circular step, and Table 6 further shows the method's sensitivity to the choice of LLM, confirming the prompts act as an input whose quality affects performance rather than a fitted prediction. Overall, the derivation is self-contained and benchmarked against held-out data, so circularity is minimal.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

No new physical or scientific entities are introduced; the architectural components (dual-scale prompts, prototype-guided decoder, context-guided text decoder) are computational constructs. The core assumptions are the reliability of frozen CLIP features, the accuracy of GPT-generated prompts, and the complementarity of the two chosen magnifications.

free parameters (4)
  • Number of prototypes Np = 16
    Tuned on TIHD-RCC ablation; best value in Figure S5.
  • Number of learnable text vectors M = 16
    Tuned on TIHD-RCC ablation; best value in Figure S5.
  • Scale weight alpha_l, alpha_h = 1, 1
    Set by hand without search; Eq. (9).
  • Context vectors and decoder weights = Learned
    Learned from the 16-shot training set via cross-entropy loss.
assumptions (4)
  • domain assumption Frozen CLIP encoders (ResNet-50 image, Transformer text) provide a meaningful shared embedding space for pathology patches and text.
    The method leverages CLIP without fine-tuning; its utility is assumed from prior CLIP results and tested only in ablation with PLIP/QuiltNet.
  • domain assumption GPT-3.5-generated dual-scale text prompts contain accurate, class-discriminative pathological descriptions consistent with human diagnosis.
    Validated by only three pathologists on two cancer types (Supplementary A); the prompts for other classes are not independently verified.
  • domain assumption Low- and high-magnification patches provide complementary diagnostic information captured by the two-scale architecture.
    The paper assumes the two scales are sufficient and does not test other magnifications or combinations.
  • standard math Multiple instance learning assumption: each slide is a bag of patches and the slide label is determined by a subset of patches.
    Standard MIL assumption used in Section 3.3.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ViLa-MIL: Dual-scale Vision-Language Multiple Instance Learning for Whole Slide Image Classification." pith.science (2026). https://pith.science/paper/K32DELWV

@misc{pith2026250208391,
  author       = {Pith},
  title        = {Pith review of: ViLa-MIL: Dual-scale Vision-Language Multiple Instance Learning for Whole Slide Image Classification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K32DELWV}},
  note         = {Machine review of arXiv:2502.08391}
}
read the original abstract

Multiple instance learning (MIL)-based framework has become the mainstream for processing the whole slide image (WSI) with giga-pixel size and hierarchical image context in digital pathology. However, these methods heavily depend on a substantial number of bag-level labels and solely learn from the original slides, which are easily affected by variations in data distribution. Recently, vision language model (VLM)-based methods introduced the language prior by pre-training on large-scale pathological image-text pairs. However, the previous text prompt lacks the consideration of pathological prior knowledge, therefore does not substantially boost the model's performance. Moreover, the collection of such pairs and the pre-training process are very time-consuming and source-intensive.To solve the above problems, we propose a dual-scale vision-language multiple instance learning (ViLa-MIL) framework for whole slide image classification. Specifically, we propose a dual-scale visual descriptive text prompt based on the frozen large language model (LLM) to boost the performance of VLM effectively. To transfer the VLM to process WSI efficiently, for the image branch, we propose a prototype-guided patch decoder to aggregate the patch features progressively by grouping similar patches into the same prototype; for the text branch, we introduce a context-guided text decoder to enhance the text features by incorporating the multi-granular image contexts. Extensive studies on three multi-cancer and multi-center subtyping datasets demonstrate the superiority of ViLa-MIL.

Figures

Figures reproduced from arXiv: 2502.08391 by the authors.

Figure 1
Figure 1. Comparison of our ViLa-MIL with existing MIL- and [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Pipeline of the proposed ViLa-MIL framework. The input of ViLa-MIL is a Question and WSI. The question is passed through [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. (a) Prototype-guided patch decoder; (b) Context-guided [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Slide-level feature clustering results of different methods [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Interpretability analysis (yellow for cancer) of several exemplars from the TIHD-RCC and TCGA-RCC datasets. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

51 extracted references · 42 canonical work pages

  1. [24]

    Visual language pre- trained multiple instance zero-shot transfer for histopathol- ogy images

    Ming Y Lu, Bowen Chen, Andrew Zhang, Drew FK Williamson, Richard J Chen, Tong Ding, Long Phi Le, Yung- Sung Chuang, and Faisal Mahmood. Visual language pre- trained multiple instance zero-shot transfer for histopathol- ogy images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19764– 19775, 2023. 2, 3

    work page 2023

  2. [1]

    Lan- guage models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Sub- biah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakan- tan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Lan- guage models are few-shot learners. Advances in Neural In- formation Processing Systems, 33:1877–1901, 2020. 2, 8

    work page 1901

  3. [2]

    Histopathology whole slide image anal- ysis with heterogeneous graph representation learning

    Tsai Hor Chan, Fernando Julio Cendra, Lan Ma, Guosheng Yin, and Lequan Yu. Histopathology whole slide image anal- ysis with heterogeneous graph representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 15661–15670, 2023. 2

    work page 2023

  4. [3]

    PaLM: Scaling language modeling with Pathways

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. PaLM: Scaling language modeling with Pathways. arXiv preprint arXiv:2204.02311, 2022. 8

    arXiv 2022

  5. [4]

    Artificial intelligence and computational pathology

    Miao Cui and David Y Zhang. Artificial intelligence and computational pathology. Laboratory Investigation , 101(4):412–422, 2021. 1

    work page 2021

  6. [5]

    Fast graph represen- tation learning with PyTorch Geometric

    Matthias Fey and Jan Eric Lenssen. Fast graph represen- tation learning with PyTorch Geometric. arXiv preprint arXiv:1903.02428, 2019. 12

    arXiv 1903

  7. [6]

    Multiple instance cap- tioning: Learning representations from histopathology text- books and articles

    Jevgenij Gamper and Nasir Rajpoot. Multiple instance cap- tioning: Learning representations from histopathology text- books and articles. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 16549–16559, 2021. 2

    work page 2021

  8. [7]

    CLIP-Adapter: Better vision-language models with feature adapters

    Peng Gao, Shijie Geng, Renrui Zhang, Teli Ma, Rongyao Fang, Yongfeng Zhang, Hongsheng Li, and Yu Qiao. CLIP-Adapter: Better vision-language models with feature adapters. International Journal of Computer Vision , pages 1–15, 2023. 3

    work page 2023

Show all 51 references
  1. [8]

    Texts as images in prompt tun- ing for multi-label image recognition

    Zixian Guo, Bowen Dong, Zhilong Ji, Jinfeng Bai, Yiwen Guo, and Wangmeng Zuo. Texts as images in prompt tun- ing for multi-label image recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2808–2817, 2023. 2, 3

    2023

  2. [9]

    E 2VPT: An ef- fective and efficient approach for visual prompt tuning

    Cheng Han, Qifan Wang, Yiming Cui, Zhiwen Cao, Wen- guan Wang, Siyuan Qi, and Dongfang Liu. E 2VPT: An ef- fective and efficient approach for visual prompt tuning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 17491–17502, 2023. 3

    2023

  3. [10]

    H2-MIL: Ex- ploring hierarchical representation with heterogeneous mul- tiple instance learning for whole slide image analysis

    Wentai Hou, Lequan Yu, Chengxuan Lin, Helong Huang, Rongshan Yu, Jing Qin, and Liansheng Wang. H2-MIL: Ex- ploring hierarchical representation with heterogeneous mul- tiple instance learning for whole slide image analysis. InPro- ceedings of the AAAI Conference on Artificial I...

    2022

  4. [11]

    ConSlide: Asynchronous hierarchical interaction Transformer with breakup-reorganize rehearsal for continual whole slide image analysis

    Yanyan Huang, Weiqin Zhao, Shujun Wang, Yu Fu, Yuming Jiang, and Lequan Yu. ConSlide: Asynchronous hierarchical interaction Transformer with breakup-reorganize rehearsal for continual whole slide image analysis. In Proceedings of the IEEE/CVF International Conference on Comput...

    2023

  5. [12]

    A visual-language foundation model for pathology image analysis using medical Twitter

    Zhi Huang, Federico Bianchi, Mert Yuksekgonul, Thomas J Montine, and James Zou. A visual-language foundation model for pathology image analysis using medical Twitter. Nature Medicine, pages 1–10, 2023. 2, 3, 14

    2023

  6. [13]

    Quilt-1M: One million image-text pairs for histopathology

    Wisdom Ikezogwo, Saygin Seyfioglu, Fatemeh Ghezloo, Dylan Geva, Fatwir Sheikh Mohammed, Pavan Kumar Anand, Ranjay Krishna, and Linda Shapiro. Quilt-1M: One million image-text pairs for histopathology. Advances in Neural Information Processing Systems, 36, 2024. 2, 3, 14

    2024

  7. [14]

    Attention-based deep multiple instance learning

    Maximilian Ilse, Jakub Tomczak, and Max Welling. Attention-based deep multiple instance learning. InProceed- ings of the International Conference on Machine Learning , pages 2127–2136, 2018. 1, 2, 3, 4, 6, 7, 12

    2018

  8. [15]

    MaPle: Multi-modal prompt learning

    Muhammad Uzair Khattak, Hanoona Rasheed, Muhammad Maaz, Salman Khan, and Fahad Shahbaz Khan. MaPle: Multi-modal prompt learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19113–19122, 2023. 3

    2023

  9. [16]

    Dual-stream multiple instance learning network for whole slide image classifica- tion with self-supervised contrastive learning

    Bin Li, Yin Li, and Kevin W Eliceiri. Dual-stream multiple instance learning network for whole slide image classifica- tion with self-supervised contrastive learning. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14318–14328, 202...

    2021

  10. [17]

    Task-specific fine-tuning via variational information bottle- neck for weakly-supervised pathology whole slide image classification

    Honglin Li, Chenglu Zhu, Yunlong Zhang, Yuxuan Sun, Zhongyi Shui, Wenwei Kuang, Sunyi Zheng, and Lin Yang. Task-specific fine-tuning via variational information bottle- neck for weakly-supervised pathology whole slide image classification. In Proceedings of the IEEE/CVF Confer...

    2023

  11. [18]

    BLIP: Bootstrapping language-image pre-training for uni- fied vision-language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. BLIP: Bootstrapping language-image pre-training for uni- fied vision-language understanding and generation. In In- ternational Conference on Machine Learning, pages 12888– 12900. PMLR, 2022. 2, 3

    2022

  12. [19]

    Weakly supervised histopathology image segmentation with self-attention

    Kailu Li, Ziniu Qian, Yingnan Han, I Eric, Chao Chang, Bingzheng Wei, Maode Lai, Jing Liao, Yubo Fan, and Yan Xu. Weakly supervised histopathology image segmentation with self-attention. Medical Image Analysis , 86:102791,

  13. [20]

    Scaling language-image pre-training via masking

    Yanghao Li, Haoqi Fan, Ronghang Hu, Christoph Feichten- hofer, and Kaiming He. Scaling language-image pre-training via masking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23390– 23400, 2023. 3

    2023

  14. [21]

    Interventional bag multi-instance learning on whole-slide pathological images

    Tiancheng Lin, Zhimiao Yu, Hongyu Hu, Yi Xu, and Chang- Wen Chen. Interventional bag multi-instance learning on whole-slide pathological images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19830–19839, 2023. 1, 2, 3, 6, 12

    2023

  15. [22]

    CLIP is also an efficient segmenter: A text-driven approach for weakly supervised semantic segmentation

    Yuqi Lin, Minghao Chen, Wenxiao Wang, Boxi Wu, Ke Li, Binbin Lin, Haifeng Liu, and Xiaofei He. CLIP is also an efficient segmenter: A text-driven approach for weakly supervised semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Re...

    2023

  16. [23]

    Towards a visual-language foundation model for computational pathol- ogy

    Ming Y Lu, Bowen Chen, Drew FK Williamson, Richard J Chen, Ivy Liang, Tong Ding, Guillaume Jaume, Igor Odintsov, Andrew Zhang, Long Phi Le, et al. Towards a visual-language foundation model for computational pathol- ogy. arXiv preprint arXiv:2307.12914, 2023. 2

    2023 arXiv

  17. [25]

    Data-efficient and weakly supervised computational pathology on whole- slide images

    Ming Y Lu, Drew FK Williamson, Tiffany Y Chen, Richard J Chen, Matteo Barbieri, and Faisal Mahmood. Data-efficient and weakly supervised computational pathology on whole- slide images. Nature Biomedical Engineering , 5(6):555– 570, 2021. 1, 3, 4, 6, 12

    2021

  18. [26]

    A framework for multiple-instance learning

    Oded Maron and Tom ´as Lozano-P ´erez. A framework for multiple-instance learning. Advances in Neural Information Processing Systems, 10:570–576, 1997. 2

    1997

  19. [27]

    Sparse multi-modal graph Transformer with shared-context processing for representation learning of giga-pixel images

    Ramin Nakhli, Puria Azadi Moghadam, Haoyang Mi, Hos- sein Farahani, Alexander Baras, Blake Gilks, and Ali Bashashati. Sparse multi-modal graph Transformer with shared-context processing for representation learning of giga-pixel images. pages 11547–11557, 2023. 1, 2

    2023

  20. [28]

    Digital pathology and artificial intelli- gence

    Muhammad Khalid Khan Niazi, Anil V Parwani, and Metin N Gurcan. Digital pathology and artificial intelli- gence. The Lancet Oncology, 20(5):e253–e261, 2019. 1

    2019

  21. [29]

    GPT-4 technical report

    OpenAI. GPT-4 technical report. arXiv preprint arXiv:2303.08774, 2023. 8

    2023 arXiv

  22. [30]

    The rise of ai language pathologists: Exploring two-level prompt learning for few-shot weakly-supervised whole slide image classification

    Linhao Qu, Kexue Fu, Manning Wang, Zhijian Song, et al. The rise of ai language pathologists: Exploring two-level prompt learning for few-shot weakly-supervised whole slide image classification. Advances in Neural Information Pro- cessing Systems, 36, 2024. 3

    2024

  23. [31]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In International Conference on Machine Learning...

    2021

  24. [32]

    DenseCLIP: Language-guided dense prediction with context-aware prompting

    Yongming Rao, Wenliang Zhao, Guangyi Chen, Yansong Tang, Zheng Zhu, Guan Huang, Jie Zhou, and Jiwen Lu. DenseCLIP: Language-guided dense prediction with context-aware prompting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 18082–...

    2022

  25. [33]

    OCELOT: Overlapped cell on tissue dataset for histopathol- ogy

    Jeongun Ryu, Aaron Valero Puche, JaeWoong Shin, Seon- wook Park, Biagio Brattoli, Jinhee Lee, Wonkyung Jung, Soo Ick Cho, Kyunghyun Paeng, Chan-Young Ock, et al. OCELOT: Overlapped cell on tissue dataset for histopathol- ogy. In Proceedings of the IEEE/CVF Conference on Com- p...

  26. [34]

    TransMIL: Transformer based correlated multiple instance learning for whole slide image classification

    Zhuchen Shao, Hao Bian, Yang Chen, Yifeng Wang, Jian Zhang, Xiangyang Ji, et al. TransMIL: Transformer based correlated multiple instance learning for whole slide image classification. Advances in Neural Information Processing Systems, 34:2136–2147, 2021. 2, 3, 6, 12

    2021

  27. [35]

    MG-Trans: Multi-scale graph transformer with information bottleneck for whole slide image classification

    Jiangbo Shi, Lufei Tang, Zeyu Gao, Yang Li, Chunbao Wang, Tieliang Gong, Chen Li, and Huazhu Fu. MG-Trans: Multi-scale graph transformer with information bottleneck for whole slide image classification. IEEE Transactions on Medical Imaging, 42(12):3871–3883, 2023. 2

    2023

  28. [36]

    A structure-aware hierarchical graph-based multiple instance learning framework for pT staging in histopathological im- age

    Jiangbo Shi, Lufei Tang, Yang Li, Xianli Zhang, Zeyu Gao, Yefeng Zheng, Chunbao Wang, Tieliang Gong, and Chen Li. A structure-aware hierarchical graph-based multiple instance learning framework for pT staging in histopathological im- age. IEEE Transactions on Medical Imaging, ...

    2023

  29. [37]

    Artificial intelligence for digital and computa- tional pathology

    Andrew H Song, Guillaume Jaume, Drew FK Williamson, Ming Y Lu, Anurag Vaidya, Tiffany R Miller, and Faisal Mahmood. Artificial intelligence for digital and computa- tional pathology. Nature Reviews Bioengineering, pages 1– 20, 2023. 1

    2023

  30. [38]

    Multiple instance learning framework with masked hard instance mining for whole slide image classification

    Wenhao Tang, Sheng Huang, Xiaoxian Zhang, Fengtao Zhou, Yi Zhang, and Bo Liu. Multiple instance learning framework with masked hard instance mining for whole slide image classification. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 4078–4087,

  31. [39]

    LLaMA: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. LLaMA: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. 8

    2023 arXiv

  32. [40]

    Visualizing data using t-SNE

    Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE. Journal of Machine Learning Research , 9(11):2579–2605, 2008. 7

    2008

  33. [41]

    CORA: Adapting CLIP for open-vocabulary detection with region prompting and anchor pre-matching

    Xiaoshi Wu, Feng Zhu, Rui Zhao, and Hongsheng Li. CORA: Adapting CLIP for open-vocabulary detection with region prompting and anchor pre-matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 7031–7040, 2023. 2

    2023

  34. [42]

    CAMEL: A weakly supervised learning framework for histopathology image segmentation

    Gang Xu, Zhigang Song, Zhuo Sun, Calvin Ku, Zhe Yang, Cancheng Liu, Shuhao Wang, Jianpeng Ma, and Wei Xu. CAMEL: A weakly supervised learning framework for histopathology image segmentation. In Proceedings of the IEEE/CVF International Conference on computer vision, pages 1068...

    2019

  35. [43]

    Task residual for tuning vision-language models

    Tao Yu, Zhihe Lu, Xin Jin, Zhibo Chen, and Xinchao Wang. Task residual for tuning vision-language models. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10899–10909, 2023. 2, 3

    2023

  36. [44]

    DTFD- MIL: Double-tier feature distillation multiple instance learn- ing for histopathology whole slide image classification

    Hongrun Zhang, Yanda Meng, Yitian Zhao, Yihong Qiao, Xiaoyun Yang, Sarah E Coupland, and Yalin Zheng. DTFD- MIL: Double-tier feature distillation multiple instance learn- ing for histopathology whole slide image classification. In Proceedings of the IEEE/CVF Conference on Comp...

    2022

  37. [45]

    Large-scale domain-specific pre- training for biomedical vision-language processing

    Sheng Zhang, Yanbo Xu, Naoto Usuyama, Jaspreet Bagga, Robert Tinn, Sam Preston, Rajesh Rao, Mu Wei, Naveen Valluri, Cliff Wong, et al. Large-scale domain-specific pre- training for biomedical vision-language processing. arXiv preprint arXiv:2303.00915, 2023. 2

    2023 arXiv

  38. [46]

    Text-guided foun- dation model adaptation for pathological image classifica- tion

    Yunkun Zhang, Jin Gao, Mu Zhou, Xiaosong Wang, Yu Qiao, Shaoting Zhang, and Dequan Wang. Text-guided foun- dation model adaptation for pathological image classifica- tion. In International Conference on Medical Image Com- puting and Computer Assisted Intervention, pages 272–28...

    2023

  39. [47]

    Gindra, Emily J

    Yi Zheng, Rushin H. Gindra, Emily J. Green, Eric J. Burks, Margrit Betke, Jennifer E. Beane, and Vijaya B. Kolacha- lama. A graph-Transformer for whole slide image classifica- tion. IEEE Transactions on Medical Imaging, 41(11):3003– 3015, 2022. 3, 6, 12

    2022

  40. [48]

    RegionCLIP: Region- based language-image pretraining

    Yiwu Zhong, Jianwei Yang, Pengchuan Zhang, Chunyuan Li, Noel Codella, Liunian Harold Li, Luowei Zhou, Xiyang Dai, Lu Yuan, Yin Li, et al. RegionCLIP: Region- based language-image pretraining. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,...

    2022

  41. [49]

    Learning to prompt for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. In- ternational Journal of Computer Vision, 130(9):2337–2348,

  42. [51]

    class name

    exhibit AUC improvements of 1.4% and 0.9 %. This indicates that VLMs pre-trained on the domain-specific data contribute to enhancing model performance further. Figure S1. Dual-scale visual descriptive text prompt for the renal cell carcinoma. By replacing the placeholder “clas...

  43. [2022]

    ABMIL + Low-scale

    3, 4 Supplementary Materials A. Dual-scale Visual Descriptive Text Prompt The specific descriptions of dual-scale visual descrip- tive text prompts for renal cell carcinoma and lung cancer are shown in Figure S1 and Figure S2, respectively. Note that three experienced patholog...

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.