Pith. sign in

REVIEW 4 major objections 3 minor 25 references

Gland Segmentation Using SAM With Cancer Grade as a Prompt

T0 review · 4 major / 3 minor · reviewed 2026-08-10 · deepseek-v4-flash

Pith's one-line read Cancer grade, encoded as a Grad-CAM++ heat map from a ViT classifier and fused into SAM through a prompt adapter, improves gland segmentation on GlaS beyond fine-tuned SAM alone.

desk verdict Grade-prompted SAM is a plausible but under-evidenced increment: the prompt effect is not isolated from added architecture and no ablation supports it. read the letter →

arxiv 2501.14718 v2 pith:Q7YJ5GC6 submitted 2025-01-24 eess.IV

classification eess.IV
keywords glandsegmentationSegmentAnythingModelcancergradepromptGrad-CAM++benignmalignantclassificationcolorectalhistologyGlaSdataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Cancer grade is a categorical clinical label, and this paper asks whether it can be turned into a spatial prompt for gland segmentation. The proposed model fine-tunes a Vision Transformer to classify benign versus malignant glands, explains that decision with Grad-CAM++, and feeds the resulting heat map through a small prompt adapter into a fine-tuned Segment Anything Model (SAM), a foundation model for image segmentation. The segmentation branch predicts gland and contour masks simultaneously, and the overlap between them is removed to separate touching glands. On the GlaS colorectal benchmark, prompted SAM-H reports F1 0.929/0.841, object Dice 0.921/0.881, and object Hausdorff 41.189/74.300 on test sets A and B, improving over fine-tuned SAM alone at all three scales and over the listed contest entries. If the idea holds, it suggests that a simple image-level label can act as a cheap, effective segmentation prompt in medical imaging.

What carries the argument

The central object is the grade prompt: a one-channel Grad-CAM++ heat map produced by a fine-tuned ViT that has just classified the gland as benign or malignant. The prompt adapter processes this heat map by concatenating it with the original image and passing the four-channel tensor through two convolutional layers with batch normalization and ReLU, then adds the result to the signal fed into SAM's gland prompt encoder. The segmentation branch splits into gland and contour predictions, each with its own prompt encoder and mask decoder, and post-processing removes the predicted contour overlap from the gland mask so adjacent glands stay separate. Training is staged: the classification branch is fine-tuned first, then the shared image encoder, prompt adapter, and gland decoder, then the contour decoder with the image encoder frozen, and a U-Net-style weight map weights the pixel-wise MSE loss.

What would settle it

Fix all training settings and replace the Grad-CAM++ heat map with a constant-valued map of the same size before it enters the prompt adapter; if the scores remain close to the prompted SAM results, the grade signal is not the active ingredient.

Watch

Extended reading notes

Core claim

The central claim is that a binary cancer-grade label can carry useful spatial information sufficient to guide a foundation segmentation model. The benign/malignant ViT classifier emits a Grad-CAM++ heat map, and that one-channel map, concatenated with the original image and compressed by two convolutional layers in a prompt adapter, becomes the prompt given to SAM's gland mask decoder. With this grade prompt, prompted SAM-H reaches F1 0.929/0.841, object Dice 0.921/0.881, and object Hausdorff 41.189/74.300 on GlaS test sets A and B, outperforming fine-tuned SAM-H and every GlaS benchmark entry listed in the paper. The paper attributes the gain to the prior knowledge of gland type embedded in the heat map, which lets the segmentation model anticipate the appearance of the target.

Load-bearing premise

The whole improvement rests on the assumption that the benign/malignant heat map, not the extra network layers added at the same time, is what makes segmentation better; the paper does not test this by removing the heat map.

Editorial extensions

If this is right

  • Prompted SAM matches or beats unprompted fine-tuned SAM at every scale tested (ViT-B, ViT-L, ViT-H), so the benefit is not tied to one model size.
  • The joint gland-and-contour prediction with overlap removal gives a practical way to separate touching glands, a known failure mode in gland segmentation.
  • A single forward path yields both a benign/malignant classification and a gland/contour segmentation, so the two clinical tasks are solved together.
  • The prompted SAM-H numbers top the GlaS benchmarks listed in the paper, so the grade-prompt recipe is competitive with task-specific gland segmentation networks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test the paper leaves undone: replace the heat map with a blank or shuffled one while freezing all adapter parameters; if segmentation stays at the prompted level, the grade information is not the cause of the gain.
  • The recipe is portable to other histology tasks where a coarse image-level label exists, such as nuclei or tissue-type segmentation, and the paper names cell segmentation as future work.
  • Because SAM already supports point, box, and text prompts, the grade heat map could be combined with those prompt types rather than used alone, which may give even stronger guidance without new annotations.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The paper proposes a two-branch model for colorectal gland segmentation evaluated on the GlaS dataset. A ViT classification branch is fine-tuned for benign/malignant gland classification, and a Grad-CAM++ heatmap derived from this classifier is used as a prompt for a modified Segment Anything Model (SAM). The segmentation branch contains a shared image encoder, separate prompt encoders and mask decoders for gland and contour prediction, a trainable prompt adapter, a stepwise training schedule, and contour-based post-processing. The paper claims that the prompted SAM variants improve over fine-tuned SAMs and that the best variant, prompted SAM-H, achieves SOTA results on GlaS. The evidence for these claims is presented in Table 1 (prompted vs. fine-tuned SAM) and Table 2 (comparison with GlaS contest baselines).

Significance. The underlying idea is interesting and potentially useful: using a grade classifier's saliency map as a conditioning signal for SAM is a plausible mechanism for injecting clinical prior knowledge into gland segmentation in histopathology. If the prompt benefit were robustly established, the paper would make a useful contribution to the medical-image segmentation literature. The paper also provides a simple prompt-adapter design and a dual-branch SAM modification that are easy to understand. However, as presented, the central claim is not supported by the experimental evidence: the improvements in Table 1 are small, inconsistent, and reported without uncertainty estimates; no ablation isolates the grade-prompt component; the spatial alignment of the heatmap is underspecified; and the SOTA comparison is limited to dated contest baselines. The significance of the work is therefore conditional on a substantially strengthened experimental evaluation.

major comments (4)
  1. [§3.3, Table 1] The central quantitative claim is not established because the improvements are inconsistent and no uncertainty is reported. Prompted SAM-H F1 on Test A decreases from 0.932 to 0.929, SAM-L Object Dice on Test B is tied at 0.846, and most other differences are at the third decimal place. With test sets of only 60 and 20 images, and no repeated runs, seeds, confidence intervals, or significance tests, these differences cannot be distinguished from noise. The authors should report multiple runs with means and standard deviations and apply paired statistical tests (e.g., Wilcoxon signed-rank or bootstrap) on object-level metrics.
  2. [§2.3–§2.7] The experimental design does not isolate the grade prompt as the cause of any observed improvement. The prompted model is not simply fine-tuned SAM plus a prompt; it also adds a trainable prompt adapter (§2.3), a second prompt encoder and mask decoder for contour prediction, a stepwise training schedule (§2.5), and contour-based post-processing that removes overlapping gland and contour regions (§2.7). Any of these components could explain the differences in Table 1. No ablation feeds a blank, constant, shuffled, or adversarial heatmap into the same architecture, so the causal role of the cancer-grade information is not tested. Such ablations are necessary before claiming that cancer grade, rather than added capacity or post-processing, drives the reported gains.
  3. [§2.2 and §3.2] The alignment between the grade heatmap and SAM's input space is underspecified. The classifier is deit-base-patch16-224, while the SAM input is stated to be 400×400 pixels, and the text only says that the heatmap "maintains the same size as the original input image." The paper must state how the Grad-CAM++ output is resized or generated at 400×400, including how DeiT's positional embeddings are handled. Without this, the method cannot be reproduced, and it is unclear whether the prompt is generated under assumptions compatible with the pretrained ViT or is a coarse upsampling that would not carry boundary-level gland information.
  4. [Tables 1–2 and §4] The claim of SOTA results is not supported by the comparison in Table 2. The table lists only GlaS contest entries from 2017, and the paper's own conclusion cites Hi-gMISnet with an F1 score of 0.932 on the overall test set, yet this method is not included in the comparison. The authors should either qualify the claim to "improves over the fine-tuned SAM baseline" or extend the comparison to modern methods evaluated under the same protocol and split.
minor comments (3)
  1. [§2.3] The adapter description is difficult to follow: the text says the heatmap is concatenated with the original image and the four-channel feature is reduced to one channel, then "the heat map is added with the output from the second convolution." Please clarify the exact residual structure and the channel dimensions at each step.
  2. [§3.1] Hyperparameters such as learning rates, batch size, number of epochs, validation split, and the exact patch-extraction and inference-overlap strategy are not reported, which limits reproducibility.
  3. [§4] There is a typo in the conclusion: "purposed method" should be "proposed method."

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the claimed prompt benefit is an empirical comparison, not a derivation that reduces to its own inputs.

full rationale

The paper's central claim is that a Grad-CAM++ heat map from a benign/malignant ViT classifier, used as a prompt to a fine-tuned SAM, improves gland segmentation. This is an empirical result: the heat map is generated from a classifier trained on benign/malignant labels, not derived from the segmentation ground truth, and no segmentation metric is fitted or reverse-engineered into the prompt. The prompt adapter, contour branch, and post-processing introduce architectural changes, but they are not circular with the evaluation. The comparison to fine-tuned SAM is a direct experimental comparison on the GlaS test sets, and while the reported gains are small and the prompt benefit is not isolated by an ablation, that is a question of experimental rigor, not circularity. There are no self-citations that carry the argument, no imported uniqueness theorems, and no fitted parameter renamed as a prediction. The derivation is self-contained as an empirical study, so the circularity score is 0.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new physical or conceptual entities. The prompt adapter is a trainable module whose behavior is fully determined by its inputs and weights, and the central claim does not depend on a new postulated object.

free parameters (2)
  • Binary threshold = 0.5
    Post-processing threshold to convert sigmoid output to a binary mask; chosen by hand and affects final metrics.
  • Input patch size = 400x400
    Crops from GlaS images with overlap; chosen to fit the SAM input and affects context and augmentation.
assumptions (3)
  • domain assumption Benign and malignant glands have sufficiently distinct morphology that grade information can guide segmentation.
    Sections 1 and 2.2 rely on this prior; if it is false, the prompt would add no signal beyond the raw image.
  • domain assumption Grad-CAM++ heat maps localize the image regions responsible for classification and are spatially aligned to the SAM input.
    Section 2.2 and Figure 2: the adapter concatenates the heat map with the original image, but no alignment or resampling details are given.
  • domain assumption Fine-tuned SAM weights provide a useful initialization for the modified segmentation branch.
    Section 2.5: transfer learning from a fine-tuned SAM is assumed to be a correct starting point without comparison to random initialization.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Gland Segmentation Using SAM With Cancer Grade as a Prompt." pith.science (2026). https://pith.science/paper/Q7YJ5GC6

@misc{pith2026250114718,
  author       = {Pith},
  title        = {Pith review of: Gland Segmentation Using SAM With Cancer Grade as a Prompt},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Q7YJ5GC6}},
  note         = {Machine review of arXiv:2501.14718}
}
read the original abstract

Cancer grade is a critical clinical criterion that can be used to determine the degree of cancer malignancy. Revealing the condition of the glands, a precise gland segmentation can assist in a more effective cancer grade classification. In machine learning, binary classification information about glands (i.e., benign and malignant) can be utilized as a prompt for gland segmentation and cancer grade classification. By incorporating prior knowledge of the benign or malignant classification of the gland, the model can anticipate the likely appearance of the target, leading to better segmentation performance. We utilize Segment Anything Model to solve the segmentation task, by taking advantage of its prompt function and applying appropriate modifications to the model structure and training strategies. We improve the results from fine-tuned Segment Anything Model and produce SOTA results using this approach.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

25 extracted references · 21 canonical work pages

  1. [1]

    A large proportion of col- orectal cancers are classified as adenocarcinomas [1]

    INTRODUCTION Colorectal cancer is one of the most common cancer types, with a notably high mortality rate. A large proportion of col- orectal cancers are classified as adenocarcinomas [1]. Col- orectal adenocarcinoma is distinguished by glandular forma- tion. Pathologists rely on the morphology of glands and the degree of glandular formation as factors to...

  2. [2]

    Gland Segmentation Using SAM With Cancer Grade as a Prompt

    METHODS The proposed model shown in Fig. 1 is designed to generate predictions for both cancer classification and gland segmen- tation simultaneously through two branches. arXiv:2501.14718v2 [eess.IV] 27 Jan 2025 Fig. 1 . The classification branch determines whether the gland is benign or malignant and generates a heat map. The prompt adapter processes th...

  3. [3]

    Data Preparation The open-source Gland Segmentation Challenge (GlaS) [15] dataset was used in this study

    EXPERIMENT AND RESULTS 3.1. Data Preparation The open-source Gland Segmentation Challenge (GlaS) [15] dataset was used in this study. There are 85 images in the training set. There are 60 and 20 images in test sets A and B. A square with a size of 400 ×400 pixels was set as the in- put for the model. The images in GlaS dataset are all larger than this sta...

  4. [4]

    A new method to provide grade prompt is designed

    CONCLUSION As benign and malignant glands exhibit varying morpholog- ical characteristics, a deep learning model can take advan- tage of cancer grade information as prior knowledge. A new method to provide grade prompt is designed. Heat maps high- lighting regions generated by Grad-CAM++ serve as a prompt Table 2. Comparison of Metrics with GlaS [15] Benc...

  5. [5]

    Primar- ily trained on a broad dataset, it demonstrates impressive zero-shot performance involving natural images

    is dedicated to the tasks of image segmentation. Primar- ily trained on a broad dataset, it demonstrates impressive zero-shot performance involving natural images. However, when applied to more specialized domains, such as medical imaging, this zero-shot performance tends to decline [6]. De- spite this, the strong generalization ability of SAM provides a ...

  6. [6]

    COMPLIANCE WITH ETHICAL STANDARDS The research was conducted using publicly available GlaS Challenge data set [15]

  7. [7]

    ACKNOWLEDGMENT Yijie Zhu is funded by China Scholarship Council - Univer- sity of Warwick Scholarship

  8. [8]

    General insight into cancer: An overview of colorectal cancer,

    Shiekhah Mohammad Alzahrani, Huda Abdulaziz Al Doghaither, and Ayat Badr Al-Ghafari, “General insight into cancer: An overview of colorectal cancer,” Molecular and clinical oncology, vol. 15, no. 6, pp. 271, 2021

Show all 25 references
  1. [9]

    Colorectal carcinoma: Pathologic aspects,

    Matthew Fleming, Sreelakshmi Ravula, Sergei F Tatishchev, and Hanlin L Wang, “Colorectal carcinoma: Pathologic aspects,” Journal of gastrointestinal oncol- ogy, vol. 3, no. 3, pp. 153, 2012

  2. [10]

    Mild-net: Minimal information loss dilated network for gland instance segmentation in colon histology images,

    Simon Graham et al., “Mild-net: Minimal information loss dilated network for gland instance segmentation in colon histology images,” Medical image analysis, vol. 52, pp. 199–211, 2019

  3. [11]

    Gcsba-net: Gabor-based and cascade squeeze bi- attention network for gland segmentation,

    Zhijie Wen, Ru Feng, Jingxin Liu, Ying Li, and Shihui Ying, “Gcsba-net: Gabor-based and cascade squeeze bi- attention network for gland segmentation,” IEEE Jour- nal of Biomedical and Health Informatics , vol. 25, no. 4, pp. 1185–1196, 2021

  4. [12]

    Segment anything,

    Alexander Kirillov et al., “Segment anything,” in Pro- ceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2023, pp. 4015–4026

  5. [13]

    Segment anything model (sam) for digital pathology: Assess zero-shot segmentation on whole slide imaging,

    Ruining Deng et al., “Segment anything model (sam) for digital pathology: Assess zero-shot segmentation on whole slide imaging,”arXiv preprint arXiv:2304.04155, 2023

  6. [14]

    Segment anything in medical images,

    Jun Ma et al., “Segment anything in medical images,” Nature Communications, vol. 15, no. 1, pp. 654, 2024

  7. [15]

    Medical sam adapter: Adapting seg- ment anything model for medical image segmentation,

    Junde Wu et al., “Medical sam adapter: Adapting seg- ment anything model for medical image segmentation,” arXiv preprint arXiv:2304.12620, 2023

  8. [16]

    Segment anything model for medical image analysis: an experimental study,

    Maciej A Mazurowski et al., “Segment anything model for medical image analysis: an experimental study,” Medical Image Analysis, vol. 89, pp. 102918, 2023

  9. [17]

    Glandular morphometrics for objective grading of colorectal adenocarcinoma histol- ogy images,

    Ruqayya Awan et al., “Glandular morphometrics for objective grading of colorectal adenocarcinoma histol- ogy images,” Scientific reports, vol. 7, no. 1, pp. 16852, 2017

  10. [18]

    Cgs-net: Classification-guided segmentation network for improved gland segmenta- tion,

    Xiaoheng Tang et al., “Cgs-net: Classification-guided segmentation network for improved gland segmenta- tion,” in 2023 IEEE 12th Data Driven Control and Learning Systems Conference (DDCLS) . IEEE, 2023, pp. 1008–1013

  11. [19]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    Alexey Dosovitskiy et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” 2021

  12. [20]

    Grad- cam++: Generalized gradient-based visual explanations for deep convolutional networks,

    Aditya Chattopadhay, Anirban Sarkar, Prantik Howlader, and Vineeth N Balasubramanian, “Grad- cam++: Generalized gradient-based visual explanations for deep convolutional networks,” in 2018 IEEE winter conference on applications of computer vision (WACV). IEEE, 2018, pp. 839–847

  13. [21]

    U-net: Convolutional networks for biomedical image segmentation,

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox, “U-net: Convolutional networks for biomedical image segmentation,” 2015

  14. [22]

    Gland segmentation in colon histology images: The glas challenge contest,

    Korsuk Sirinukunwattana et al., “Gland segmentation in colon histology images: The glas challenge contest,” Medical image analysis, vol. 35, pp. 489–502, 2017

  15. [23]

    Training data-efficient image transformers & distillation through attention,

    Hugo Touvron et al., “Training data-efficient image transformers & distillation through attention,” in Inter- national conference on machine learning. PMLR, 2021, pp. 10347–10357

  16. [24]

    Hi- gmisnet: generalized medical image segmentation us- ing dwt based multilayer fusion and dual mode atten- tion into high resolution pgan,

    Tushar Talukder Showrav and Md Kamrul Hasan, “Hi- gmisnet: generalized medical image segmentation us- ing dwt based multilayer fusion and dual mode atten- tion into high resolution pgan,” Physics in Medicine & Biology, vol. 69, no. 11, pp. 115019, 2024

  17. [25]

    Pannuke: an open pan-cancer histology dataset for nuclei instance segmentation and classification,

    Jevgenij Gamper, Navid Alemi Koohbanani, Ksenija Benet, Ali Khuram, and Nasir Rajpoot, “Pannuke: an open pan-cancer histology dataset for nuclei instance segmentation and classification,” in Digital Pathology: 15th European Congress, ECDP 2019, Warwick, UK, April 10–13, 2019, ...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.