REVIEW 4 major objections 3 minor 25 references
Gland Segmentation Using SAM With Cancer Grade as a Prompt
T0 review · 4 major / 3 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read Cancer grade, encoded as a Grad-CAM++ heat map from a ViT classifier and fused into SAM through a prompt adapter, improves gland segmentation on GlaS beyond fine-tuned SAM alone.
desk verdict Grade-prompted SAM is a plausible but under-evidenced increment: the prompt effect is not isolated from added architecture and no ablation supports it. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the grade prompt: a one-channel Grad-CAM++ heat map produced by a fine-tuned ViT that has just classified the gland as benign or malignant. The prompt adapter processes this heat map by concatenating it with the original image and passing the four-channel tensor through two convolutional layers with batch normalization and ReLU, then adds the result to the signal fed into SAM's gland prompt encoder. The segmentation branch splits into gland and contour predictions, each with its own prompt encoder and mask decoder, and post-processing removes the predicted contour overlap from the gland mask so adjacent glands stay separate. Training is staged: the classification branch is fine-tuned first, then the shared image encoder, prompt adapter, and gland decoder, then the contour decoder with the image encoder frozen, and a U-Net-style weight map weights the pixel-wise MSE loss.
What would settle it
Fix all training settings and replace the Grad-CAM++ heat map with a constant-valued map of the same size before it enters the prompt adapter; if the scores remain close to the prompted SAM results, the grade signal is not the active ingredient.
Extended reading notes
Core claim
The central claim is that a binary cancer-grade label can carry useful spatial information sufficient to guide a foundation segmentation model. The benign/malignant ViT classifier emits a Grad-CAM++ heat map, and that one-channel map, concatenated with the original image and compressed by two convolutional layers in a prompt adapter, becomes the prompt given to SAM's gland mask decoder. With this grade prompt, prompted SAM-H reaches F1 0.929/0.841, object Dice 0.921/0.881, and object Hausdorff 41.189/74.300 on GlaS test sets A and B, outperforming fine-tuned SAM-H and every GlaS benchmark entry listed in the paper. The paper attributes the gain to the prior knowledge of gland type embedded in the heat map, which lets the segmentation model anticipate the appearance of the target.
Load-bearing premise
The whole improvement rests on the assumption that the benign/malignant heat map, not the extra network layers added at the same time, is what makes segmentation better; the paper does not test this by removing the heat map.
Editorial extensions
If this is right
- Prompted SAM matches or beats unprompted fine-tuned SAM at every scale tested (ViT-B, ViT-L, ViT-H), so the benefit is not tied to one model size.
- The joint gland-and-contour prediction with overlap removal gives a practical way to separate touching glands, a known failure mode in gland segmentation.
- A single forward path yields both a benign/malignant classification and a gland/contour segmentation, so the two clinical tasks are solved together.
- The prompted SAM-H numbers top the GlaS benchmarks listed in the paper, so the grade-prompt recipe is competitive with task-specific gland segmentation networks.
Reading between the lines
- A direct test the paper leaves undone: replace the heat map with a blank or shuffled one while freezing all adapter parameters; if segmentation stays at the prompted level, the grade information is not the cause of the gain.
- The recipe is portable to other histology tasks where a coarse image-level label exists, such as nuclei or tissue-type segmentation, and the paper names cell segmentation as future work.
- Because SAM already supports point, box, and text prompts, the grade heat map could be combined with those prompt types rather than used alone, which may give even stronger guidance without new annotations.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a two-branch model for colorectal gland segmentation evaluated on the GlaS dataset. A ViT classification branch is fine-tuned for benign/malignant gland classification, and a Grad-CAM++ heatmap derived from this classifier is used as a prompt for a modified Segment Anything Model (SAM). The segmentation branch contains a shared image encoder, separate prompt encoders and mask decoders for gland and contour prediction, a trainable prompt adapter, a stepwise training schedule, and contour-based post-processing. The paper claims that the prompted SAM variants improve over fine-tuned SAMs and that the best variant, prompted SAM-H, achieves SOTA results on GlaS. The evidence for these claims is presented in Table 1 (prompted vs. fine-tuned SAM) and Table 2 (comparison with GlaS contest baselines).
Significance. The underlying idea is interesting and potentially useful: using a grade classifier's saliency map as a conditioning signal for SAM is a plausible mechanism for injecting clinical prior knowledge into gland segmentation in histopathology. If the prompt benefit were robustly established, the paper would make a useful contribution to the medical-image segmentation literature. The paper also provides a simple prompt-adapter design and a dual-branch SAM modification that are easy to understand. However, as presented, the central claim is not supported by the experimental evidence: the improvements in Table 1 are small, inconsistent, and reported without uncertainty estimates; no ablation isolates the grade-prompt component; the spatial alignment of the heatmap is underspecified; and the SOTA comparison is limited to dated contest baselines. The significance of the work is therefore conditional on a substantially strengthened experimental evaluation.
major comments (4)
- [§3.3, Table 1] The central quantitative claim is not established because the improvements are inconsistent and no uncertainty is reported. Prompted SAM-H F1 on Test A decreases from 0.932 to 0.929, SAM-L Object Dice on Test B is tied at 0.846, and most other differences are at the third decimal place. With test sets of only 60 and 20 images, and no repeated runs, seeds, confidence intervals, or significance tests, these differences cannot be distinguished from noise. The authors should report multiple runs with means and standard deviations and apply paired statistical tests (e.g., Wilcoxon signed-rank or bootstrap) on object-level metrics.
- [§2.3–§2.7] The experimental design does not isolate the grade prompt as the cause of any observed improvement. The prompted model is not simply fine-tuned SAM plus a prompt; it also adds a trainable prompt adapter (§2.3), a second prompt encoder and mask decoder for contour prediction, a stepwise training schedule (§2.5), and contour-based post-processing that removes overlapping gland and contour regions (§2.7). Any of these components could explain the differences in Table 1. No ablation feeds a blank, constant, shuffled, or adversarial heatmap into the same architecture, so the causal role of the cancer-grade information is not tested. Such ablations are necessary before claiming that cancer grade, rather than added capacity or post-processing, drives the reported gains.
- [§2.2 and §3.2] The alignment between the grade heatmap and SAM's input space is underspecified. The classifier is deit-base-patch16-224, while the SAM input is stated to be 400×400 pixels, and the text only says that the heatmap "maintains the same size as the original input image." The paper must state how the Grad-CAM++ output is resized or generated at 400×400, including how DeiT's positional embeddings are handled. Without this, the method cannot be reproduced, and it is unclear whether the prompt is generated under assumptions compatible with the pretrained ViT or is a coarse upsampling that would not carry boundary-level gland information.
- [Tables 1–2 and §4] The claim of SOTA results is not supported by the comparison in Table 2. The table lists only GlaS contest entries from 2017, and the paper's own conclusion cites Hi-gMISnet with an F1 score of 0.932 on the overall test set, yet this method is not included in the comparison. The authors should either qualify the claim to "improves over the fine-tuned SAM baseline" or extend the comparison to modern methods evaluated under the same protocol and split.
minor comments (3)
- [§2.3] The adapter description is difficult to follow: the text says the heatmap is concatenated with the original image and the four-channel feature is reduced to one channel, then "the heat map is added with the output from the second convolution." Please clarify the exact residual structure and the channel dimensions at each step.
- [§3.1] Hyperparameters such as learning rates, batch size, number of epochs, validation split, and the exact patch-extraction and inference-overlap strategy are not reported, which limits reproducibility.
- [§4] There is a typo in the conclusion: "purposed method" should be "proposed method."
Circularity Check
No significant circularity: the claimed prompt benefit is an empirical comparison, not a derivation that reduces to its own inputs.
full rationale
The paper's central claim is that a Grad-CAM++ heat map from a benign/malignant ViT classifier, used as a prompt to a fine-tuned SAM, improves gland segmentation. This is an empirical result: the heat map is generated from a classifier trained on benign/malignant labels, not derived from the segmentation ground truth, and no segmentation metric is fitted or reverse-engineered into the prompt. The prompt adapter, contour branch, and post-processing introduce architectural changes, but they are not circular with the evaluation. The comparison to fine-tuned SAM is a direct experimental comparison on the GlaS test sets, and while the reported gains are small and the prompt benefit is not isolated by an ablation, that is a question of experimental rigor, not circularity. There are no self-citations that carry the argument, no imported uniqueness theorems, and no fitted parameter renamed as a prediction. The derivation is self-contained as an empirical study, so the circularity score is 0.
Assumptions & free parameters
free parameters (2)
- Binary threshold =
0.5
- Input patch size =
400x400
assumptions (3)
- domain assumption Benign and malignant glands have sufficiently distinct morphology that grade information can guide segmentation.
- domain assumption Grad-CAM++ heat maps localize the image regions responsible for classification and are spatially aligned to the SAM input.
- domain assumption Fine-tuned SAM weights provide a useful initialization for the modified segmentation branch.
Cite this review
Pith. "Pith review of Gland Segmentation Using SAM With Cancer Grade as a Prompt." pith.science (2026). https://pith.science/paper/Q7YJ5GC6
@misc{pith2026250114718,
author = {Pith},
title = {Pith review of: Gland Segmentation Using SAM With Cancer Grade as a Prompt},
year = {2026},
howpublished = {\url{https://pith.science/paper/Q7YJ5GC6}},
note = {Machine review of arXiv:2501.14718}
}
read the original abstract
Cancer grade is a critical clinical criterion that can be used to determine the degree of cancer malignancy. Revealing the condition of the glands, a precise gland segmentation can assist in a more effective cancer grade classification. In machine learning, binary classification information about glands (i.e., benign and malignant) can be utilized as a prompt for gland segmentation and cancer grade classification. By incorporating prior knowledge of the benign or malignant classification of the gland, the model can anticipate the likely appearance of the target, leading to better segmentation performance. We utilize Segment Anything Model to solve the segmentation task, by taking advantage of its prompt function and applying appropriate modifications to the model structure and training strategies. We improve the results from fine-tuned Segment Anything Model and produce SOTA results using this approach.
Reference graph
Works this paper leans on
-
[1]
A large proportion of col- orectal cancers are classified as adenocarcinomas [1]
INTRODUCTION Colorectal cancer is one of the most common cancer types, with a notably high mortality rate. A large proportion of col- orectal cancers are classified as adenocarcinomas [1]. Col- orectal adenocarcinoma is distinguished by glandular forma- tion. Pathologists rely on the morphology of glands and the degree of glandular formation as factors to...
-
[2]
Gland Segmentation Using SAM With Cancer Grade as a Prompt
METHODS The proposed model shown in Fig. 1 is designed to generate predictions for both cancer classification and gland segmen- tation simultaneously through two branches. arXiv:2501.14718v2 [eess.IV] 27 Jan 2025 Fig. 1 . The classification branch determines whether the gland is benign or malignant and generates a heat map. The prompt adapter processes th...
work page Pith review arXiv 2025
-
[3]
EXPERIMENT AND RESULTS 3.1. Data Preparation The open-source Gland Segmentation Challenge (GlaS) [15] dataset was used in this study. There are 85 images in the training set. There are 60 and 20 images in test sets A and B. A square with a size of 400 ×400 pixels was set as the in- put for the model. The images in GlaS dataset are all larger than this sta...
-
[4]
A new method to provide grade prompt is designed
CONCLUSION As benign and malignant glands exhibit varying morpholog- ical characteristics, a deep learning model can take advan- tage of cancer grade information as prior knowledge. A new method to provide grade prompt is designed. Heat maps high- lighting regions generated by Grad-CAM++ serve as a prompt Table 2. Comparison of Metrics with GlaS [15] Benc...
-
[5]
is dedicated to the tasks of image segmentation. Primar- ily trained on a broad dataset, it demonstrates impressive zero-shot performance involving natural images. However, when applied to more specialized domains, such as medical imaging, this zero-shot performance tends to decline [6]. De- spite this, the strong generalization ability of SAM provides a ...
-
[6]
COMPLIANCE WITH ETHICAL STANDARDS The research was conducted using publicly available GlaS Challenge data set [15]
-
[7]
ACKNOWLEDGMENT Yijie Zhu is funded by China Scholarship Council - Univer- sity of Warwick Scholarship
-
[8]
General insight into cancer: An overview of colorectal cancer,
Shiekhah Mohammad Alzahrani, Huda Abdulaziz Al Doghaither, and Ayat Badr Al-Ghafari, “General insight into cancer: An overview of colorectal cancer,” Molecular and clinical oncology, vol. 15, no. 6, pp. 271, 2021
work page 2021
Show all 25 references
-
[9]
Colorectal carcinoma: Pathologic aspects,
Matthew Fleming, Sreelakshmi Ravula, Sergei F Tatishchev, and Hanlin L Wang, “Colorectal carcinoma: Pathologic aspects,” Journal of gastrointestinal oncol- ogy, vol. 3, no. 3, pp. 153, 2012
2012
-
[10]
Mild-net: Minimal information loss dilated network for gland instance segmentation in colon histology images,
Simon Graham et al., “Mild-net: Minimal information loss dilated network for gland instance segmentation in colon histology images,” Medical image analysis, vol. 52, pp. 199–211, 2019
2019
-
[11]
Gcsba-net: Gabor-based and cascade squeeze bi- attention network for gland segmentation,
Zhijie Wen, Ru Feng, Jingxin Liu, Ying Li, and Shihui Ying, “Gcsba-net: Gabor-based and cascade squeeze bi- attention network for gland segmentation,” IEEE Jour- nal of Biomedical and Health Informatics , vol. 25, no. 4, pp. 1185–1196, 2021
2021
-
[12]
Segment anything,
Alexander Kirillov et al., “Segment anything,” in Pro- ceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2023, pp. 4015–4026
2023
-
[13]
Segment anything model (sam) for digital pathology: Assess zero-shot segmentation on whole slide imaging,
Ruining Deng et al., “Segment anything model (sam) for digital pathology: Assess zero-shot segmentation on whole slide imaging,”arXiv preprint arXiv:2304.04155, 2023
2023 arXiv
-
[14]
Segment anything in medical images,
Jun Ma et al., “Segment anything in medical images,” Nature Communications, vol. 15, no. 1, pp. 654, 2024
2024
-
[15]
Medical sam adapter: Adapting seg- ment anything model for medical image segmentation,
Junde Wu et al., “Medical sam adapter: Adapting seg- ment anything model for medical image segmentation,” arXiv preprint arXiv:2304.12620, 2023
2023 arXiv
-
[16]
Segment anything model for medical image analysis: an experimental study,
Maciej A Mazurowski et al., “Segment anything model for medical image analysis: an experimental study,” Medical Image Analysis, vol. 89, pp. 102918, 2023
2023
-
[17]
Glandular morphometrics for objective grading of colorectal adenocarcinoma histol- ogy images,
Ruqayya Awan et al., “Glandular morphometrics for objective grading of colorectal adenocarcinoma histol- ogy images,” Scientific reports, vol. 7, no. 1, pp. 16852, 2017
2017
-
[18]
Cgs-net: Classification-guided segmentation network for improved gland segmenta- tion,
Xiaoheng Tang et al., “Cgs-net: Classification-guided segmentation network for improved gland segmenta- tion,” in 2023 IEEE 12th Data Driven Control and Learning Systems Conference (DDCLS) . IEEE, 2023, pp. 1008–1013
2023
-
[19]
An image is worth 16x16 words: Transformers for image recognition at scale,
Alexey Dosovitskiy et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” 2021
2021
-
[20]
Grad- cam++: Generalized gradient-based visual explanations for deep convolutional networks,
Aditya Chattopadhay, Anirban Sarkar, Prantik Howlader, and Vineeth N Balasubramanian, “Grad- cam++: Generalized gradient-based visual explanations for deep convolutional networks,” in 2018 IEEE winter conference on applications of computer vision (WACV). IEEE, 2018, pp. 839–847
2018
-
[21]
U-net: Convolutional networks for biomedical image segmentation,
Olaf Ronneberger, Philipp Fischer, and Thomas Brox, “U-net: Convolutional networks for biomedical image segmentation,” 2015
2015
-
[22]
Gland segmentation in colon histology images: The glas challenge contest,
Korsuk Sirinukunwattana et al., “Gland segmentation in colon histology images: The glas challenge contest,” Medical image analysis, vol. 35, pp. 489–502, 2017
2017
-
[23]
Training data-efficient image transformers & distillation through attention,
Hugo Touvron et al., “Training data-efficient image transformers & distillation through attention,” in Inter- national conference on machine learning. PMLR, 2021, pp. 10347–10357
2021
-
[24]
Hi- gmisnet: generalized medical image segmentation us- ing dwt based multilayer fusion and dual mode atten- tion into high resolution pgan,
Tushar Talukder Showrav and Md Kamrul Hasan, “Hi- gmisnet: generalized medical image segmentation us- ing dwt based multilayer fusion and dual mode atten- tion into high resolution pgan,” Physics in Medicine & Biology, vol. 69, no. 11, pp. 115019, 2024
2024
-
[25]
Pannuke: an open pan-cancer histology dataset for nuclei instance segmentation and classification,
Jevgenij Gamper, Navid Alemi Koohbanani, Ksenija Benet, Ali Khuram, and Nasir Rajpoot, “Pannuke: an open pan-cancer histology dataset for nuclei instance segmentation and classification,” in Digital Pathology: 15th European Congress, ECDP 2019, Warwick, UK, April 10–13, 2019, ...
2019
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.