Pith. sign in

REVIEW 4 major objections 6 minor 31 references

Colorectal Cancer Tumor Grade Segmentation in Digital Histopathology Images: From Giga to Mini Challenge

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Pith's one-line read This paper reports that an open challenge on the CCTGS colorectal cancer dataset raised the best automated tumor grade segmentation macro F-score from 62.92 to 70.2, with the gain driven mainly by full-resolution whole-slide input and…

desk verdict Useful benchmark report with a believable new best score, but the headline gain lacks statistical support and the annotation-style confounder is real. read the letter →

arxiv 2507.04681 v3 pith:7AZ7F43P submitted 2025-07-07 cs.CV

classification cs.CV
keywords colorectalcancertumorgradesegmentationwhole-slideimagesdigitalhistopathologysemanticchallengemodelensemblingmacroF-score
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper reports the results of an open challenge in which 39 teams built algorithms to segment colorectal cancer tumor grades directly from whole-slide histopathology images. The task matters because tumor grade guides prognosis and treatment, yet routine grading is subjective and pathologists are scarce. The winning pipeline, a Visual Attention Network paired with an UperNet decoder, reached a macro F-score of 70.2 on the held-out test set, clearly above the previous best of 62.92 recorded with a Swin Transformer baseline. The challenge also found that the two submissions using original high-resolution whole-slide images outperformed every method that trained on downsampled images, and that voting ensembles beat single models. These results establish a new reference point for automated colorectal cancer grading on this benchmark.

What carries the argument

The load-bearing mechanism is the challenge evaluation itself: 103 whole-slide images split into training, validation, and test sets, hidden pixel-level test labels, and ranking by macro F-score across five classes, with mandatory containerized code submission so results are reproducible. Inside the winning method, the operative components are an active cropping scheme that samples 512-by-512 patches in proportion to class-weighted pixel content; a three-fold ensemble of Visual Attention Network backbones with UperNet decoders trained with Dice plus cross-entropy loss; overlapping 1024-by-1024 inference patches weighted by a Gaussian kernel and averaged across the three models; and a post-processing step that redraws external contours to fill holes and repair fragmented shapes.

What would settle it

Have a second expert pathologist independently re-annotate the held-out test whole-slide images for the five tissue classes, then rescore the winning and baseline masks against those fresh labels; if the macro F-score gap between the winner and the Swin Transformer baseline shrinks or reverses, the reported improvement is substantially an artifact of annotation style rather than true segmentation quality.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central claim is that the CCTGS benchmark has a new best result: 70.2 macro F-score, achieved by averaging three Visual Attention Network plus UperNet models trained on full-resolution whole-slide images with class-weighted active cropping. Six independently submitted methods exceeded the previous Swin Transformer baseline of 62.92, and the two highest-scoring methods were the only ones to use the original whole-slide files, finishing at least three points ahead of every downsampled-only method. The paper interprets this as evidence that high-resolution morphological detail and multi-model voting are the main levers for tumor grade segmentation accuracy, while noting that distinguishing Grade 1 from Grade 2 remains the dominant residual error.

Load-bearing premise

The ranking depends on the hidden test labels being accurate and annotated in a style consistent with the training labels; the paper itself reports that training masks split into smoothed-boundary and detailed-edge styles, so if the same style gap reaches the test set, the point differences between methods could reflect annotation style rather than segmentation skill.

Editorial extensions

If this is right

  • Using original whole-slide images instead of downsampled copies is worth at least three macro F-score points on this benchmark, since both of the top two methods used the original files and outscored every downsampled-only method.
  • Ensembling by voting, whether hard majority voting on masks or soft averaging of probability maps, consistently improves test performance over any single model.
  • The residual error pattern is concentrated in Grade 1 versus Grade 2 confusion, while Grade 3 detection and normal-mucosa boundary fidelity are comparatively strong in the winning method.
  • The public dataset and evaluation server leave a fixed benchmark that future tumor grade segmentation methods can be measured against directly.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the reported advantage of full-resolution whole-slide images is observational rather than controlled, because the teams that chose to process the large files also invested more compute and engineering; the causal contribution of image resolution alone is not separated in the challenge data.
  • Editorial inference: the paper's observation of two annotation styles, smoothed boundaries versus highly detailed edges, implies that a future re-annotation with consensus labels could change absolute scores and possibly the ranking, not just the top-line number.
  • Editorial inference: because the hardest residual errors are between adjacent tumor grades, an architecture or loss function that treats grades as ordered, rather than as independent classes, is a concrete and testable next step.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper reports on the ICIP Grand Challenge on Colorectal Cancer Tumor Grading and Segmentation, using the METU CCTGS dataset of 103 whole-slide images with five-class expert pixel-level annotations. The authors describe the challenge protocol, the top six participating methods, and their evaluation results. The headline result is that the winning method (VAN+UperNet) achieves a macro F-score of 70.2, outperforming the Swin Transformer baseline of 62.92. The paper also observes that the top two teams used full-resolution WSIs and ensembles and that ensembling generally helps. The dataset and evaluation infrastructure are made publicly available, and Docker-based reproducibility checks were performed.

Significance. If the reported results are reliable, the paper provides a useful public benchmark for tumor grade segmentation, with a well-defined challenge protocol, a moderately large multi-class histopathology dataset, and reproducible Docker submissions from six methods. The explicit release of the dataset and evaluation infrastructure is a valuable community contribution. However, the headline performance claims currently rest on a small hidden test set (about 15 WSIs) with no uncertainty quantification, and the paper itself acknowledges an annotation-style inconsistency that could bias comparisons. The significance of the benchmark would be substantially strengthened by addressing these statistical and annotation-robustness issues.

major comments (4)
  1. [Section 3, Table 1] The headline claim that the winning method outperforms the Swin Transformer baseline by 7.3 macro-F points rests on a single aggregate score per method. The test split is 15% of 103 WSIs (approximately 15 slides), and macro-F over five classes is highly sensitive to one or two atypical slides. The paper should report per-slide macro-F scores, their standard deviation or confidence intervals, and a paired significance test (e.g., Wilcoxon signed-rank) comparing the top methods against the baseline. Without this, the reported advantage is not distinguished from sampling variability.
  2. [Section 3] The paper explicitly acknowledges that the training annotations exhibit two distinct styles — one with smoothed boundaries and one with highly detailed edges — and states that this complicates model evaluation. However, it does not check whether the stratified random split balances these styles across training, validation, and test sets. If the hidden test set is dominated by one annotation style, the ranking of methods could reflect style-specific bias rather than grading skill. The authors should quantify the annotation-style distribution across splits and, if imbalanced, evaluate robustness (e.g., per-style scores or agreement with a re-annotated subset).
  3. [Section 3] The statement that using full-resolution WSIs provides a 'clear benefit' is confounded with ensembling: the top two teams used both WSIs and ensembles, while the third team used ensembling without WSIs. Without an ablation that varies resolution while holding the model and ensemble strategy fixed (e.g., running the winning pipeline on downsampled images), the observed margin cannot be attributed to WSI resolution. The paper should either provide such an ablation or present this observation as a hypothesis rather than a conclusion.
  4. [Section 1 and Table 1] The baseline Swin Transformer score (62.92) is cited from the METU CCTGS dataset paper [11], but the present paper does not state whether this baseline was trained on the same 70/15/15 split and evaluated on the same hidden test set used in the challenge. If the baseline was produced under a different data partition or preprocessing, the comparison in Table 1 is not apples-to-apples. Please specify the baseline's training and evaluation protocol, or rerun the baseline under the challenge protocol to confirm the comparison.
minor comments (6)
  1. [Section 2.2] The hyperparameters of the Top-N Soft Biased Voting method (N, M, and the class-specific bias vector) and the area threshold epsilon in the postprocessing pipeline are not specified; please report their values and how they were chosen, to ensure reproducibility.
  2. [Table 1] Reporting only macro-averaged metrics hides class-level performance; for a tumor grading task, per-grade F-scores (Grade 1/2/3, normal, others) are essential for assessing clinically relevant confusion patterns and should be included.
  3. [Section 1] The stratification is described as 'pixel-level' class balance; please clarify whether this is computed across all pixels in the dataset or per-slide, and report the class distribution for each split.
  4. [Sections 2.1 and 2.2] The method names 'V AN+UperNet' and 'DPT+MaxViT' contain typographic spaces; please use consistent formatting throughout the paper.
  5. [Section 3] The qualitative analysis of Figure 2 would benefit from a quantitative confusion-matrix analysis, since subjective claims such as 'VAN+UperNet has high precision' are not directly traceable to Table 1.
  6. [Section 1] The paper does not state how many Codalab submissions were allowed or whether participants could see test-set metrics during the development phase; if repeated submissions were permitted, the risk of test-set overfitting should be discussed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the challenge results rest on held-out test evaluation, not on fitted inputs or self-cited derivation.

full rationale

This paper is a challenge overview and benchmark report rather than a derivation. The reported macro-F scores are obtained from a hidden Codalab test set with held-out labels, and the top methods were selected by external scoring on that test set. No parameter is fitted to the test labels and then renamed a prediction; the method descriptions are independent of the reported outcome. The only self-citation is the Swin Transformer baseline from the authors' own METU CCTGS dataset paper [11], but that baseline is used purely as a comparison point and is not load-bearing for any claimed derivation. The observation that the top two teams used whole-slide images is a post-hoc correlation from the results, not a result forced by construction. There are no equations, uniqueness theorems, or ansatz-smuggling citations that reduce the claims to their inputs. The acknowledged annotation-style inconsistency and the lack of confidence intervals are validity and robustness concerns, not circularity. Therefore the paper is not circular.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The paper makes no theoretical derivation; it reports empirical benchmark results. Its assumptions are mainly about data quality and evaluation design.

free parameters (2)
  • Post-processing area threshold epsilon = unspecified
    In Section 2.2, DPT+MaxViT removes connected components with area below a dataset-specific threshold epsilon; the value is not given.
  • Soft voting bias vector for tumor classes = unspecified
    In Section 2.2, a bias vector is added to favor tumor-related classes in soft voting; values are not disclosed.
assumptions (3)
  • domain assumption Expert pixel-level annotations in the METU CCTGS dataset are correct and consistent for grading tasks.
    The entire evaluation relies on these labels as ground truth; the paper itself notes annotation style inconsistency (Section 3).
  • domain assumption Macro F-score is the appropriate ranking metric for this 5-class segmentation task.
    The challenge chose this metric; it treats all classes equally, which may not align with clinical priorities.
  • domain assumption The hidden test set is representative of the dataset distribution and large enough for meaningful comparison.
    The test set contains about 15 WSIs (15% of 103); no confidence intervals are given.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Colorectal Cancer Tumor Grade Segmentation in Digital Histopathology Images: From Giga to Mini Challenge." pith.science (2026). https://pith.science/paper/7AZ7F43P

@misc{pith2026250704681,
  author       = {Pith},
  title        = {Pith review of: Colorectal Cancer Tumor Grade Segmentation in Digital Histopathology Images: From Giga to Mini Challenge},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7AZ7F43P}},
  note         = {Machine review of arXiv:2507.04681}
}
read the original abstract

Colorectal cancer (CRC) is the third most diagnosed cancer and the second leading cause of cancer-related death worldwide. Accurate histopathological grading of CRC is essential for prognosis and treatment planning but remains a subjective process prone to observer variability and limited by global shortages of trained pathologists. To promote automated and standardized solutions, we organized the ICIP Grand Challenge on Colorectal Cancer Tumor Grading and Segmentation using the publicly available METU CCTGS dataset. The dataset comprises 103 whole-slide images with expert pixel-level annotations for five tissue classes. Participants submitted segmentation masks via Codalab, evaluated using metrics such as macro F-score and mIoU. Among 39 participating teams, six outperformed the Swin Transformer baseline (62.92 F-score). This paper presents an overview of the challenge, dataset, and the top-performing methods

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

31 extracted references · 29 canonical work pages

  1. [11]

    The eighth edition ajcc cancer staging manual: continuing to build a bridge from a population-based to a more “personalized

    Mahul B Amin, Frederick L Greene, Stephen B Edge, Carolyn C Compton, Jeffrey E Gershenwald, Robert K Brookland, Laura Meyer, Donna M Gress, David R Byrd, and David P Winchester, “The eighth edition ajcc cancer staging manual: continuing to build a bridge from a population-based to a more “personalized” ap- proach to cancer staging,”CA: a cancer journal fo...

  2. [1]

    INTRODUCTION Colorectal cancer (CRC) represents the third most frequently diagnosed malignancy globally, accounting for over 1.8 mil- lion new cases annually [1], and stands as the second lead- ing cause of cancer-related mortality worldwide [2]. Epi- demiological projections anticipate a significant increase in 1This work has been supported by Middle Eas...

  3. [2]

    HardNet+Lawin

    METHODS This section summarizes the top six methods, in the order of decreasing performance. The methods range from novel com- binations of different models to careful fine-tuning of already existing models. Notably, only the top two teams utilized the full-resolution whole-slide images (i.e., large SVS files), and their performance was significantly bett...

  4. [3]

    RESULTS Performances of the six contributed methods can be seen in Table 1. Although the dataset included large whole-slide im- ages (WSIs), only two teams used them—likely because han- dling and processing such high-resolution files is technically challenging. However, we see a clear benefit of using WSIs on test metrics. The top two teams both utilized ...

  5. [4]

    The challenge attracted broad participation, with 39 teams developing segmentation models for complex multi-class tissue grading and segmentation under limited supervision

    CONCLUSION The ICIP Grand Challenge on Colorectal Cancer Tumor Grading and Segmentation aimed to address the pressing need for accurate models for tumor grading and segmen- tation in histopathology by benchmarking algorithms on a realistic dataset. The challenge attracted broad participation, with 39 teams developing segmentation models for complex multi-...

  6. [5]

    Global cancer statistics 2020: Globocan estimates of incidence and mortality world- wide for 36 cancers in 185 countries,

    Hyuna Sung, Jacques Ferlay, Rebecca L Siegel, Math- ieu Laversanne, Isabelle Soerjomataram, Ahmedin Je- mal, and Freddie Bray, “Global cancer statistics 2020: Globocan estimates of incidence and mortality world- wide for 36 cancers in 185 countries,”CA: a cancer journal for clinicians, vol. 71, no. 3, pp. 209–249, 2021

  7. [6]

    Colorectal cancer statistics, 2020,

    Rebecca L Siegel, Kimberly D Miller, Ann God- ing Sauer, Stacey A Fedewa, Lynn F Butterly, Joseph C Anderson, Andrea Cercek, Robert A Smith, and Ahmedin Jemal, “Colorectal cancer statistics, 2020,” CA: a cancer journal for clinicians, vol. 70, no. 3, pp. 145–164, 2020

  8. [7]

    Global colorectal cancer bur- den in 2020 and projections to 2040,

    Yue Xi and Pengfei Xu, “Global colorectal cancer bur- den in 2020 and projections to 2040,”Translational on- cology, vol. 14, no. 10, pp. 101174, 2021

Show all 31 references
  1. [8]

    The consensus molecular subtypes of colorectal cancer,

    Justin Guinney, Rodrigo Dienstmann, Xin Wang, Aur´elien De Reynies, Andreas Schlicker, Charlotte Soneson, Laetitia Marisa, Paul Roepman, Gift Nyamun- danda, Paolo Angelino, et al., “The consensus molecular subtypes of colorectal cancer,”Nature medicine, vol. 21, no. 11, pp. 13...

  2. [9]

    Histological grade predicts survival time asso- ciated with recurrence after resection for colorectal can- cer.,

    Yong Beom Cho, Ho-Kyung Chun, Hae Ran Yun, Hee Cheol Kim, Seong Hyeon Yun, and Woo Yong Lee, “Histological grade predicts survival time asso- ciated with recurrence after resection for colorectal can- cer.,”Hepato-gastroenterology, vol. 56, no. 94-95, pp. 1335–1340, 2009

  3. [10]

    Interobserver agree- ment in grading of colorectal cancers—findings from a nationwide web-based survey of histopathologists,

    IP Chandler and RS Houlston, “Interobserver agree- ment in grading of colorectal cancers—findings from a nationwide web-based survey of histopathologists,” Histopathology, vol. 52, no. 4, pp. 494–499, 2008

  4. [12]

    Artificial intelligence for solid tu- mour diagnosis in digital pathology,

    Christophe Klein, Qinghe Zeng, Floriane Arbaretaz, Es- telle Devˆevre, Julien Calderaro, Nicolas Lomenie, and Maria Chiara Maiuri, “Artificial intelligence for solid tu- mour diagnosis in digital pathology,”British Journal of Pharmacology, vol. 178, no. 21, pp. 4291–4315, 2021

  5. [13]

    Observer variation in the histolog- ical grading of rectal carcinoma.,

    GD Thomas, MF Dixon, NC Smeeton, and NS Williams, “Observer variation in the histolog- ical grading of rectal carcinoma.,”Journal of Clinical pathology, vol. 36, no. 4, pp. 385–391, 1983

  6. [14]

    Constant demand, patchy supply,

    A Bychkov and M Schubert, “Constant demand, patchy supply,”Pathologist, vol. 88, no. 88, pp. 18–27, 2023

  7. [15]

    Colorectal cancer tumor grade segmentation: A new dataset and baseline results,

    Duygu Arslan, Sina Sehlaver, Erce Guder, Mehmet Arda Temena, Alper Bahcekapili, Umut Ozdemir, Duriye Ozer Turkay, Gunes Guner, Servet Guresci, Cenk Sokmensuer, Emre Akbas, and Ahmet Acar, “Colorectal cancer tumor grade segmentation: A new dataset and baseline results,”Heliyon,...

  8. [16]

    Unified perceptual parsing for scene un- derstanding,

    Tete Xiao, Yingcheng Liu, Bolei Zhou, Yuning Jiang, and Jian Sun, “Unified perceptual parsing for scene un- derstanding,” inProceedings of the European confer- ence on computer vision (ECCV), 2018, pp. 418–434

  9. [17]

    Visual attention net- work,

    Meng-Hao Guo, Cheng-Ze Lu, Zheng-Ning Liu, Ming- Ming Cheng, and Shi-Min Hu, “Visual attention net- work,”Computational visual media, vol. 9, no. 4, pp. 733–752, 2023

  10. [18]

    Fully convolutional networks for semantic segmenta- tion,

    Jonathan Long, Evan Shelhamer, and Trevor Darrell, “Fully convolutional networks for semantic segmenta- tion,” inProceedings of the IEEE conference on com- puter vision and pattern recognition, 2015, pp. 3431– 3440

  11. [19]

    Towards robust monoc- ular depth estimation: Mixing datasets for zero-shot cross-dataset transfer,

    Ren ´e Ranftl, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun, “Towards robust monoc- ular depth estimation: Mixing datasets for zero-shot cross-dataset transfer,”IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2020

  12. [20]

    MaxViT: Multi-axis vision transformer,

    Zhengzhong Tu, Hossein Talebi, Han Zhang, Feng Yang, Peyman Milanfar, Alan Bovik, and Yinxiao Li, “MaxViT: Multi-axis vision transformer,”ECCV, 2022

  13. [21]

    Adaptive augmen- tation policy optimization with LLM feedback,

    Ant Duru and Alptekin Temizel, “Adaptive augmen- tation policy optimization with LLM feedback,”arXiv preprint arXiv:2410.13453, 2024

  14. [22]

    Sur- vey: interpolation methods for whole slide image pro- cessing,

    L Roszkowiak, A Korzynska, J Zak, Dorota Pijanowska, Z Swiderska-Chadaj, and Tomasz Markiewicz, “Sur- vey: interpolation methods for whole slide image pro- cessing,”Journal of microscopy, vol. 265, no. 2, pp. 148–158, 2017

  15. [23]

    Hardnet-dfus: Enhancing backbone and decoder of hardnet-mseg for diabetic foot ulcer image segmentation,

    Ting-Yu Liao, Ching-Hui Yang, Yu-Wen Lo, Kuan-Ying Lai, Po-Huai Shen, and Youn-Long Lin, “Hardnet-dfus: Enhancing backbone and decoder of hardnet-mseg for diabetic foot ulcer image segmentation,” inDiabetic Foot Ulcers Grand Challenge, pp. 21–30. Springer, 2022

  16. [24]

    Hardnet: A low memory traffic network,

    Ping Chao, Chao-Yang Kao, Yushan Ruan, Chien- Hsiang Huang, and Youn-Long Lin, “Hardnet: A low memory traffic network,” in2019 IEEE/CVF Interna- tional Conference on Computer Vision (ICCV), 2019, pp. 3551–3560

  17. [25]

    Lawin transformer: Improving semantic segmentation trans- former with multi-scale representations via large win- dow attention,

    Haotian Yan, Chuang Zhang, and Ming Wu, “Lawin transformer: Improving semantic segmentation trans- former with multi-scale representations via large win- dow attention,”arXiv preprint arXiv:2201.01615, 2022

  18. [26]

    100,000 histological images of human colorec- tal cancer and healthy tissue,

    Jakob Nikolas Kather, Niels Halama, and Alexander Marx, “100,000 histological images of human colorec- tal cancer and healthy tissue,” Apr. 2018

  19. [27]

    Nct-crc-he: Not all histopathological datasets are equally useful,

    Andrey Ignatov and Grigory Malivenko, “Nct-crc-he: Not all histopathological datasets are equally useful,” inEuropean Conference on Computer Vision (ECCV 2024), 2024

  20. [28]

    Overcoming catastrophic forgetting in neural networks,

    James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell, “Overcoming catastrophic forgetting ...

  21. [29]

    MMSegmentation: Openmmlab semantic segmentation toolbox and bench- mark,

    MMSegmentation Contributors, “MMSegmentation: Openmmlab semantic segmentation toolbox and bench- mark,”https://github.com/open-mmlab/ mmsegmentation, 2020

  22. [30]

    Towards a general-purpose foundation model for computational pathology,

    Richard J Chen, Tong Ding, Ming Y Lu, Drew FK Williamson, Guillaume Jaume, Bowen Chen, Andrew Zhang, Daniel Shao, Andrew H Song, Muhammad Sha- ban, et al., “Towards a general-purpose foundation model for computational pathology,”Nature Medicine, 2024

  23. [31]

    Vision trans- former adapter for dense predictions,

    Zhe Chen, Yuchen Duan, Wenhai Wang, Junjun He, Tong Lu, Jifeng Dai, and Yu Qiao, “Vision trans- former adapter for dense predictions,”arXiv preprint arXiv:2205.08534, 2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.