pith. sign in

arxiv: 1907.09478 · v1 · pith:I6YTFIZ7new · submitted 2019-07-22 · 📡 eess.IV · cs.LG· stat.ML

Context-Aware Convolutional Neural Network for Grading of Colorectal Cancer Histology Images

Muhammad Shaban , Ruqayya Awan , Muhammad Moazam Fraz , Ayesha Azam , David Snead , Nasir M. Rajpoot This is my paper

Pith reviewed 2026-05-24 17:43 UTC · model grok-4.3

classification 📡 eess.IV cs.LGstat.ML
keywords colorectal cancerhistology imagesconvolutional neural networkcontext-awarecancer gradingfeature aggregationdigital pathologybreast cancer classification
0
0 comments X

The pith

A neural network processes 1792x1792 histology images by extracting local features then aggregating them spatially to grade colorectal cancer more accurately than patch-based methods.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces a context-aware CNN designed to handle much larger image regions than the small patches typical in standard CNN analysis of histology slides. It first learns detailed local representations from the full 1792x1792 pixel image and then combines those representations while preserving their spatial layout before making a grading decision. This design is tested on colorectal cancer grading and breast cancer classification tasks. The authors report that the approach produces higher accuracy than traditional patch-based CNNs, problem-specific techniques, and prior context-based methods, with a measured improvement of 3.61 percent. If the reported gain stems from the spatial aggregation step, it indicates that position-aware combination of high-dimensional features supplies diagnostic signals not captured by independent patch processing.

Core claim

The central claim is that a context-aware neural network can incorporate high-resolution contextual information by first encoding the local representation of a 1792x1792 histology image into high-dimensional features and then aggregating those features according to their spatial organization, yielding more accurate predictions for colorectal cancer grading and breast cancer classification than patch-based or existing context-based alternatives.

What carries the argument

Context-aware neural network that encodes local representations from large images into high-dimensional features and aggregates them by spatial organization.

If this is right

  • The spatial aggregation of high-dimensional features from 1792x1792 images produces higher grading accuracy than processing independent small patches.
  • The same architecture improves performance on both colorectal cancer grading and breast cancer classification tasks.
  • The method exceeds the accuracy of traditional patch-based CNNs as well as existing context-based and problem-specific approaches by a margin of 3.61%.
  • Larger image sizes become feasible for CNN analysis without discarding spatial relationships among local features.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If spatial aggregation is the decisive factor, then other position-encoding mechanisms such as coordinate channels or attention maps might achieve comparable gains with lower computational cost.
  • The approach could be extended to additional high-resolution medical imaging domains where tissue architecture spans scales larger than a single patch.
  • Adopting this workflow might reduce reliance on exhaustive patch sampling and manual region selection in digital pathology pipelines.

Load-bearing premise

The spatial aggregation step on features from the large images supplies diagnostic information unavailable to standard small-patch processing and is the main reason for any observed accuracy gain.

What would settle it

Training and testing both the proposed context-aware model and a standard small-patch CNN on identical colorectal cancer histology data and obtaining equal or lower accuracy for the context-aware version.

Figures

Figures reproduced from arXiv: 1907.09478 by Ayesha Azam, David Snead, Muhammad Moazam Fraz, Muhammad Shaban, Nasir M. Rajpoot, Ruqayya Awan.

Figure 1
Figure 1. Figure 1: Three visual field regions of colorectal tissue which highlight the importance of larger context for correct grading. Each cell of the overlaid grid [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Flow diagram of the proposed context-aware framework for CRC grading. The top row shows the local representation learning. The bottom row [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: (Left) Results of 24 experiments using best performing local representation features (Xception). (Right) Legend represents the feature pooling type, [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Visual results of CRC grading are shown for patch classifier, existing context, and the proposed method on an image of size [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
read the original abstract

Digital histology images are amenable to the application of convolutional neural network (CNN) for analysis due to the sheer size of pixel data present in them. CNNs are generally used for representation learning from small image patches (e.g. 224x224) extracted from digital histology images due to computational and memory constraints. However, this approach does not incorporate high-resolution contextual information in histology images. We propose a novel way to incorporate larger context by a context-aware neural network based on images with a dimension of 1,792x1,792 pixels. The proposed framework first encodes the local representation of a histology image into high dimensional features then aggregates the features by considering their spatial organization to make a final prediction. The proposed method is evaluated for colorectal cancer grading and breast cancer classification. A comprehensive analysis of some variants of the proposed method is presented. Our method outperformed the traditional patch-based approaches, problem-specific methods, and existing context-based methods quantitatively by a margin of 3.61%. Code and dataset related information is available at this link: https://tia-lab.github.io/Context-Aware-CNN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 3 minor

Summary. The paper proposes a context-aware CNN architecture that processes 1792×1792 histology images by first encoding local representations into high-dimensional features and then performing spatial aggregation of those features for final prediction. It evaluates the approach on colorectal cancer grading and breast cancer classification tasks, analyzes several variants of the method, and reports a 3.61% quantitative improvement over traditional patch-based CNNs, problem-specific methods, and prior context-based approaches. Code and dataset links are provided.

Significance. If the experimental results hold under rigorous validation, the work offers a practical route to incorporating larger spatial context in computational pathology without the memory costs of full whole-slide processing. The explicit variant analysis and public code release strengthen reproducibility and allow direct testing of whether spatial aggregation is the primary driver of gains.

major comments (2)
  1. [§4] §4 (Experiments), Table 2/3: the central claim of a 3.61% margin over context-based baselines is presented without reported standard deviations across folds, p-values, or confidence intervals; this is load-bearing because the abstract and conclusion treat the numeric margin as evidence of superiority.
  2. [§3.2] §3.2 (Spatial aggregation): the description of feature aggregation does not include an explicit ablation that isolates the spatial-organization step from other design choices (e.g., input resolution or backbone depth), leaving open whether the reported gain truly stems from the claimed mechanism.
minor comments (3)
  1. [Abstract] Abstract: the datasets, number of images, and train/test split protocol are not stated, even though the full manuscript later supplies them; adding one sentence would improve standalone readability.
  2. [Figure 3] Figure 3 and §4.1: axis labels and legend entries use inconsistent abbreviations (e.g., “CA-CNN” vs. “Context-Aware”); standardize for clarity.
  3. [§5] §5 (Discussion): the claim that the method “captures additional diagnostic information” is stated without reference to any pathologist review or saliency-map analysis that would support the interpretation.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the positive evaluation and recommendation of minor revision. We address each major comment below and will incorporate the suggested improvements in the revised manuscript.

read point-by-point responses

  1. Referee: [§4] §4 (Experiments), Table 2/3: the central claim of a 3.61% margin over context-based baselines is presented without reported standard deviations across folds, p-values, or confidence intervals; this is load-bearing because the abstract and conclusion treat the numeric margin as evidence of superiority.

    Authors: We agree that the lack of standard deviations, p-values, or confidence intervals weakens the presentation of the 3.61% margin. The original experiments used a single train/validation/test split. In the revised manuscript we will rerun the key comparisons using 5-fold cross-validation on the colorectal cancer dataset and report mean performance with standard deviations and paired statistical tests against the context-based baselines. revision: yes

  2. Referee: [§3.2] §3.2 (Spatial aggregation): the description of feature aggregation does not include an explicit ablation that isolates the spatial-organization step from other design choices (e.g., input resolution or backbone depth), leaving open whether the reported gain truly stems from the claimed mechanism.

    Authors: The existing variant analysis compares different aggregation functions but does not fully decouple spatial aggregation from resolution and backbone depth. We will add a new controlled ablation in the revised manuscript that fixes input resolution at 1792×1792 and backbone architecture while varying only the presence and type of spatial aggregation; results will be reported alongside the original tables. revision: yes

Circularity Check

0 steps flagged

No significant circularity: empirical architecture comparison

full rationale

The paper presents a CNN architecture for histology grading and reports empirical performance gains on colorectal and breast cancer tasks. No equations, derivations, fitted parameters renamed as predictions, or self-citation chains appear in the abstract or described full text. The central claim rests on experimental comparisons against baselines rather than any self-referential reduction. This matches the default expectation for non-circular empirical ML papers.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the domain assumption that larger spatial context improves grading accuracy and on standard supervised CNN training assumptions; no free parameters or invented entities are identifiable from the abstract alone.

axioms (1)
  • domain assumption CNNs trained on image patches can extract useful local representations that remain informative when aggregated spatially
    Implicit in the two-stage encode-then-aggregate design described in the abstract.

pith-pipeline@v0.9.0 · 5753 in / 1214 out tokens · 46133 ms · 2026-05-24T17:43:44.272445+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 40 canonical work pages · 5 internal anchors

  1. [1]

    Whole slide imaging in pathology: advantages, limitations, and emerging perspectives,

    N. Farahani, A. V . Parwani, and L. Pantanowitz, “Whole slide imaging in pathology: advantages, limitations, and emerging perspectives,” Pathol Lab Med Int , vol. 7, pp. 23–33, 2015. 10

    work page 2015

  2. [2]

    Nuclei detection using mixture density networks,

    N. A. Koohababni, M. Jahanifar, A. Gooya, and N. Rajpoot, “Nuclei detection using mixture density networks,” in International Workshop on Machine Learning in Medical Imaging . Springer, 2018, pp. 241– 248

    work page 2018

  3. [3]

    Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images,

    K. Sirinukunwattana, S. E. A. Raza, Y .-W. Tsang, D. R. Snead, I. A. Cree, and N. M. Rajpoot, “Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images,” IEEE transactions on medical imaging , vol. 35, no. 5, pp. 1196–1206, 2016

    work page 2016

  4. [4]

    Simultaneous cell detection and classification in bone marrow histology images,

    T.-H. Song, V . Sanchez, H. ElDaly, and N. Rajpoot, “Simultaneous cell detection and classification in bone marrow histology images,” IEEE journal of biomedical and health informatics , 2018

    work page 2018

  5. [5]

    Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer,

    B. E. Bejnordi, M. Veta, P. J. Van Diest, B. Van Ginneken, N. Karsse- meijer, G. Litjens, J. A. Van Der Laak, M. Hermsen, Q. F. Manson, M. Balkenhol et al., “Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer,” Jama, vol. 318, no. 22, pp. 2199–2210, 2017

    work page 2017

  6. [6]

    Significance of hyperparameter optimization for metastasis detection in breast histology images,

    N. A. Koohbanani, T. Qaisar, M. Shaban, J. Gamper, and N. Rajpoot, “Significance of hyperparameter optimization for metastasis detection in breast histology images,” in Computational Pathology and Ophthalmic Medical Image Analysis . Springer, 2018, pp. 139–147

    work page 2018

  7. [7]

    Fast scannet: Fast and dense analysis of multi-gigapixel whole-slide images for cancer metastasis detection,

    H. Lin, H. Chen, S. Graham, Q. Dou, N. Rajpoot, and P.-A. Heng, “Fast scannet: Fast and dense analysis of multi-gigapixel whole-slide images for cancer metastasis detection,” IEEE transactions on medical imaging, 2019

    work page 2019

  8. [8]

    Persistent homology for fast tumor segmentation in whole slide histology images,

    T. Qaiser, K. Sirinukunwattana, K. Nakane, Y .-W. Tsang, D. Epstein, and N. Rajpoot, “Persistent homology for fast tumor segmentation in whole slide histology images,” Procedia Computer Science, vol. 90, pp. 119–124, 2016

    work page 2016

  9. [9]

    Automated gleason grading of prostate cancer tissue microarrays via deep learning,

    E. Arvaniti, K. S. Fricker, M. Moret, N. J. Rupp, T. Hermanns, C. Fankhauser, N. Wey, P. J. Wild, J. H. Rueschoff, and M. Claassen, “Automated gleason grading of prostate cancer tissue microarrays via deep learning,” bioRxiv, p. 280024, 2018

    work page 2018

  10. [10]

    Semantic segmentation for prostate cancer grading by convolutional neural networks,

    N. Ing, Z. Ma, J. Li, H. Salemi, C. Arnold, B. S. Knudsen, and A. Gertych, “Semantic segmentation for prostate cancer grading by convolutional neural networks,” in Medical Imaging 2018: Digital Pathology, vol. 10581. International Society for Optics and Photonics, 2018, p. 105811B

    work page 2018

  11. [11]

    Glandular morpho- metrics for objective grading of colorectal adenocarcinoma histology images,

    R. Awan, K. Sirinukunwattana, D. Epstein, S. Jefferyes, U. Qidwai, Z. Aftab, I. Mujeeb, D. Snead, and N. Rajpoot, “Glandular morpho- metrics for objective grading of colorectal adenocarcinoma histology images,” Scientific reports, vol. 7, no. 1, p. 16852, 2017

    work page 2017

  12. [12]

    World cancer report 2014,

    B. Stewart, C. P. Wild et al., “World cancer report 2014,” Health, 2017

    work page 2014

  13. [13]

    Histopathology reporting in large bowel cancer

    W. Blenkinsopp, S. Stewart-Brown, L. Blesovsky, G. Kearney, and L. Fielding, “Histopathology reporting in large bowel cancer.” Journal of clinical pathology , vol. 34, no. 5, pp. 509–513, 1981

    work page 1981

  14. [14]

    The grading of rectal cancer: historical perspectives and a multivariate analysis of 447 cases,

    J. Jass, W. Atkin, J. Cuzick, H. Bussey, B. Morson, J. Northover, and I. Todd, “The grading of rectal cancer: historical perspectives and a multivariate analysis of 447 cases,” Histopathology, vol. 10, no. 5, pp. 437–459, 1986

    work page 1986

  15. [15]

    Classification of breast cancer histology image using ensemble of pre-trained neural networks,

    S. S. Chennamsetty, M. Safwan, and V . Alex, “Classification of breast cancer histology image using ensemble of pre-trained neural networks,” in International Conference Image Analysis and Recognition. Springer, 2018, pp. 804–811

    work page 2018

  16. [16]

    Assess- ment of breast cancer histology using densely connected convolutional networks,

    M. Kohl, C. Walz, F. Ludwig, S. Braunewell, and M. Baust, “Assess- ment of breast cancer histology using densely connected convolutional networks,” in International Conference Image Analysis and Recognition. Springer, 2018, pp. 903–913

    work page 2018

  17. [17]

    Hierarchical resnext models for breast can- cer histology image classification,

    I. Kon ´e and L. Boulmane, “Hierarchical resnext models for breast can- cer histology image classification,” in International Conference Image Analysis and Recognition . Springer, 2018, pp. 796–803

    work page 2018

  18. [18]

    Weakly supervised learning for whole slide lung cancer image classification,

    X. Wang, H. Chen, C. Gan, H. Lin, Q. Dou, Q. Huang, M. Cai, and P.-A. Heng, “Weakly supervised learning for whole slide lung cancer image classification,” 2018

    work page 2018

  19. [19]

    High- throughput adaptive sampling for whole-slide histopathology image analysis (hashi) via convolutional neural networks: Application to in- vasive breast cancer detection,

    A. Cruz-Roa, H. Gilmore, A. Basavanhally, M. Feldman, S. Ganesan, N. Shih, J. Tomaszewski, A. Madabhushi, and F. Gonz ´alez, “High- throughput adaptive sampling for whole-slide histopathology image analysis (hashi) via convolutional neural networks: Application to in- vasive breast cancer detection,” PloS one , vol. 13, no. 5, p. e0196828, 2018

    work page 2018

  20. [20]

    A multi-resolution deep learning framework for lung adenocarcinoma growth pattern classification,

    N. Alsubaie, M. Shaban, D. Snead, A. Khurram, and N. Rajpoot, “A multi-resolution deep learning framework for lung adenocarcinoma growth pattern classification,” in Annual Conference on Medical Image Understanding and Analysis . Springer, 2018, pp. 3–11

    work page 2018

  21. [21]

    A struc- tured latent model for ovarian carcinoma subtyping from histopathology slides,

    A. BenTaieb, H. Li-Chang, D. Huntsman, and G. Hamarneh, “A struc- tured latent model for ovarian carcinoma subtyping from histopathology slides,” Medical image analysis , vol. 39, pp. 194–205, 2017

    work page 2017

  22. [22]

    Detecting cancer metastases on gigapixel pathology images,

    Y . Liu, K. Gadepalli, M. Norouzi, G. E. Dahl, T. Kohlberger, A. Boyko, S. Venugopalan, A. Timofeev, P. Q. Nelson, G. S. Corrado et al. , “Detecting cancer metastases on gigapixel pathology images,” arXiv preprint arXiv:1703.02442, 2017

    work page arXiv 2017

  23. [23]

    Representation-Aggregation Networks for Segmentation of Multi-Gigapixel Histology Images

    A. Agarwalla, M. Shaban, and N. M. Rajpoot, “Representation- aggregation networks for segmentation of multi-gigapixel histology images,” arXiv preprint arXiv:1707.08814 , 2017

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  24. [24]

    Cancer metastasis detection via spatially structured deep network,

    B. Kong, X. Wang, Z. Li, Q. Song, and S. Zhang, “Cancer metastasis detection via spatially structured deep network,” in International Con- ference on Information Processing in Medical Imaging. Springer, 2017, pp. 236–248

    work page 2017

  25. [25]

    Cancer detection in histopathology whole-slide images using conditional random fields on deep embedded spaces,

    F. G. Zanjani, S. Zinger et al. , “Cancer detection in histopathology whole-slide images using conditional random fields on deep embedded spaces,” in Medical Imaging 2018: Digital Pathology , vol. 10581. International Society for Optics and Photonics, 2018, p. 105810I

    work page 2018

  26. [26]

    Cancer Metastasis Detection With Neural Conditional Random Field

    Y . Li and W. Ping, “Cancer metastasis detection with neural conditional random field,” arXiv preprint arXiv:1806.07064 , 2018

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  27. [27]

    Context-Aware Learning using Transferable Features for Classification of Breast Cancer Histology Images

    R. Awan, N. A. Koohbanani, M. Shaban, A. Lisowska, and N. Rajpoot, “Context-aware learning using transferable features for classification of breast cancer histology images,” arXiv preprint arXiv:1803.00386, 2018

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  28. [28]

    Context- aware stacked convolutional neural networks for classification of breast carcinomas in whole-slide histopathology images,

    B. E. Bejnordi, G. Zuidhof, M. Balkenhol, M. Hermsen, P. Bult, B. van Ginneken, N. Karssemeijer, G. Litjens, and J. van der Laak, “Context- aware stacked convolutional neural networks for classification of breast carcinomas in whole-slide histopathology images,” Journal of Medical Imaging, vol. 4, no. 4, p. 044504, 2017

    work page 2017

  29. [29]

    Improving Whole Slide Segmentation Through Visual Context - A Systematic Study

    K. Sirinukunwattana, N. K. Alham, C. Verrill, and J. Rittscher, “Im- proving whole slide segmentation through visual context-a systematic study,” arXiv preprint arXiv:1806.04259 , 2018

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  30. [30]

    Automated gland and nuclei segmentation for grading of prostate and breast cancer histopathology,

    S. Naik, S. Doyle, S. Agner, A. Madabhushi, M. Feldman, and J. Tomaszewski, “Automated gland and nuclei segmentation for grading of prostate and breast cancer histopathology,” in Biomedical Imaging: From Nano to Macro, 2008. ISBI 2008. 5th IEEE International Sympo- sium on. IEEE, 2008, pp. 284–287

    work page 2008

  31. [31]

    An image analysis approach for automatic malignancy determination of prostate pathological images,

    R. Farjam, H. Soltanian-Zadeh, K. Jafari-Khouzani, and R. A. Zoroofi, “An image analysis approach for automatic malignancy determination of prostate pathological images,” Cytometry Part B: Clinical Cytometry, vol. 72, no. 4, pp. 227–240, 2007

    work page 2007

  32. [32]

    Prostate cancer grading: Gland segmentation and structural features,

    K. Nguyen, B. Sabata, and A. K. Jain, “Prostate cancer grading: Gland segmentation and structural features,”Pattern Recognition Letters, vol. 33, no. 7, pp. 951–961, 2012

    work page 2012

  33. [33]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778

    work page 2016

  34. [34]

    MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications

    A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “Mobilenets: Efficient convo- lutional neural networks for mobile vision applications,” arXiv preprint arXiv:1704.04861, 2017

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  35. [35]

    Going deeper with convolutions

    C. Szegedy, W. Liu, Y . Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V . Vanhoucke, A. Rabinovichet al., “Going deeper with convolutions.” Cvpr, 2015

    work page 2015

  36. [36]

    Xception: Deep learning with depthwise separable convo- lutions,

    F. Chollet, “Xception: Deep learning with depthwise separable convo- lutions,” arXiv preprint, pp. 1610–02 357, 2017

    work page 2017

  37. [37]

    Mdnet: A se- mantically and visually interpretable medical image diagnosis network,

    Z. Zhang, Y . Xie, F. Xing, M. McGough, and L. Yang, “Mdnet: A se- mantically and visually interpretable medical image diagnosis network,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 6428–6436

    work page 2017

  38. [38]

    Neural networks for machine learning lecture 6a overview of mini-batch gradient descent

    G. Hinton, N. Srivastava, and K. Swersky, “Neural networks for machine learning lecture 6a overview of mini-batch gradient descent.”

    work page

  39. [39]

    Classification of breast cancer histology images using convolutional neural networks,

    T. Ara ´ujo, G. Aresta, E. Castro, J. Rouco, P. Aguiar, C. Eloy, A. Pol´onia, and A. Campilho, “Classification of breast cancer histology images using convolutional neural networks,” PloS one, vol. 12, no. 6, p. e0177544, 2017

    work page 2017

  40. [40]

    Imagenet classification with deep convolutional neural networks,

    A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in neural infor- mation processing systems , 2012, pp. 1097–1105

    work page 2012