pith. sign in

arxiv: 1907.06082 · v1 · pith:OAL6VC4Mnew · submitted 2019-07-13 · 💻 cs.CV

Adaptive Context Encoding Module for Semantic Segmentation

Congcong Wang , Faouzi Alaya Cheikh , Azeddine Beghdadi , Ole Jakob Elle This is my paper

Pith reviewed 2026-05-24 21:55 UTC · model grok-4.3

classification 💻 cs.CV
keywords semantic segmentationdeformable convolutioncontext aggregationmulti-scale contextadaptive encodingPascal-Context datasetADE20K dataset
0
0 comments X

The pith

A module of three deformable convolution blocks captures multi-scale context adaptively and outperforms PPM and ASPP on segmentation benchmarks.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces an Adaptive Context Encoding (ACE) module to handle diverse object sizes in semantic segmentation by using deformable convolutions for adaptive context aggregation. Traditional methods such as pyramid pooling module and atrous spatial pyramid pooling require manual selection of pooling sizes or rates, which is not optimal. The ACE module consists of three deformable convolution blocks that can be easily embedded into CNNs. It demonstrates higher mean Intersection over Union on Pascal-Context and ADE20K datasets compared to PPM and ASPP. This indicates that learning adaptive sampling locations improves context capture for objects of varying scales.

Core claim

The proposed ACE module based on deformable convolution adaptively augments multiple scale information and, with only three blocks, achieves higher mIoU than PPM and ASPP on Pascal-Context and ADE20K datasets.

What carries the argument

The Adaptive Context Encoding (ACE) module, consisting of deformable convolution blocks that adaptively sample context information at multiple scales.

If this is right

  • Networks for semantic segmentation can integrate the ACE module to improve accuracy without complex manual tuning of scale parameters.
  • The use of deformable convolutions enables the model to adjust sampling based on the actual object shapes and sizes in the image.
  • Since the module is lightweight with only three blocks, it adds minimal computational overhead while providing better performance.
  • Results on two challenging datasets suggest the approach generalizes across different scene types and object varieties.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Similar adaptive mechanisms could be applied to other dense prediction tasks like instance segmentation or depth estimation.
  • Exploring combinations of ACE with transformer-based architectures might yield further improvements in context modeling.
  • Validating the module on additional datasets with different resolutions or domains would strengthen the evidence for its adaptability.

Load-bearing premise

The observed performance advantage stems from the adaptive sampling enabled by deformable convolutions and not from unmentioned differences in how the models were trained or configured.

What would settle it

If experiments that match the training protocol, network backbone, and all hyperparameters exactly between ACE, PPM, and ASPP show no mIoU improvement for ACE, the claim that the module design is responsible would be falsified.

Figures

Figures reproduced from arXiv: 1907.06082 by Azeddine Beghdadi, Congcong Wang, Faouzi Alaya Cheikh, Ole Jakob Elle.

Figure 1
Figure 1. Figure 1: (a) Pyramid pooling module (PPM) proposed in PSPNet [ [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
read the original abstract

The object sizes in images are diverse, therefore, capturing multiple scale context information is essential for semantic segmentation. Existing context aggregation methods such as pyramid pooling module (PPM) and atrous spatial pyramid pooling (ASPP) design different pooling size or atrous rate, such that multiple scale information is captured. However, the pooling sizes and atrous rates are chosen manually and empirically. In order to capture object context information adaptively, in this paper, we propose an adaptive context encoding (ACE) module based on deformable convolution operation to argument multiple scale information. Our ACE module can be embedded into other Convolutional Neural Networks (CNN) easily for context aggregation. The effectiveness of the proposed module is demonstrated on Pascal-Context and ADE20K datasets. Although our proposed ACE only consists of three deformable convolution blocks, it outperforms PPM and ASPP in terms of mean Intersection of Union (mIoU) on both datasets. All the experiment study confirms that our proposed module is effective as compared to the state-of-the-art methods.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper proposes an Adaptive Context Encoding (ACE) module consisting of three deformable convolution blocks to adaptively capture multi-scale context for semantic segmentation. It contrasts this with manually chosen pooling sizes in PPM or atrous rates in ASPP, claims the module can be easily embedded in CNNs, and asserts that ACE outperforms PPM and ASPP in mIoU on Pascal-Context and ADE20K despite its simplicity.

Significance. If the mIoU gains hold under controlled conditions and are due to the adaptive sampling of deformable convolutions, ACE would offer a compact, plug-in alternative for context aggregation that avoids hand-tuned hyperparameters in existing pyramid modules.

major comments (2)
  1. [Abstract] Abstract: The claim that ACE 'outperforms PPM and ASPP in terms of mean Intersection of Union (mIoU) on both datasets' is stated without any numerical mIoU values, error bars, or ablation results, preventing assessment of the effect size or statistical reliability of the reported advantage.
  2. [Experiments] Experiments section: No statement confirms that PPM and ASPP baselines were re-implemented inside the identical backbone, inserted at the same feature-map stage, and trained with the same optimizer, learning-rate schedule, data augmentation, and epoch count as ACE. Without this, the performance delta cannot be isolated to the adaptive sampling property of the three deformable convolution blocks.
minor comments (2)
  1. [Abstract] Typo: 'to argument multiple scale information' should read 'to augment multiple scale information'.
  2. [Abstract] Grammar: 'All the experiment study confirms' should be 'All experimental studies confirm'.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments. We address each major comment below and will revise the manuscript to improve clarity and provide the requested details on quantitative results and experimental controls.

read point-by-point responses

  1. Referee: [Abstract] Abstract: The claim that ACE 'outperforms PPM and ASPP in terms of mean Intersection of Union (mIoU) on both datasets' is stated without any numerical mIoU values, error bars, or ablation results, preventing assessment of the effect size or statistical reliability of the reported advantage.

    Authors: We agree that including numerical values in the abstract would allow better assessment of the claimed advantage. In the revised version we will update the abstract to report the specific mIoU scores achieved by ACE versus PPM and ASPP on Pascal-Context and ADE20K, and we will reference the ablation studies already present in the experiments section. revision: yes

  2. Referee: [Experiments] Experiments section: No statement confirms that PPM and ASPP baselines were re-implemented inside the identical backbone, inserted at the same feature-map stage, and trained with the same optimizer, learning-rate schedule, data augmentation, and epoch count as ACE. Without this, the performance delta cannot be isolated to the adaptive sampling property of the three deformable convolution blocks.

    Authors: We confirm that the baselines were re-implemented under fully controlled conditions using the identical backbone, insertion stage, optimizer, learning-rate schedule, data augmentation, and epoch count. We will add an explicit paragraph in the Experiments section stating these details so that the performance differences can be attributed to the adaptive sampling of the deformable convolution blocks. revision: yes

Circularity Check

0 steps flagged

No circularity in derivation chain

full rationale

The paper proposes an architectural module (ACE) using three deformable convolution blocks for adaptive context encoding and reports empirical mIoU gains over PPM and ASPP on Pascal-Context and ADE20K. No equations, parameter-fitting steps, or self-citations are present that reduce any performance claim to an input quantity defined by the authors themselves. The comparison is presented as an external benchmark result rather than a self-referential derivation, satisfying the criteria for a self-contained empirical claim with no load-bearing circular steps.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review supplies no information on free parameters, background axioms, or newly postulated entities.

pith-pipeline@v0.9.0 · 5713 in / 1006 out tokens · 18382 ms · 2026-05-24T21:55:31.312562+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

37 extracted references · 37 canonical work pages · 4 internal anchors

  1. [1]

    Amirul Islam, M

    M. Amirul Islam, M. Rochan, N. D. Bruce, and Y . Wang. Gated feedback refinement network for dense image label- ing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 3751–3759. IEEE,

    work page

  2. [2]

    Badrinarayanan, A

    V . Badrinarayanan, A. Kendall, and R. Cipolla. Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE transactions on pattern analysis and machine intelligence, 39(12):2481–2495, 2017. 1, 6

    work page 2017

  3. [3]

    L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille. Semantic image segmentation with deep con- volutional nets and fully connected crfs. In International Conference on Learning Representations, 2015. 2

    work page 2015

  4. [4]

    L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully con- nected crfs. IEEE transactions on pattern analysis and ma- chine intelligence, 40(4):834–848, 2018. 1, 2, 3, 5, 6 6

    work page 2018

  5. [5]

    L.-C. Chen, G. Papandreou, F. Schroff, and H. Adam. Re- thinking atrous convolution for semantic image segmenta- tion. arXiv preprint arXiv:1706.05587, 2017. 1, 2, 3, 5

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  6. [6]

    L.-C. Chen, Y . Zhu, G. Papandreou, F. Schroff, and H. Adam. Encoder-decoder with atrous separable convolution for se- mantic image segmentation. In Proceedings of the European Conference on Computer Vision , pages 801–818. Springer,

    work page

  7. [7]

    F. Chollet. Xception: Deep learning with depthwise sepa- rable convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 1251–

    work page

  8. [8]

    J. Dai, H. Qi, Y . Xiong, Y . Li, G. Zhang, H. Hu, and Y . Wei. Deformable convolutional networks. In Proceedings of the IEEE International Conference on Computer Vision , pages 764–773. IEEE, 2017. 2, 4

    work page 2017

  9. [9]

    Farabet, C

    C. Farabet, C. Couprie, L. Najman, and Y . LeCun. Learning hierarchical features for scene labeling. IEEE transactions on pattern analysis and machine intelligence , 35(8):1915– 1929, 2013. 1

    work page 1915

  10. [10]

    Z. Gu, J. Cheng, H. Fu, K. Zhou, H. Hao, Y . Zhao, T. Zhang, S. Gao, and J. Liu. Ce-net: Context encoder network for 2d medical image segmentation. IEEE transactions on medical imaging, 2019. 3

    work page 2019

  11. [11]

    K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learn- ing for image recognition. In Proceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition, pages 770–778. IEEE, 2016. 2, 4, 5

    work page 2016

  12. [12]

    H. T. Y . L. Y . B. Z. F. a. H. L. Jun Fu, Jing Liu. Dual atten- tion network for scene segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion. IEEE, 2019. 6

    work page 2019

  13. [13]

    G. Lin, A. Milan, C. Shen, and I. Reid. Refinenet: Multi-path refinement networks for high-resolution semantic segmenta- tion. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 1925–1934. IEEE,

    work page 1925

  14. [14]

    G. Lin, C. Shen, A. Van Den Hengel, and I. Reid. Effi- cient piecewise training of deep structured models for se- mantic segmentation. In Proceedings of the IEEE Confer- ence on Computer Vision and Pattern Recognition , pages 3194–3203. IEEE, 2016. 1, 6

    work page 2016

  15. [15]

    W. Liu, A. Rabinovich, and A. C. Berg. Parsenet: Looking wider to see better. arXiv preprint arXiv:1506.04579, 2015. 2, 3, 5, 6

    work page internal anchor Pith review Pith/arXiv arXiv 2015

  16. [16]

    J. Long, E. Shelhamer, and T. Darrell. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion, pages 3431–3440. IEEE, 2015. 1, 6

    work page 2015

  17. [17]

    S. Mallat. A wavelet tour of signal processing . Elsevier,

    work page

  18. [18]

    Mohammed, S

    A. Mohammed, S. Yildirim, I. Farup, M. Pedersen, and Ø. Hovde. Y-net: A deep convolutional neural network for polyp detection. arXiv preprint arXiv:1806.01907, 2018. 1

    work page arXiv 2018

  19. [19]

    Mohammed, S

    A. Mohammed, S. Yildirim, I. Farup, M. Pedersen, and Ø. Hovde. Streoscennet: surgical stereo robotic scene seg- mentation. In Medical Imaging 2019: Image-Guided Proce- dures, Robotic Interventions, and Modeling, volume 10951, page 109510P. International Society for Optics and Photon- ics, 2019. 1

    work page 2019

  20. [20]

    H. Noh, S. Hong, and B. Han. Learning deconvolution net- work for semantic segmentation. In Proceedings of the IEEE International Conference on Computer Vision, pages 1520–

    work page

  21. [21]

    Oliva and A

    A. Oliva and A. Torralba. The role of context in object recog- nition. Trends in cognitive sciences, 11(12):520–527, 2007. 2, 4, 5

    work page 2007

  22. [22]

    Paszke, S

    A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. De- Vito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer. Auto- matic differentiation in pytorch. 2017. 5

    work page 2017

  23. [23]

    C. Peng, X. Zhang, G. Yu, G. Luo, and J. Sun. Large ker- nel mattersimprove semantic segmentation by global convo- lutional network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 1743–

    work page

  24. [24]

    P. H. Pinheiro and R. Collobert. Recurrent convolutional neural networks for scene labeling. In International Con- ference on Machine Learning, 2014. 1

    work page 2014

  25. [25]

    Ronneberger, P

    O. Ronneberger, P. Fischer, and T. Brox. U-net: Convolu- tional networks for biomedical image segmentation. In In- ternational Conference on Medical Image Computing and Computer-assisted Intervention , pages 234–241. Springer,

    work page

  26. [26]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. Attention is all you need. In Advances in neural information processing sys- tems, pages 5998–6008, 2017. 3

    work page 2017

  27. [27]

    H. Wu, J. Zhang, K. Huang, K. Liang, and Y . Yu. Fastfcn: Rethinking dilated convolution in the backbone for semantic segmentation. arXiv preprint arXiv:1903.11816, 2019. 2, 5, 6

    work page internal anchor Pith review Pith/arXiv arXiv 1903

  28. [28]

    C. Yu, J. Wang, C. Peng, C. Gao, G. Yu, and N. Sang. Bisenet: Bilateral segmentation network for real-time se- mantic segmentation. In Proceedings of the European Con- ference on Computer Vision, pages 325–341. Springer, 2018. 2

    work page 2018

  29. [29]

    C. Yu, J. Wang, C. Peng, C. Gao, G. Yu, and N. Sang. Learn- ing a discriminative feature network for semantic segmenta- tion. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 1857–1866. IEEE,

    work page

  30. [30]

    Yu and V

    F. Yu and V . Koltun. Multi-scale context aggregation by di- lated convolutions. In International Conference on Learning Representations, 2016. 6

    work page 2016

  31. [31]

    Yuan and J

    Y . Yuan and J. Wang. Ocnet: Object context network for scene parsing. arXiv preprint arXiv:1809.00916, 2018. 3

    work page arXiv 2018

  32. [32]

    M. D. Zeiler and R. Fergus. Visualizing and understand- ing convolutional networks. In Proceedings of the European Conference on Computer Vision , pages 818–833. Springer,

    work page

  33. [33]

    Zhang, K

    H. Zhang, K. Dana, J. Shi, Z. Zhang, X. Wang, A. Tyagi, and A. Agrawal. Context encoding for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7151–7160. IEEE, 2018. 5, 6 7

    work page 2018

  34. [34]

    H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia. Pyramid scene parsing network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 2881–

    work page

  35. [35]

    1, 2, 3, 5, 6

    IEEE, 2017. 1, 2, 3, 5, 6

    work page 2017

  36. [36]

    B. Zhou, H. Zhao, X. Puig, S. Fidler, A. Barriuso, and A. Tor- ralba. Scene parsing through ade20k dataset. In Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 633–641. IEEE, 2017. 2, 4, 5, 6

    work page 2017

  37. [37]

    X. Zhu, H. Hu, S. Lin, and J. Dai. Deformable con- vnets v2: More deformable, better results. arXiv preprint arXiv:1811.11168, 2018. 2, 4 8

    work page internal anchor Pith review Pith/arXiv arXiv 2018