REVIEW 4 major objections 5 minor 29 references
A Performance Increment Strategy for Semantic Segmentation of Low-Resolution Images from Damaged Roads
T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read This paper claims that a systematic 14-experiment training strategy, including removal of the ResNet max-pooling layer, raises semantic segmentation of low-resolution damaged-road images to state-of-the-art accuracy on two benchmarks.
desk verdict A systematic, code-backed ablation recipe for low-res road segmentation, but the SOTA claim is fragile because single-run numbers and a validation-selected best checkpoint leave the reported margins inside the paper's own documented noise. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the encoder's effective stride at two points. In DeepLabV3+, high-level features come from the ASPP block at a chosen output stride (ratio of input to feature-map resolution), and low-level features come from the ResNet stem. The paper removes the stem's max-pooling layer so low-level features are extracted after a 7x7 convolution with stride 2 instead of after pooling with stride 4; combined with output stride 16, the decoder receives both fine low-level detail and broad context. The rest of PISSS is a staged ablation protocol, with each experiment selecting the best hypothesis on the validation set and feeding it into the next, culminating in a flipped multiscale prediction ensemble.
What would settle it
Re-run the RTK experiment with the exact PISSS recipe (DeepLabV3+, ResNet-50, output stride 16, max-pooling removed, crop plus cutmix, cross-entropy loss, flipped ensemble) for at least five random seeds on a fixed test split, and compare against the reported 79.8 mIoU and against each prior published RTK result under an identical evaluation protocol; if the best seed does not reproduce the gain or the margin over prior work vanishes under matched protocol, the state-of-the-art claim fails.
Extended reading notes
Core claim
The central discovery is that DeepLabV3+'s small-object failures on the RTK dataset trace to early feature downsampling, and that deleting the ResNet max-pooling layer while keeping output stride 16 fixes it: the model then concatenates low-level features with a stride of 2 rather than 4, and the small cat's-eye and crack blobs survive into the decoder. Built on top of that fix, the paper's additive recipe (cropping augmentation, cutmix, cross-entropy loss, and a flipped multiscale prediction ensemble) lifts RTK from a 73.9 baseline to 79.8 mIoU and TAS500 to 68.8 mIoU, which the paper reports as best published results. It also shows that the best recipe differs sharply between the two datasets, so the gain is not a single universal setting but a methodology for finding one.
Load-bearing premise
The state-of-the-art claim depends on the assumption that earlier published scores were computed with the same test split and checkpoint-selection rules, and that choosing the best recipe and best checkpoint on the validation set does not inflate the reported test mIoU.
Editorial extensions
If this is right
- Removing the ResNet max-pooling layer and setting output stride 16 should be the first thing to try when segmenting tiny road-surface objects in low-resolution images, because it preserves small blobs that early downsampling destroys.
- The full PISSS recipe lifts RTK from its 73.9 mIoU baseline to 79.8 mIoU, so training procedures alone, without a new architecture, can bring state-of-the-art accuracy on this benchmark.
- Because the best hypotheses differ between RTK and TAS500, no single fixed training setup should be expected to transfer across datasets; a custom additive ablation beats a one-size-fits-all recipe.
- Cross-entropy loss outperforms mIoU and dice surrogate losses once the rest of the training setup is well tuned, which runs counter to the intuition that optimizing a surrogate of the metric is always better.
Reading between the lines
- An untested extension: the max-pooling-removal benefit should also appear in other encoder-decoder segmenters that concatenate low- and high-level features, not only DeepLabV3+, but the paper only demonstrates it within DeepLabV3+.
- The paper's numbers are selected from 14 validation-driven hypotheses and a best checkpoint, with roughly 1% validation oscillation reported; an independent locked-test replication is needed to know how much of the state-of-the-art margin is protocol-specific.
- The authors' suggestion that cutmix helps specifically on rough transitions between road surfaces and damage classes could be tested by applying the same recipe to other damage-annotated road datasets, which would isolate when that augmentation pays off.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents PISSS, an additive sequence of 14 training experiments for semantic segmentation of low-resolution images of damaged roads. Starting from the RTK authors' previous U-Net baseline, the authors test longer training, single-stage training, data augmentation, deeper encoders, DeepLabV3+ variants, max-pooling removal, output stride, transposed convolutions, hybrid dilation modules, cutmix, SGD, alternative losses, and prediction ensembles. On the RTK test set they report 79.8 mIoU using a flipped ensemble and the best-validation checkpoint; on TAS500 they report 68.8 mIoU, surpassing a 2021 CodaLab leaderboard entry of 67.5. The paper also proposes an architectural explanation: removing the ResNet max-pooling layer preserves low-level detail for small objects in DeepLabV3+. The claimed contributions are a performance-boosting training recipe, a new state-of-the-art on two datasets, and an analysis of DeepLabV3+ pitfalls for small-object segmentation.
Significance. If the reported results are reliable, the paper would provide a practical and transferable training recipe for a genuinely underserved regime: low-resolution, poorly maintained road images with many small, underrepresented damage classes. The paper's main conceptual asset is the max-pooling-removal observation, which connects low-level feature stride to small-object segmentation in DeepLabV3+ and is plausibly useful beyond these two datasets. The paper also ships code, which is a concrete reproducibility strength. However, the central benchmark claim is currently under-supported: the state-of-the-art predicate is not demonstrated by the evidence presented, and the magnitudes of the reported improvements are comparable to the validation oscillation the authors themselves document. The practical value of the recipe is plausible, but the quantitative claims need substantially stronger statistical and protocol evidence before they can be accepted.
major comments (4)
- [Sec. 3.2, Sec. 4.5, Table 5] The central claim 'state-of-the-art results of 79.8 mIoU on RTK' is not supported by the evidence in the manuscript. No prior published RTK mIoU results are cited or compared, and the 79.8 number is a single best-checkpoint result from a flipped ensemble, selected after sequentially testing 14 hypotheses on the same validation split. The authors need to compare against all available RTK methods under an identical evaluation protocol and report the variability of the result over multiple training runs or multiple checkpoint-selection strategies.
- [Sec. 6, Table 6 and CodaLab footnote] The TAS500 comparison is protocol-unequal: the paper compares against a single 2021 CodaLab leaderboard entry of 67.5 mIoU without documenting whether the class list, test split, metric implementation, input resolution, or evaluation pipeline match. The claimed margin is 1.34 mIoU, which is within the roughly 1% validation oscillation documented in Sec. 3.2. The authors should provide the exact evaluation protocol, report test-set error bars or multiple-run statistics, and clearly state whether the cited leaderboard entry uses the same test set and metric.
- [Secs. 4.1-4.5, Tables 1-5] The sequential additive selection of hypotheses on the same validation split is a form of validation-set overfitting. With 14 hypotheses and single-run evaluation, the best recipe is likely to look better than it would on a fresh test set. The authors should either report test-set results for all hypotheses, or use a separate validation split for model selection and a held-out test set for the final reported numbers, and they should report the number of runs for each configuration.
- [Sec. 4.2, Table 2 and Figs. 5-6] The architectural claim that removing the ResNet max-pooling layer improves small-object segmentation rests on validation mIoU differences of 0.1 to 1.2 points with no error bars, and the qualitative figures are illustrative rather than quantitative. To make this claim load-bearing, the authors should provide repeated-run statistics, confidence intervals, and ideally object-size-stratified metrics showing that the improvement is concentrated on small objects.
minor comments (5)
- [Sec. 1 and Abstract] The phrase 'state-of-the-art' is used repeatedly, but the manuscript only substantiates the TAS500 comparison with a single leaderboard entry; it would be clearer to name the exact benchmark source for each dataset.
- [Fig. 7, Sec. 5.1] The claim of a 'clear correlation' between object size and mIoU is not quantified; a Pearson or Spearman correlation with a confidence interval would make the statement precise.
- [Sec. 6] The evaluation resolution for TAS500 is not stated explicitly; the text mentions cropped training parts of 1024x512, but the test-time resolution should be reported.
- [Code availability] The GitHub link is useful, but the manuscript should cite a specific commit or version and include a license to make the reproducibility claim concrete.
- [Global formatting] The running header still contains the placeholder 'F. Author et al.', which should be replaced with the actual author list before publication.
Circularity Check
No significant circularity: the paper reports measured benchmark results from external test sets and the only self-citation is a non-load-bearing comparison baseline.
full rationale
This is an empirical ablation study rather than a derivation, and I find no step in which a claimed prediction reduces to its own input by construction. The central claims are the test-set mIoU numbers of 79.8 on RTK and 68.8 on TAS500. These are measured on held-out test sets: RTK is an external dataset released by the authors' group but with fixed annotations, and TAS500 is evaluated through the external Outdoor Semantic Segmentation Challenge CodaLab leaderboard. No fitted constant is renamed as a prediction, and no equation defines the reported mIoU in terms of the training choices. The only self-referential elements are the baseline from the RTK authors' own earlier work (Ref. [18]) and the use of the authors' own RTK dataset (Ref. [17]); neither makes the result circular, because the baseline is used only as a comparison point and the reported improvements are externally measurable against that starting point. The PISSS experiments (Tables 1-5) select hypotheses on a validation split and the final numbers come from a best checkpoint, which is a methodological risk about selection-induced optimism and protocol comparability, not a circularity of the kind where a quantity is defined in terms of itself. The weaknesses flagged in the read-through, such as the unsubstantiated 'state-of-the-art' predicate for RTK and the lack of error bars, are correctness and evidence-quality concerns, not evidence that the derivation collapses into its inputs. Therefore no circular step meets the required evidentiary bar, and the appropriate score is 0.
Assumptions & free parameters
free parameters (4)
- RTK best recipe =
crop 224x224, cutmix p=0.8, DeepLabV3+ ResNet-50 OS16 without max-pooling, CE loss, Adam lr=1e-4, batch 8
- TAS500 best recipe =
resize+crop 1024x512, OS16, CE+dice, SGD, lr=5e-5, batch 4
- Checkpoint and ensemble selection =
best-validation checkpoint; flipped multi-scale ensemble at 288x224, 352x288, 448x352
- Encoder output stride and max-pooling =
OS 16, no MP for RTK; OS 16 with MP for TAS500
assumptions (4)
- domain assumption RTK and TAS500 test labels and evaluation protocols are comparable to those of the cited prior results.
- domain assumption Validation mIoU averaged over the last ten training steps is a reliable model-selection signal.
- domain assumption The additive, validation-driven ablation of 14 experiments did not overfit the validation set.
- domain assumption Meaningful mIoU differences are larger than the observed validation noise.
Cite this review
Pith. "Pith review of A Performance Increment Strategy for Semantic Segmentation of Low-Resolution Images from Damaged Roads." pith.science (2026). https://pith.science/paper/WOHMTUUG
@misc{pith2026241116295,
author = {Pith},
title = {Pith review of: A Performance Increment Strategy for Semantic Segmentation of Low-Resolution Images from Damaged Roads},
year = {2026},
howpublished = {\url{https://pith.science/paper/WOHMTUUG}},
note = {Machine review of arXiv:2411.16295}
}
read the original abstract
Autonomous driving needs good roads, but 85% of Brazilian roads have damages that deep learning models may not regard as most semantic segmentation datasets for autonomous driving are high-resolution images of well-maintained urban roads. A representative dataset for emerging countries consists of low-resolution images of poorly maintained roads and includes labels of damage classes; in this scenario, three challenges arise: objects with few pixels, objects with undefined shapes, and highly underrepresented classes. To tackle these challenges, this work proposes the Performance Increment Strategy for Semantic Segmentation (PISSS) as a methodology of 14 training experiments to boost performance. With PISSS, we reached state-of-the-art results of 79.8 and 68.8 mIoU on the Road Traversing Knowledge (RTK) and Technik Autonomer Systeme 500 (TAS500) test sets, respectively. Furthermore, we also offer an analysis of DeepLabV3+ pitfalls for small object segmentation.
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[1]
Advances in Neural Information Processing Systems34, 22614–22627 (2021)
Bello, I., Fedus, W., Du, X., Cubuk, E.D., Srinivas, A., Lin, T.Y., Shlens, J., Zoph, B.: Revisiting resnets: Improved training and scaling strategies. Advances in Neural Information Processing Systems34, 22614–22627 (2021)
work page 2021
-
[2]
Pattern Recognition Letters30(2), 88–97 (2009)
Brostow, G.J., Fauqueur, J., Cipolla, R.: Semantic object classes in video: A high- definition ground truth database. Pattern Recognition Letters30(2), 88–97 (2009)
work page 2009
-
[3]
IEEE transactions on pattern analysis and machine intelli- gence 40(4), 834–848 (2017)
Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Deeplab: Se- mantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE transactions on pattern analysis and machine intelli- gence 40(4), 834–848 (2017)
2017
-
[4]
In: Proceedings of the IEEE conference on computer vision and pattern recognition
Chen, L.C., Yang, Y., Wang, J., Xu, W., Yuille, A.L.: Attention to scale: Scale- aware semantic image segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 3640–3649 (2016)
work page 2016
-
[5]
In: Proceedings of the European conference on computer vision (ECCV)
Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European conference on computer vision (ECCV). pp. 801–818 (2018)
2018
-
[6]
SEST SENAT (2021), https:// pesquisarodovias.cnt.org.br/downloads/ultimaversao/
CNT: Pesquisa CNT de rodovias 2021. SEST SENAT (2021), https:// pesquisarodovias.cnt.org.br/downloads/ultimaversao/
work page 2021
-
[7]
In: Proceedings of the IEEE conference on computer vision and pattern recognition
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., Schiele, B.: The cityscapes dataset for semantic urban scene understanding. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 3213–3223 (2016)
2016
-
[8]
IEEE transactions on pattern analysis and machine intelligence 43(2), 652–662 (2019)
Gao, S.H., Cheng, M.M., Zhao, K., Zhang, X.Y., Yang, M.H., Torr, P.: Res2net: A new multi-scale backbone architecture. IEEE transactions on pattern analysis and machine intelligence 43(2), 652–662 (2019)
work page 2019
Show all 29 references
-
[9]
The International Journal of Robotics Research32(11), 1231–1237 (2013)
Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: The kitti dataset. The International Journal of Robotics Research32(11), 1231–1237 (2013)
2013
-
[10]
In: 2018 IEEE winter conference on applications of computer vision (WACV)
Hamaguchi, R., Fujita, A., Nemoto, K., Imaizumi, T., Hikosaka, S.: Effective use of dilated convolutions for segmenting small object instances in remote sensing imagery. In: 2018 IEEE winter conference on applications of computer vision (WACV). pp. 1442–1450. IEEE (2018)
2018
-
[11]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
He, T., Zhang, Z., Zhang, H., Zhang, Z., Xie, J., Li, M.: Bag of tricks for image clas- sification with convolutional neural networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 558–567 (2019)
2019
-
[12]
In: Proceedings of the IEEE conference on computer vision and pattern recognition
Hu, P., Ramanan, D.: Finding tiny faces. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 951–959 (2017) Title Suppressed Due to Excessive Length 15
2017
-
[13]
arXiv preprint arXiv:1712.07628 (2017)
Keskar, N.S., Socher, R.: Improving generalization performance by switching from adam to sgd. arXiv preprint arXiv:1712.07628 (2017)
2017 arXiv
-
[14]
In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition
Kirillov,A.,Wu,Y.,He,K.,Girshick,R.:Pointrend:Imagesegmentationasrender- ing. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 9799–9808 (2020)
2020
-
[15]
Sensors 19(19), 4115 (2019)
Li, Y., Peng, B., He, L., Fan, K., Li, Z., Tong, L.: Road extraction from unmanned aerial vehicle remote sensing images based on improved neural networks. Sensors 19(19), 4115 (2019)
2019
-
[16]
In: 2020 25th International Conference on Pattern Recognition (ICPR)
Metzger, K.A., Mortimer, P., Wuensche, H.J.: A fine-grained dataset and its ef- ficient semantic segmentation for unstructured driving scenarios. In: 2020 25th International Conference on Pattern Recognition (ICPR). pp. 7892–7899. IEEE (2021)
2021
-
[17]
Revista de Informática Teórica e Aplicada26(3), 50–64 (2019)
Rateke, T., Justen, K.A., Von Wangenheim, A.: Road surface classification with images captured from low-cost camera-road traversing knowledge (rtk) dataset. Revista de Informática Teórica e Aplicada26(3), 50–64 (2019)
2019
-
[18]
Autonomous Robots45(2), 299–312 (2021)
Rateke, T., Von Wangenheim, A.: Road surface detection and differentiation con- sidering surface damages. Autonomous Robots45(2), 299–312 (2021)
2021
-
[19]
In: Proceedings of the IEEE international conference on computer vision
Shen, Z., Liu, Z., Li, J., Jiang, Y.G., Chen, Y., Xue, X.: Dsod: Learning deeply supervised object detectors from scratch. In: Proceedings of the IEEE international conference on computer vision. pp. 1919–1927 (2017)
2017
-
[20]
In: 2016 IEEE 19th international conference on intelligent transportation systems (ITSC)
Shinzato, P.Y., dos Santos, T.C., Rosero, L.A., Ridel, D.A., Massera, C.M., Alen- car, F., Batista, M.P., Hata, A.Y., Osório, F.S., Wolf, D.F.: Carina dataset: An emerging-country urban scenario benchmark for road detection systems. In: 2016 IEEE 19th international conference ...
2016
-
[21]
arXiv preprint arXiv:2005.10821 (2020)
Tao, A., Sapra, K., Catanzaro, B.: Hierarchical multi-scale attention for semantic segmentation. arXiv preprint arXiv:2005.10821 (2020)
2020 arXiv
-
[22]
In: 2019 IEEE Winter Conference on Applications of Computer Vision (WACV)
Varma, G., Subramanian, A., Namboodiri, A., Chandraker, M., Jawahar, C.: Idd: A dataset for exploring problems of autonomous navigation in unconstrained envi- ronments. In: 2019 IEEE Winter Conference on Applications of Computer Vision (WACV). pp. 1743–1751. IEEE (2019)
2019
-
[23]
In: 2018 IEEE winter conference on applications of computer vision (WACV)
Wang, P., Chen, P., Yuan, Y., Liu, D., Huang, Z., Hou, X., Cottrell, G.: Under- standing convolution for semantic segmentation. In: 2018 IEEE winter conference on applications of computer vision (WACV). pp. 1451–1460. Ieee (2018)
2018
-
[24]
arXiv preprint arXiv:2110.00476 (2021)
Wightman, R., Touvron, H., Jégou, H.: Resnet strikes back: An improved training procedure in timm. arXiv preprint arXiv:2110.00476 (2021)
2021 arXiv
-
[25]
International Journal of Computer Vision129(11), 3051–3068 (2021)
Yu, C., Gao, C., Wang, J., Yu, G., Shen, C., Sang, N.: Bisenet v2: Bilateral network with guided aggregation for real-time semantic segmentation. International Journal of Computer Vision129(11), 3051–3068 (2021)
2021
-
[26]
arXiv preprint arXiv:1909.11065 (2019)
Yuan, Y., Chen, X., Chen, X., Wang, J.: Segmentation transformer: Object-contextual representations for semantic segmentation. arXiv preprint arXiv:1909.11065 (2019)
2019 arXiv
-
[27]
In: Proceedings of the IEEE conference on computer vision and pattern recognition
Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 2881–2890 (2017)
2017
-
[28]
Advances in Neural Information Processing Systems33, 21285–21296 (2020)
Zhou, P., Feng, J., Ma, C., Xiong, C., Hoi, S.C.H., et al.: Towards theoretically understanding why sgd generalizes better than adam in deep learning. Advances in Neural Information Processing Systems33, 21285–21296 (2020)
2020
-
[29]
In: proceedings of the IEEE conference on computer vision and pattern recognition
Zhou, P., Ni, B., Geng, C., Hu, J., Xu, Y.: Scale-transferrable object detection. In: proceedings of the IEEE conference on computer vision and pattern recognition. pp. 528–537 (2018)
2018
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.