REVIEW 3 major objections 4 minor 34 references
Texture Underfitting for Domain Adaptation
T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Randomly stylizing images with paintings and a two-stage training schedule makes neural segmentation models rely on shape rather than texture, improving synthetic-to-real domain adaptation.
desk verdict A simple, useful stylization trick for domain adaptation in segmentation, but the unsupervised evaluation protocol is post-hoc and needs fixing before the headline claim is solid. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing objects are the stylized dataset and the two-stage training schedule. Each image from the source and target datasets is restyled by adaptive instance normalization using the style of a random painting, so textures vary from image to image while content structure is preserved; training then alternates between a combined dataset (stylized and conventional examples drawn with equal probability) for pre-training and the conventional dataset for fine-tuning. This schedule is what lets the model first learn a texture-invariant, shape-based representation that is more similar between the two domains, and then pick up the target-domain textural cues needed for fine-grained boundaries.
What would settle it
Compare the combined-dataset pre-training schedule against a control where style transfer is replaced by a texture randomization that provably preserves edges, such as color-only jitter or noise confined to texture regions. If that control matches the 33.93 mIoU result, the mechanism is texture randomization, not stylization; if it falls back to the 32.84 baseline, the specific style-transfer perturbation is essential. A second check is to vary the pre-training/fine-tuning iteration split: if the gain disappears when the switch happens earlier or later, the two-stage schedule is the load-bearing component.
Extended reading notes
Core claim
The central discovery is that randomizing image textures at training time, and scheduling training so that shape-based pre-training precedes texture-based fine-tuning, improves transfer from synthetic driving scenes to real ones. On the synthetic-to-real semantic segmentation task, stylizing both source and target images with arbitrary painting styles destroys the class-texture correlation, forcing the network to rely on object boundaries and global structure—information that transfers across domains. The authors demonstrate this on the AdaptSegNet unsupervised pipeline (33.93 versus 32.84 mean IoU) and on supervised fine-tuning with 5, 10, and 20 labeled Cityscapes images, where pre-training on the combined stylized-plus-conventional data outperforms conventional GTA pre-training until the gap narrows at 20 images.
Load-bearing premise
The load-bearing premise is that random painting stylization destroys texture cues while leaving the object shapes and boundaries that segmentation depends on sufficiently intact; the authors themselves note that stylization inevitably perturbs local structure.
Editorial extensions
If this is right
- Because stylization is a data-engineering step, it can be plugged into existing unsupervised or supervised domain-adaptation methods without modifying the architecture or loss functions.
- The benefit is largest when target-domain labels are scarce: with 5 Cityscapes images, combined pre-training improves mIoU from 38.38 to 39.65, while the gain nearly vanishes at 20 images.
- Pre-training on stylized data alone hurts (32.43 mIoU with 5 images), so the improvement depends on mixing conventional and stylized data in pre-training, not on using stylization exclusively.
- Qualitative results indicate gains concentrate on well-defined shapes such as sidewalks and buildings and reduce small isolated mislabeled regions, suggesting the mechanism improves boundary awareness.
Reading between the lines
- If the mechanism is texture randomization rather than the specific painting style, cheaper corruptions such as color jitter, channel shuffling, or noise confined to texture regions should produce comparable gains; the paper does not test this.
- The near-tie at 20 Cityscapes images suggests that the value of stylization scales inversely with the amount of target-domain supervision, so the method may matter again for rare classes or imbalanced label distributions even when total labels are plentiful.
- The same two-stage texture-underfitting recipe could transfer to other dense prediction tasks where shape dominates, such as depth estimation or instance segmentation, but generalization beyond semantic segmentation is not established.
- Because style transfer perturbs local structure, a style pool curated to preserve edges, or an edge-aware regularization after stylization, might extend the gains; this is a testable design choice rather than a paper claim.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a data-augmentation and training procedure for domain adaptation in semantic segmentation, motivated by the texture bias of CNNs. The authors use AdaIN-based random image stylization to create texture-randomized versions of source and target images, then pre-train on the combined conventional plus stylized datasets and fine-tune on conventional data. They evaluate on GTA-to-Cityscapes transfer in two settings: unsupervised domain adaptation with AdaptSegNet and supervised domain adaptation by fine-tuning on 5, 10, or 20 labeled Cityscapes images. The central claim is that this texture-underfitting approach outperforms conventional training methods.
Significance. If the claims are sustained, the paper offers a simple, orthogonal plug-in strategy for domain adaptation that connects texture-bias findings in classification to dense prediction tasks. The experimental design has strengths: ten runs for the unsupervised comparison, use of public datasets, and a clear baseline comparison. However, the headline unsupervised result is weakened by a post-hoc evaluation protocol and missing variance reporting, and the supervised results are not consistently significant across data amounts. The paper would be a useful contribution if the evaluation is made fairer and the scope of the claims is made more precise.
major comments (3)
- [IV-A, Figure 3, Table I] The unsupervised comparison is not made under a fixed evaluation protocol. The text states that the averaging window length is seven time-steps and that "we optimize its location for each method individually," with the conventional method evaluated at iterations 110k-135k and the combined method at 125k-150k. Selecting the window location on the same validation curves that are being compared can systematically inflate the reported 1.09 mIoU advantage (33.93 vs. 32.84). The stated rationale for window averaging is to approximate the practical setting where the exact stopping point is unknown, which argues for a fixed or pre-specified rule rather than per-method optimization. No standard deviations or significance tests are reported despite ten runs being performed, so the reader cannot assess run-to-run variability. Please re-evaluate both methods under a fixed checkpoint-selection policy (e.g., a pre-registered iteration or a fixed window at the end of training) and report error bars or pairwise significance information.
- [IV-B, Table II] The supervised experiments do not consistently support the general claim that the proposed approach outperforms conventional training. At 20 Cityscapes images, CGTA (41.92 mIoU) is essentially tied with GTA pre-training (41.91 mIoU); at 10 images the gain is 0.51 mIoU; only at 5 images is the gain substantial (1.27 mIoU). The table reports a single checkpoint per setup, selected by best validation performance on a held-out set, with no multiple runs or significance testing. The claim in the abstract that the approach "outperforms conventional training methods" is therefore too broad. Please either restrict the claim to the low-data regime or provide statistical support such as multiple runs with confidence intervals, or a significance test across the different labeled-set sizes.
- [V and Table II] The mechanism is not sufficiently isolated by the experiments. Stylized-only pre-training (SGTA) is substantially worse than conventional pre-training (e.g., 32.43 vs. 38.38 mIoU with 5 images), and the authors acknowledge in Section V that "image stylization inevitably perturbs local structure" and impedes segmentation. The success of the combined procedure may therefore hinge on the carefully chosen pre-training/fine-tuning split and on the sampling probability p=0.5, but no ablations are reported for these hyperparameters. Please add ablations varying p and the iteration split, and ideally compare against a control augmentation that is not texture-targeted, to support the interpretation that texture underfitting rather than generic data augmentation is responsible for the observed gains.
minor comments (4)
- [Table I caption] The table caption says results are evaluated "over 10 runs and a window of 6 iterations," while Section IV-A says the window length is seven time-steps; please reconcile this discrepancy.
- [Figure 3] The legend and caption contain typos: "Conventional datased" should be "Conventional dataset" and "Combined datased" should be "Combined dataset."
- [III-C] There are several typos in this section: "disciminator" should be "discriminator," and "providing he final high-quality segmentation masks" should be "providing the final high-quality segmentation masks."
- [IV-A] The text lists fine-tuning duration as "additional 75.000 iterations" with a period as thousands separator; this should be "75,000" or "75 000" for consistency with the rest of the paper.
Circularity Check
No circularity found: the method is an empirical, externally benchmarked data-augmentation pipeline; no prediction reduces by construction to its inputs.
full rationale
The paper's central claim is an empirical comparison against external benchmarks (GTA to Cityscapes) using an external stylization method (AdaIN, [15]), an external baseline (AdaptSegNet, [28]), and an external texture-bias premise ([11], Geirhos et al.). No equation in the paper defines a quantity in terms of the quantity it is supposed to predict; no fitted parameter is renamed as a prediction. The two-stage training procedure (pre-train on combined stylized plus conventional data, then fine-tune on conventional data) is evaluated through measured mIoU on the Cityscapes validation and test sets, so the outcome is not imposed by construction. Self-citations in Section II-C ([5], [6], [26]) are background references to the authors' prior domain-adaptation work and are not load-bearing for the new claim. Section V explicitly acknowledges that stylization perturbs local structure and that stylized-only pre-training (SGTA) degrades performance (Table II, 32.43 vs 38.38 mIoU with 5 images); this is an honest limitation, not a circular step. The main caveat is the unsupervised evaluation protocol in Section IV-A: the averaging window is optimized separately for each method ('we optimize its location for each method individually'), and ten runs are reported without error bars. This weakens the strength of the headline gain but is a statistical rigor concern, not circularity: the reported numbers still come from the actual validation curves and are not algebraically forced by the method's definition. Overall, the derivation chain is self-contained against external evidence.
Assumptions & free parameters
free parameters (4)
- pre-training iterations =
85,000
- fine-tuning iterations =
75,000
- combined dataset sampling probability p =
0.5
- evaluation window length and location =
length 7; conventional 110k-135k, combined 125k-150k
assumptions (4)
- domain assumption Segmentation maps share structural similarities across source and target domains.
- domain assumption In synthetic-to-real domain adaptation, structures (shapes) are more consistent across domains than textures.
- domain assumption Random image stylization randomizes textures while preserving structures sufficiently for segmentation.
- domain assumption Neural networks are biased towards texture over structure.
Cite this review
Pith. "Pith review of Texture Underfitting for Domain Adaptation." pith.science (2026). https://pith.science/paper/3WULIUWW
@misc{pith2026190811215,
author = {Pith},
title = {Pith review of: Texture Underfitting for Domain Adaptation},
year = {2026},
howpublished = {\url{https://pith.science/paper/3WULIUWW}},
note = {Machine review of arXiv:1908.11215}
}
read the original abstract
Comprehensive semantic segmentation is one of the key components for robust scene understanding and a requirement to enable autonomous driving. Driven by large scale datasets, convolutional neural networks show impressive results on this task. However, a segmentation algorithm generalizing to various scenes and conditions would require an enormously diverse dataset, making the labour intensive data acquisition and labeling process prohibitively expensive. Under the assumption of structural similarities between segmentation maps, domain adaptation promises to resolve this challenge by transferring knowledge from existing, potentially simulated datasets to new environments where no supervision exists. While the performance of this approach is contingent on the concept that neural networks learn a high level understanding of scene structure, recent work suggests that neural networks are biased towards overfitting to texture instead of learning structural and shape information. Considering the ideas underlying semantic segmentation, we employ random image stylization to augment the training dataset and propose a training procedure that facilitates texture underfitting to improve the performance of domain adaptation. In experiments with supervised as well as unsupervised methods for the task of synthetic-to-real domain adaptation, we show that our approach outperforms conventional training methods.
Figures
Reference graph
Works this paper leans on
-
[1]
Approximating CNNs with bag-of-local- features models works surprisingly well on imagenet,
W. Brendel and M. Bethge, “Approximating CNNs with bag-of-local- features models works surprisingly well on imagenet,” in International Conference on Learning Representations , 2019
work page 2019
-
[2]
L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “DeepLab: Semantic Image Segmentation with Deep Con- volutional Nets, Atrous Convolution, and Fully Connected CRFs,” arXiv:1606.00915 [cs] , June 2016
arXiv 2016
-
[3]
Road: Reality oriented adaptation for semantic segmentation of urban scenes,
Y . Chen, W. Li, and L. Van Gool, “Road: Reality oriented adaptation for semantic segmentation of urban scenes,” in CVPR, 2018
work page 2018
-
[4]
The Cityscapes Dataset for Semantic Urban Scene Understanding,
M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Be- nenson, U. Franke, S. Roth, and B. Schiele, “The Cityscapes Dataset for Semantic Urban Scene Understanding,” in CVPR, 2016
work page 2016
-
[5]
Model adaptation with synthetic and real data for semantic dense foggy scene under- standing,
D. Dai, C. Sakaridis, S. Hecker, and L. Van Gool, “Model adaptation with synthetic and real data for semantic dense foggy scene under- standing,” International Journal of Computer Vision (IJCV) , 2019
work page 2019
-
[6]
Dark model adaptation: Semantic image segmentation from daytime to nighttime,
D. Dai and L. Van Gool, “Dark model adaptation: Semantic image segmentation from daytime to nighttime,” in IEEE International Conference on Intelligent Transportation Systems , 2018
work page 2018
-
[7]
CARLA: An Open Urban Driving Simulator,
A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V . Koltun, “CARLA: An Open Urban Driving Simulator,” in Annual Conference on Robot Learning , vol. 78, 2017
work page 2017
-
[8]
Unsupervised Domain Adaptation by Backpropagation,
Y . Ganin and V . Lempitsky, “Unsupervised Domain Adaptation by Backpropagation,” in Proceedings of International Conference on International Conference on Machine Learning , 2015
work page 2015
Show all 34 references
-
[9]
Domain-Adversarial Training of Neural Networks,
Y . Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Lavio- lette, M. Marchand, and V . Lempitsky, “Domain-Adversarial Training of Neural Networks,” arXiv:1505.07818 [cs, stat] , May 2015
2015 arXiv
-
[10]
Texture and art with deep neural networks,
L. A. Gatys, A. S. Ecker, and M. Bethge, “Texture and art with deep neural networks,” Current Opinion in Neurobiology , vol. 46, pp. 178– 186, Oct. 2017
2017
-
[11]
ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness,
R. Geirhos, P. Rubisch, C. Michaelis, M. Bethge, F. A. Wich- mann, and W. Brendel, “ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness,” arXiv:1811.12231 [cs, q-bio, stat] , Nov. 2018
2018 arXiv
-
[12]
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition,
K. He, X. Zhang, S. Ren, and J. Sun, “Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition,” arXiv:1406.4729 [cs], vol. 8691, pp. 346–361, 2014
2014 arXiv
-
[13]
Deep residual learning for image recognition,
——, “Deep residual learning for image recognition,” in IEEE Con- ference on Computer Vision and Pattern Recognition , 2016
2016
-
[14]
CyCADA: Cycle-consistent adversarial domain adap- tation,
J. Hoffman, E. Tzeng, T. Park, J.-Y . Zhu, P. Isola, K. Saenko, A. Efros, and T. Darrell, “CyCADA: Cycle-consistent adversarial domain adap- tation,” in International Conference on Machine Learning , 2018
2018
-
[15]
Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization,
X. Huang and S. Belongie, “Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization,” in IEEE International Conference on Computer Vision (ICCV) , 2017
2017
-
[16]
Deep Neural Networks: A New Framework for Mod- eling Biological Vision and Brain Information Processing,
N. Kriegeskorte, “Deep Neural Networks: A New Framework for Mod- eling Biological Vision and Brain Information Processing,” Annual Review of Vision Science , vol. 1, no. 1, pp. 417–446, 2015
2015
-
[17]
Deep learning,
Y . LeCun, Y . Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, May 2015
2015
-
[18]
Fully convolutional networks for semantic segmentation,
J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in CVPR, 2015
2015
-
[19]
The Map- illary Vistas Dataset for Semantic Understanding of Street Scenes,
G. Neuhold, T. Ollmann, S. R. Bulo, and P. Kontschieder, “The Map- illary Vistas Dataset for Semantic Understanding of Street Scenes,” in 2017 IEEE International Conference on Computer Vision (ICCV) . Venice: IEEE, Oct. 2017, pp. 5000–5009
2017
-
[20]
A Survey on Transfer Learning,
S. J. Pan and Q. Yang, “A Survey on Transfer Learning,” IEEE Transactions on Knowledge and Data Engineering , vol. 22, no. 10, pp. 1345–1359, Oct. 2010
2010
-
[21]
Playing for Benchmarks,
S. R. Richter, Z. Hayder, and V . Koltun, “Playing for Benchmarks,” in IEEE International Conference on Computer Vision (ICCV) , 2017
2017
-
[22]
Playing for Data: Ground Truth from Computer Games,
S. R. Richter, V . Vineet, S. Roth, and V . Koltun, “Playing for Data: Ground Truth from Computer Games,” in Computer Vision ECCV 2016, B. Leibe, J. Matas, N. Sebe, and M. Welling, Eds. Cham: Springer International Publishing, 2016, vol. 9906, pp. 102–118
2016
-
[23]
The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes,
G. Ros, L. Sellart, J. Materzynska, D. Vazquez, and A. M. Lopez, “The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Las Vegas, NV , USA: IEEE, June...
2016
-
[24]
ImageNet Large Scale Visual Recognition Challenge,
O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei, “ImageNet Large Scale Visual Recognition Challenge,” International Journal of Computer Vision (IJCV) , vol. 115, no. 3, pp. 211–252, 2015
2015
-
[25]
Semantic foggy scene understanding with synthetic data,
C. Sakaridis, D. Dai, and L. Van Gool, “Semantic foggy scene understanding with synthetic data,” International Journal of Computer Vision, 2018
2018
-
[26]
Guided curriculum model adaptation and uncertainty-aware evaluation for semantic nighttime image segmentation,
——, “Guided curriculum model adaptation and uncertainty-aware evaluation for semantic nighttime image segmentation,” in Interna- tional Conference on Computer Vision (ICCV) , 2019
2019
-
[27]
Learning from simulated and unsupervised images through adversarial training,
A. Shrivastava, T. Pfister, O. Tuzel, J. Susskind, W. Wang, and R. Webb, “Learning from simulated and unsupervised images through adversarial training,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2017
2017
-
[28]
Learning to Adapt Structured Output Space for Semantic Segmentation,
Y .-H. Tsai, W.-C. Hung, S. Schulter, K. Sohn, M.-H. Yang, and M. Chandraker, “Learning to Adapt Structured Output Space for Semantic Segmentation,” in IEEE Conference on Computer Vision and Pattern Recognition, 2018
2018
-
[29]
Domain Adap- tation for Structured Output via Discriminative Patch Representations,
Y .-H. Tsai, K. Sohn, S. Schulter, and M. Chandraker, “Domain Adap- tation for Structured Output via Discriminative Patch Representations,” arXiv:1901.05427 [cs] , Jan. 2019
1901 arXiv
-
[30]
Simultaneous deep transfer across domains and tasks,
E. Tzeng, J. Hoffman, T. Darrell, and K. Saenko, “Simultaneous deep transfer across domains and tasks,” in IEEE International Conference on Computer Vision (ICCV) , 2015
2015
-
[31]
Deep visual domain adaptation: A survey,
M. Wang and W. Deng, “Deep visual domain adaptation: A survey,” Neurocomputing, vol. 312, pp. 135–153, Oct. 2018
2018
-
[32]
Synscapes: A Photorealistic Synthetic Dataset for Street Scene Parsing,
M. Wrenninge and J. Unger, “Synscapes: A Photorealistic Synthetic Dataset for Street Scene Parsing,” arXiv:1810.08705 [cs] , Oct. 2018
2018 arXiv
-
[33]
Multi-Scale Context Aggregation by Dilated Convolutions,
F. Yu and V . Koltun, “Multi-Scale Context Aggregation by Dilated Convolutions,” in ICLR, 2016
2016
-
[34]
Curriculum domain adaptation for semantic segmentation of urban scenes,
Y . Zhang, P. David, and B. Gong, “Curriculum domain adaptation for semantic segmentation of urban scenes,” in IEEE International Conferenceon Computer Vision (ICCV) , 2017
2017
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.