REVIEW 4 major objections 6 minor 31 references
Semantic Segmentation of Panoramic Images Using a Synthetic Dataset
T0 review · 4 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Training a real-time segmentation network on stitched panoramic images—especially 180-degree crops—improves accuracy and distortion resistance on panoramic test imagery compared with training on ordinary directional views.
desk verdict Useful dataset and an interesting 180-degree FoV result, but the anti-distortion experiment is internally contradictory and the paper needs major revision before its central claims can be trusted. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the SYNTHIA-PANO dataset, produced by projecting each 100-degree direction image from SYNTHIA onto a cylinder with focal length f = 532.740352 and then stitching the four projections via horizontal translations separated by d ≈ 835 pixels, estimated by scanning overlapping regions for the minimum pixel-wise discrepancy. This yields 3340-by-760, 360-degree panoramas with pixel labels intact. The argument then runs on controlled comparisons of this data: the same ICNet architecture is trained on original views versus panoramas and on 90-degree, 180-degree, and 360-degree crops, isolating the effect of training-view geometry and field of view.
What would settle it
Evaluate a SYNTHIA-PANO-trained model on real panoramic street images and compare its mIoU and distortion resistance with a model trained on the original directional SYNTHIA views; if the panorama-trained model does not improve on real images, the central claim fails. A second check: use a network whose receptive field covers the whole 360-degree panorama; if 360-degree training then beats 180-degree training, the field-of-view finding is architecture-dependent.
Extended reading notes
Core claim
The paper sets out to show that the view geometry of training data, not just the network architecture, limits panoramic semantic segmentation. Using the ICNet real-time segmenter with class-balanced training, the authors find that a model trained on four-direction SYNTHIA images mis-segments objects appearing in left or right views of a panorama, while models trained on stitched SYNTHIA-PANO images fix this defect. Among training fields of view, 180 degrees gives the best mIoU and pixel accuracy on a held-out panorama sequence, while 360 degrees underperforms it because the extreme aspect ratio and finite receptive field hurt feature extraction. The same 180-degree-panorama model also keeps higher mIoU than the baseline when test panoramas are re-projected with shorter focal lengths, meaning under stronger distortion. The claimed payoff is that panoramic training data improves both accuracy and resistance to distortion for surrounding-scene segmentation.
Load-bearing premise
The whole comparison assumes that panoramas synthetically stitched from SYNTHIA's four fixed 100-degree cameras, with a single estimated seam distance d ≈ 835, behave like real panoramic imagery well enough that conclusions about training data transfer to actual 360-degree cameras.
Editorial extensions
If this is right
- Panoramic training data lets an unmodified real-time segmenter handle off-forward views that appear in 360-degree sensing, so surrounding-perception systems need not rely on separately labeled directional images.
- A 180-degree training field of view is a practical sweet spot for this architecture: it preserves enough spatial context to recognize whole objects while avoiding the feature dilution of the 4:1 aspect-ratio 360-degree panorama.
- Panorama-trained models resist cylindrical distortion better, which matters for fisheye and annular cameras whose images are commonly unwrapped by cylindrical projection.
- The stitching method transfers labels automatically from the source images, so a labeled panorama dataset can be generated from any multi-camera synthetic sequence with known overlap, avoiding manual annotation.
Reading between the lines
- An untested implication: the 180-degree-over-360-degree result likely reflects ICNet's square kernels and finite receptive field, so a panorama-adapted architecture could make full 360-degree training the best choice.
- A related implication: because the stitching order of directions is randomized, the model is implicitly taught to treat direction as irrelevant, which may contribute to the distortion resistance and deserves isolation as its own experimental variable.
- A testable transfer: the method should be validated on real panoramic images; the paper's own conclusion acknowledges the synthetic-to-real gap, so the geometric benefit could be outweighed in practice even if it is real on synthetic data.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes SYNTHIA-PANO, a synthetic panoramic semantic segmentation dataset built by cylindrical projection and stitching of the four fixed 100-degree views in SYNTHIA-Seqs, and studies how training data view direction and FoV affect ICNet segmentation performance. The authors compare models trained on original four-direction images versus panoramic crops with 90, 180, and 360 degree FoVs, and report that 180-degree FoV training yields the best accuracy and mIoU on a panoramic test sequence. The paper further claims that the model trained on panoramic images has better anti-distortion ability, evaluated by feeding both models cylindrically projected images with focal lengths from 400 to 700.
Significance. If the empirical claims were reliable, the released dataset and simple region-matching stitching method would be a useful resource for studying panoramic semantic segmentation with standard architectures. The FoV comparison is partly controlled by fixing epoch and iteration counts and by feeding the same image content across batch sizes, and the authors ship code and dataset artifacts. However, the central claims are entirely empirical and are supported by a single synthetic test sequence with no variance reporting. The anti-distortion claim is internally contradicted by the paper's own text and figure caption, and the 180-degree optimum is selected on the same evaluation set used for final numbers. The practical relevance for real panoramic imagery is acknowledged by the authors as an open gap, which further limits the scope of the conclusions.
major comments (4)
- [Sec. 4.4 and Fig. 12] The anti-distortion experiment is presented with mutually incompatible conclusions. The text says, "When the focal length is shorter than 625, Morin performs better than Mpano180," and then says, "When the focal length is shorter than 625, the model Mpano180 trained with panoramic images always preforms better," while the caption of Fig. 12 says, "When the focal length is in 700 to 400, the mIoU of Morin is higher than that of Mpano180." These statements cannot all be true, and the abstract's third central claim about improved distortion robustness depends on Mpano180 outperforming Morin under severe distortion. This internal contradiction leaves the anti-distortion conclusion unsupported as written; the authors must reconcile the text, caption, and underlying numbers before the claim can be evaluated.
- [Sec. 4.3, Fig. 10, Table 4] The claim that 180-degree FoV is the best training choice is based on evaluation on a single sequence (Seqs05-summer) with no repeated seeds, no error bars, and no statistical comparison. The same evaluation set is used both to select the best FoV and to report the final numbers, so the result is vulnerable to selection on the test set. A controlled comparison with multiple training runs, mean and standard deviation, and a held-out validation set is needed to support the central "180-degree is best" finding.
- [Sec. 3.2, Eq. (2)] The inverse cylindrical projection in Eq. (2) does not appear to be the inverse of the forward mapping in Eq. (1). With r = f, Eq. (1) gives x' = f * atan(x/f), whose inverse is x = f * tan(x'/f); Eq. (2) instead states x = f * atan(x'/r), which is not the inverse. If this equation reflects the implemented mapping, the stitching pipeline may be incorrect. The authors should verify the forward/backward pair and report the correct equations.
- [Sec. 5 and Abstract] The abstract and conclusion generalize the findings to "panoramic images" and to practical applications such as autonomous driving, but all training and evaluation are performed on synthetic SYNTHIA-PANO images built from the same SYNTHIA source used for training. The paper itself acknowledges in the conclusion that the gap between synthetic and real data is a key limiting factor. The representativeness of these cylindrical synthetic panoramas for real panoramic imagery, including real lens distortion and stitching artifacts, is untested, so the practical conclusions overstate the evidence.
minor comments (6)
- [Sec. 3.3, Eq. (7)] The discrepancy measure in Eq. (7) sums over p in R1 and q in R2 without specifying a correspondence; presumably it computes the sum of absolute differences over aligned pixel positions, and this should be stated explicitly.
- [Sec. 3.4] The statement that "the order of the directions is randomly chosen" is unclear: if the directional order varies across panoramas, the fixed transformation structure in Eq. (3) and the equal spacing assumption in Eq. (5) need to be reconciled.
- [Sec. 4.1] The learning-rate decay policy is described only as "learning rate decay policy is used"; the specific schedule and decay parameters should be reported for reproducibility.
- [Table 4 and Sec. 4.4] There are minor typos: "orin" should be "origin" in Table 4, and "preforms" should be "performs" in Sec. 4.4.
- [Sec. 1] There are several typographical errors, including "SYHTHIA dateset" for "SYNTHIA dataset" and "a new dataset of panoramic image" in the abstract; these should be corrected.
- [References] Reference [28] lists "Shum, H.-Y." three times in the author list and should be cleaned up.
Circularity Check
No circularity: central claims are empirical comparisons; anti-distortion text contradiction is a correctness issue, not a circular one.
full rationale
The paper's central claims are empirical comparisons against independent baselines. The stitching pipeline (cylindrical projection with r=f=532.740352 and d estimated by region matching) is a dataset-construction step, and the trained models are then compared on held-out SYNTHIA-PANO Seqs05; no quantity is fitted and then renamed as a prediction. The FoV comparison (Morin vs Mpano90/180/360) is likewise an empirical sweep, not a derivation from the projection equations. The only near-circular concern is the anti-distortion evaluation: Sec. 3.3 fixes the focal length for the training panoramas, while Sec. 4.4 generates evaluation images by cylindrical projection with f varying from 700 to 400, so Mpano180 is tested in the same image-formation family in which it was trained, a domain-match confound relative to Morin. However, the paper does not define 'anti-distortion' as 'performance on cylindrical projection' nor derive the result by equation; it is an experimental outcome, so this is not circularity by the paper's own construction. More importantly, Sec. 4.4 contains mutually contradictory sentences ('When the focal length is shorter than 625, Morin performs better than Mpano180' vs 'When the focal length is shorter than 625, the model Mpano180 trained with panoramic images always preforms better' vs Fig. 12 caption 'When the focal length is in 700 to 400, the mIoU of Morin is higher than that of Mpano180'), which undermines the anti-distortion claim as a correctness matter, not a circularity matter. Self-citations in Sec. 2 are non-load-bearing related-work references. Therefore no significant circularity; score 0.
Assumptions & free parameters
free parameters (2)
- d (stitching translation) =
835 pixels
- Training FoV =
180 degrees
assumptions (5)
- domain assumption The four SYNTHIA-Seqs cameras are fixed with 90-degree angular separation and identical height, so after cylindrical projection a single horizontal translation d stitches all four images (Eqs. 5 and 6).
- standard math The backward cylindrical projection in Eq. 2 is the inverse of the forward mapping in Eq. 1.
- domain assumption Stitching the label images with the same transformations as RGB images gives correct pixel-level ground truth.
- domain assumption SYNTHIA-PANO is representative enough of panoramic driving scenes to support conclusions about segmentation accuracy and distortion robustness.
- domain assumption Training runs with fixed epochs and iterations converge comparably across models.
Cite this review
Pith. "Pith review of Semantic Segmentation of Panoramic Images Using a Synthetic Dataset." pith.science (2026). https://pith.science/paper/ZSZXVU3M
@misc{pith2026190900532,
author = {Pith},
title = {Pith review of: Semantic Segmentation of Panoramic Images Using a Synthetic Dataset},
year = {2026},
howpublished = {\url{https://pith.science/paper/ZSZXVU3M}},
note = {Machine review of arXiv:1909.00532}
}
read the original abstract
Panoramic images have advantages in information capacity and scene stability due to their large field of view (FoV). In this paper, we propose a method to synthesize a new dataset of panoramic image. We managed to stitch the images taken from different directions into panoramic images, together with their labeled images, to yield the panoramic semantic segmentation dataset denominated as SYNTHIA-PANO. For the purpose of finding out the effect of using panoramic images as training dataset, we designed and performed a comprehensive set of experiments. Experimental results show that using panoramic images as training data is beneficial to the segmentation result. In addition, it has been shown that by using panoramic images with a 180 degree FoV as training data the model has better performance. Furthermore, the model trained with panoramic images also has a better capacity to resist the image distortion.
Figures
Figures from the paper (9 more)
Reference graph
Works this paper leans on
-
[1]
Unifying terrain awareness through real-time semantic segmentation,
Yang, K., Bergasa, L. M., Romera, E., Cheng, R., Chen, T., and Wang, K., “Unifying terrain awareness through real-time semantic segmentation,” in [2018 IEEE Intelligent Vehicles Symposium (IV) ], 1033–1038, IEEE (June 2018)
work page 2018
-
[2]
Unifying terrain awareness for the visually impaired through real-time semantic segmentation,
Yang, K., Wang, K., Bergasa, L., Romera, E., Hu, W., Sun, D., Sun, J., Cheng, R., Chen, T., and L´ opez, E., “Unifying terrain awareness for the visually impaired through real-time semantic segmentation,” Sensors 18(5), 1506 (2018)
2018
-
[3]
Yang, K., Hu, X., Bergasa, L. M., Romera, E., Huang, X., Sun, D., and Wang, K., “Can we pass beyond the field of view? panoramic annular semantic segmentation for real-world surrounding perception,” in [ 2019 IEEE Intelligent Vehicles Symposium (IV). IEEE ], 374–381 (June 2019)
work page 2019
-
[4]
Robustifying semantic cognition of traversability across wearable rgb-depth cameras,
Yang, K., Bergasa, L. M., Romera, E., and Wang, K., “Robustifying semantic cognition of traversability across wearable rgb-depth cameras,” Applied optics 58(12), 3141–3155 (2019)
2019
-
[5]
Ros, G., Sellart, L., Materzynska, J., Vazquez, D., and Lopez, A. M., “The synthia dataset: A large collection of synthetic images for semantic segmentation of urban scenes,” in [ The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) ], (June 2016)
work page 2016
-
[6]
Icnet for real-time semantic segmentation on high-resolution images,
Zhao, H., Qi, X., Shen, X., Shi, J., and Jia, J., “Icnet for real-time semantic segmentation on high-resolution images,” in [ Proceedings of the European Conference on Computer Vision (ECCV) ], 405–420 (2018)
2018
-
[7]
Omnidirectional multisensory perception fusion for long-term place recognition,
Siva, S. and Zhang, H., “Omnidirectional multisensory perception fusion for long-term place recognition,” in [2018 IEEE International Conference on Robotics and Automation (ICRA) ], 1–9 (May 2018)
work page 2018
-
[8]
Cheng, R., Wang, K., Lin, S., Hu, W., Yang, K., Huang, X., Li, H., Sun, D., and Bai, J., “Panoramic annular localizer: Tackling the variation challenges of outdoor localization using panoramic annular images and active deep descriptors,” arXiv preprint arXiv:1905.05425 (2019)
arXiv 2019
Show all 31 references
-
[9]
Palvo: visual odometry based on panoramic annular lens,
Chen, H., Wang, K., Hu, W., Yang, K., Cheng, R., Huang, X., and Bai, J., “Palvo: visual odometry based on panoramic annular lens,” Optics express (2019)
2019
-
[10]
Im2pano3d: Extrapolating 360 structure and semantics beyond the field of view,
Song, S., Zeng, A., Chang, A. X., Savva, M., Savarese, S., and Funkhouser, T., “Im2pano3d: Extrapolating 360 structure and semantics beyond the field of view,” in [2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition ], 3847–3856, IEEE (2018)
2018
-
[11]
Eliminating the blind spot: Adapting 3d object detection and monocular depth estimation to 360 panoramic imagery,
Payen de La Garanderie, G., Atapour Abarghouei, A., and Breckon, T. P., “Eliminating the blind spot: Adapting 3d object detection and monocular depth estimation to 360 panoramic imagery,” in [The European Conference on Computer Vision (ECCV) ], (September 2018)
2018
-
[12]
Distortion-aware convolutional filters for dense prediction in panoramic images,
Tateno, K., Navab, N., and Tombari, F., “Distortion-aware convolutional filters for dense prediction in panoramic images,” in [ Proceedings of the European Conference on Computer Vision (ECCV) ], 707–722 (2018)
2018
-
[13]
Unsupervised learning of depth and ego-motion from panoramic video,
Sharma, A. and Ventura, J., “Unsupervised learning of depth and ego-motion from panoramic video,” arXiv preprint arXiv:1901.00979 (2019)
2019 arXiv
-
[14]
Cnn based semantic segmentation for urban traffic scenes using fisheye camera,
Deng, L., Yang, M., Qian, Y., Wang, C., and Wang, B., “Cnn based semantic segmentation for urban traffic scenes using fisheye camera,” in [ 2017 IEEE Intelligent Vehicles Symposium (IV) ], 231–236 (June 2017)
2017
-
[15]
Pyramid scene parsing network,
Zhao, H., Shi, J., Qi, X., Wang, X., and Jia, J., “Pyramid scene parsing network,” in [ The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) ], (July 2017)
2017
-
[16]
Restricted deformable convolution based road scene semantic segmentation using surround view cameras,
Deng, L., Yang, M., Li, H., Li, T., Hu, B., and Wang, C., “Restricted deformable convolution based road scene semantic segmentation using surround view cameras,” arXiv preprint arXiv:1801.00708 (2018)
2018 arXiv
-
[17]
Adapting semantic segmentation models for changes in illumination and camera perspective,
Zhou, W., Zyner, A., Worrall, S., and Nebot, E., “Adapting semantic segmentation models for changes in illumination and camera perspective,” IEEE Robotics and Automation Letters 4, 461–468 (April 2019)
2019
-
[18]
Super-sensor for 360-degree environment perception: Point cloud segmentation using image features,
Varga, R., Costea, A., Florea, H., Giosan, I., and Nedevschi, S., “Super-sensor for 360-degree environment perception: Point cloud segmentation using image features,” in [ 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC) ], 1–8 (Oct 2017)
2017
-
[19]
Understanding 3d semantic structure around the vehicle with monocular cameras,
Narioka, K., Nishimura, H., Itamochi, T., and Inomata, T., “Understanding 3d semantic structure around the vehicle with monocular cameras,” in [ 2018 IEEE Intelligent Vehicles Symposium (IV) ], 132–137 (June 2018)
2018
-
[20]
The cityscapes dataset for semantic urban scene understanding,
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., and Schiele, B., “The cityscapes dataset for semantic urban scene understanding,” in [ The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) ], (June 2016)
2016
-
[21]
The mapillary vistas dataset for semantic understanding of street scenes,
Neuhold, G., Ollmann, T., Rota Bulo, S., and Kontschieder, P., “The mapillary vistas dataset for semantic understanding of street scenes,” in [The IEEE International Conference on Computer Vision (ICCV) ], (Oct 2017)
2017
-
[22]
Orientation-aware semantic segmentation on icosahe- dron spheres,
Zhang, C., Liwicki, S., Smith, W., and Cipolla, R., “Orientation-aware semantic segmentation on icosahe- dron spheres,” arXiv preprint arXiv:1907.12849 (2019)
2019 arXiv
-
[23]
Train here, deploy there: Robust segmentation in unseen domains,
Romera, E., Bergasa, L. M., Alvarez, J. M., and Trivedi, M., “Train here, deploy there: Robust segmentation in unseen domains,” in [ 2018 IEEE Intelligent Vehicles Symposium (IV) ], 1828–1833 (June 2018)
2018
-
[24]
Bridging the day and night domain gap for semantic segmentation,
Romera, E., Bergasa, L. M., Yang, K., Alvarez, J. M., and Barea, R., “Bridging the day and night domain gap for semantic segmentation,” in [ 2019 IEEE Intelligent Vehicles Symposium (IV). IEEE ], 1184–1190 (June 2019)
2019
-
[25]
Acnet: Attention based network to exploit complementary features for rgbd semantic segmentation,
Hu, X., Yang, K., Fei, L., and Wang, K., “Acnet: Attention based network to exploit complementary features for rgbd semantic segmentation,” arXiv preprint arXiv:1905.10089 (2019)
2019 arXiv
-
[26]
Importance-aware semantic segmentation with efficient pyramidal context network for navigational assistant systems,
Xiang, K., Wang, K., and Yang, K., “Importance-aware semantic segmentation with efficient pyramidal context network for navigational assistant systems,” arXiv preprint arXiv:1907.11066 (2019)
2019 arXiv
-
[27]
A comparative study of high-recall real-time semantic segmentation based on swift factorized network,
Xiang, K., Wang, K., and Yang, K., “A comparative study of high-recall real-time semantic segmentation based on swift factorized network,” arXiv preprint arXiv:1907.11394 (2019)
2019 arXiv
-
[28]
Creating full view panoramic image mosaics and environment maps,
Szeliski, R., Shum, H.-Y., Shum, H.-Y., and Shum, H.-Y., “Creating full view panoramic image mosaics and environment maps,” in [ Proceedings of the 24th annual conference on Computer graphics and interactive techniques], 251–258, ACM Press/Addison-Wesley Publishing Co. (1997)
1997
-
[29]
Image alignment and stitching: A tutorial,
Szeliski, R. et al., “Image alignment and stitching: A tutorial,” Foundations and Trends R© in Computer Graphics and Vision 2(1), 1–104 (2007)
2007
-
[30]
Orb: An efficient alternative to sift or surf.,
Rublee, E., Rabaud, V., Konolige, K., and Bradski, G. R., “Orb: An efficient alternative to sift or surf.,” in [ICCV], 11(1), 2, Citeseer (2011)
2011
-
[31]
Adam: A method for stochastic optimization,
Kingma, D. P. and Ba, J., “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980 (2014)
2014 arXiv
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.