REVIEW 4 major objections 6 minor 29 references
360-Degree Full-view Image Segmentation by Spherical Convolution compatible with Large-scale Planar Pre-trained Models
T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read This paper claims that a fixed spherical resampling of an equirectangular panorama, computed only from the image width, lets an unmodified planar ConvNeXt pretrained on ordinary photos reach state-of-the-art semantic segmentation on…
desk verdict A plausible idea for reusing planar pretrained models on 360° imagery, but the spherical sampling geometry is internally inconsistent and the headline experiment uses an incomplete model. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The carrying mechanism is a spherical discrete sampling of the equirectangular input: for the $4\times4$ convolution, two circles centered at the north pole, with radii $\frac{2\pi}{W}\cdot 2$ and $\frac{2\pi}{3W/2}$ and four and twelve sampled points respectively, are rotated to the kernel's center and back-projected to ERP coordinates; for $7\times7$ convolution the image is expanded sevenfold and three circles with spacings $\pi/8$, $\pi/16$, $\pi/24$ are used, with the stride changed to 7. These point positions can be precomputed and stored as a lookup table, so adapting the pretrained model costs only bilinear interpolation at runtime. The second component is a dual-branch attention head that computes per-channel masks $m_i$ from spherical features and fuses them into the planar predictions as $\mathrm{sem} = \mathrm{ReLU}(\mathrm{sem}_i + \mathrm{sem}_i \ast m_i)$.
What would settle it
A reader could run the full model at 512x1024 on Stanford2D3D with all four spherical downsampling layers and compare its mIoU against (a) the reported 54.58, (b) a plain ConvNeXt trained on the original ERP image with the same decoder, and (c) SGAT4PASS's 55.30; if the spherical branch does not beat the plain ERP baseline, the claimed distortion correction is not producing the reported gains.
Extended reading notes
Core claim
The central claim is that planar convolution kernels can be made spherical by resampling the ERP image itself. For each kernel size the paper selects spherical points around the kernel's center, with circle radii derived from the image width $W$ and point spacings of $\pi/4$ and $\pi/12$ for $4\times4$ kernels and $\pi/8$, $\pi/16$, $\pi/24$ for $7\times7$ kernels, projects those points back to ERP coordinates, and rearranges the sampled pixels into the original kernel-shaped matrix. The pretrained ConvNeXt weights are then used unchanged, because the network's view of each neighborhood is spherical rather than planar. On Stanford2D3D the full dual-branch model achieves 53.91 mIoU with RGB-D and 52.03 mIoU with RGB at $256\times512$, while an incomplete model at $512\times1024$ reports 54.58 mIoU, surpassing several published baselines while remaining below SGAT4PASS's 55.30.
Load-bearing premise
The load-bearing premise is that the fixed spherical resampling computed from the panorama width alone maps standard planar convolution kernels onto spherical neighborhoods faithfully enough that unmodified pretrained ConvNeXt weights keep working for segmentation, a claim the paper never tests with the complete model at 512x1024, the resolution where it reports comparisons against complete state-of-the-art systems.
Editorial extensions
If this is right
- If the spherical resampling works as claimed, any pretrained planar backbone can be adapted to 360-degree data by resampling the input image, with no architectural changes and no extra branch for distortion removal.
- The reported 66%-to-96% jump on MNIST when pretrained weights are added suggests the pretrained weights remain usable after spherical resampling, and that the same recipe may transfer to other planar backbones.
- At $256\times512$, the full model outperforms PanelNet by 5.73 mIoU in RGB input and PanoFormer by 5.01 mIoU in RGB-D input, indicating the dual-branch attention adds value beyond the spherical backbone alone.
- At $512\times1024$, even the incomplete model reports mIoU above PanoFormer, CBFC, SFSS, Trans4PASS+, and 360BEV, though below SGAT4PASS, so the method is positioned as a strong RGB-only panorama segmentation baseline.
Reading between the lines
- A natural next test is applying the same input resampling to ResNet or ViT backbones; the paper's recipe is tied to ConvNeXt's specific kernel sizes, so the generality of 'planar pretrained models' remains an open extension.
- If the resampling is as cheap as a precomputed lookup table plus bilinear interpolation, the method could also be dropped into other dense 360-degree tasks such as depth estimation or object detection without retraining the backbone.
- The reported $512\times1024$ result comes from a deliberately incomplete model using only the first spherical feature layer, so the full-model comparison at the standard resolution is a direct, feasible check of whether the claimed direct utilization holds at scale.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a spherical sampling method that rearranges equirectangular panoramic images so that unmodified planar pretrained ConvNeXt models can be applied 'directly' without additional distortion modules. The method constructs spherical convolution kernels by selecting points on a sphere and back-projecting them to the ERP plane, with the mapping precomputed as a lookup table. For semantic segmentation, the authors add a spherical branch whose features generate per-channel attention masks that modulate the planar backbone's predictions. Experiments are reported on Stanford2D3D at 256x512 and 512x1024 resolutions, claiming improved or competitive mIoU against several prior methods.
Significance. If the proposed construction were valid, the idea of reusing large planar pretrained backbones for panoramic segmentation through a fixed spherical resampling would be practically appealing: it avoids additional trainable distortion modules and could make panoramic fine-tuning cheaper. The offline precomputable lookup table is a sensible engineering choice, and the MNIST ablation (Section 4.4) provides some evidence that pretrained weights help within the spherical framework. However, the central geometric construction as written is internally inconsistent, and the headline 512x1024 results come from an incomplete model. These issues affect the core claims of the paper, so the current significance is not established.
major comments (4)
- [Section 3.2] The radii for the 4x4 spherical kernel are geometrically inverted. The text states that the 'inner circle' has radius 2π/W * 2 (i.e., 4π/W) and the 'outer circle' has radius 2π / (3W/2) (i.e., 4π/(3W)). Since 4π/(3W) is one-third of 4π/W, the so-called outer circle lies closer to the pole than the inner circle. A valid 4-point-inner / 12-point-outer kernel requires the outer radius to be larger than the inner radius. As written, the 16 sampled points cannot be arranged into a 4x4 kernel that preserves the spatial ordering of the planar convolution kernel, so the described method is not implementable and the claim of direct compatibility with pretrained planar weights is unsupported.
- [Sections 4.1 and 4.4] The 512x1024 results in Table 1 are produced by an incomplete model. Section 4.1 states that the authors 'were unable to complete full model training at the 512×1024 resolution' and used an incomplete model; Section 4.4 specifies that at 512x1024 only Layer 1 of the spherical branch was used. Nevertheless, Section 4.2 compares these incomplete-model numbers against complete state-of-the-art systems and claims improvements (e.g., 2.18 over PanoFormer and 2.38 over CBFC in mIoU). This comparison does not support the paper's claim of superior performance for the full proposed method at 512x1024.
- [Section 4.2 vs Section 4.3] The comparison protocol for SGAT4PASS is inconsistent. Section 4.2 says that Table 1 uses the original results from SGAT4PASS, while the same section says the authors were unable to reproduce SGAT4PASS's reported performance and Section 4.3 states that the SGAT4PASS results shown in the figures were reproduced by the authors. Mixing original-paper numbers with self-reproduced numbers without a standardized evaluation protocol makes the quantitative comparison unreliable.
- [Table 1 and Section 4.2] No error bars or repeated-run statistics are reported. Several claimed advantages are very small (e.g., 0.11 mIoU over CBFC at 256x512), and on a dataset of 1,413 images such differences may be within random variation. Without variance estimates, the 'superior performance' claim is not statistically supported.
minor comments (6)
- [Equation (1)] Equation (1) is not properly formatted and its variables are not defined; please clarify the projection formula, the meaning of (x', y', z'), and the convention for arctan2.
- [Section 3.2] The phrase 'the 2.5th row and column as a center' is unclear; specify how the fractional row/column index corresponds to a spherical coordinate.
- [References] References [3] and [17] are duplicates of the same paper (Tateno et al., ECCV 2018); please merge or remove the duplicate.
- [Throughout] There are numerous typographical and grammatical errors, such as 'futhermore', 'each blocks is formed', and 'we proposes'; the manuscript would benefit from thorough proofreading.
- [Table 2] The header 'Base layer1 layer2 layer3 layer4 Cat' is ambiguous; please label the columns clearly and describe what each checkmark indicates.
- [Section 4.4] The MNIST ablation is not clearly connected to spherical sampling; explain how planar MNIST digits are mapped to the sphere and what 'spherical pre-training model' means in this experiment.
Circularity Check
No significant circularity: the spherical sampling geometry is a fixed geometric remapping, not a fitted quantity, and the empirical claims rest on external benchmarks.
full rationale
The derivation chain is not circular. The spherical sampling positions in Section 3.2 are fixed functions of the ERP width W and the pretrained kernel sizes (2, 4, and 7); they are not optimized against the Stanford2D3D labels or the segmentation predictions, so no fitted parameter is relabeled as a prediction. The claim of compatibility with planar pretrained weights is implemented by resampling the input before the unmodified ConvNeXt backbone, and the method's success is then tested against external baselines in Table 1, not entailed by the construction. The only citation with author overlap is [22], which is used as design inspiration ('Inspired by previous works[22]') for spherical kernels; the paper's own ablation (MNIST 66% to 96%, Table 2 layer and fusion ablations) and Stanford2D3D comparisons carry the empirical claim, so the self-citation is not load-bearing circularity. The admitted limitations—training an incomplete model at 512x1024 and using unreproduced SGAT4PASS numbers in Table 1—are validity and correctness concerns, not instances of the derivation reducing to its inputs.
Assumptions & free parameters
free parameters (4)
- inner circle radius =
2π/W × 2 (as stated in the text)
- outer circle radius =
2π/(3W/2) (as stated in the text)
- inter-point spacings =
π/4 and π/12
- center offset =
"2.5th row and column"
assumptions (3)
- standard math Equirectangular projection formula (Eq. 1) correctly maps sphere points to image coordinates
- domain assumption Pretrained planar convolution weights transfer to spherically resampled inputs without adaptation
- ad hoc to paper The chosen radii and spacings provide adequate spherical sampling density
Cite this review
Pith. "Pith review of 360-Degree Full-view Image Segmentation by Spherical Convolution compatible with Large-scale Planar Pre-trained Models." pith.science (2026). https://pith.science/paper/UJG6QT2Y
@misc{pith2026250709216,
author = {Pith},
title = {Pith review of: 360-Degree Full-view Image Segmentation by Spherical Convolution compatible with Large-scale Planar Pre-trained Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/UJG6QT2Y}},
note = {Machine review of arXiv:2507.09216}
}
read the original abstract
Due to the current lack of large-scale datasets at the million-scale level, tasks involving panoramic images predominantly rely on existing two-dimensional pre-trained image benchmark models as backbone networks. However, these networks are not equipped to recognize the distortions and discontinuities inherent in panoramic images, which adversely affects their performance in such tasks. In this paper, we introduce a novel spherical sampling method for panoramic images that enables the direct utilization of existing pre-trained models developed for two-dimensional images. Our method employs spherical discrete sampling based on the weights of the pre-trained models, effectively mitigating distortions while achieving favorable initial training values. Additionally, we apply the proposed sampling method to panoramic image segmentation, utilizing features obtained from the spherical model as masks for specific channel attentions, which yields commendable results on commonly used indoor datasets, Stanford2D3D.
Figures
Reference graph
Works this paper leans on
-
[1]
Omnidirectional stereo depth estimation based on spherical deep network,
M. Li, X. Hu, J. Dai, Y . Li, and S. Du, “Omnidirectional stereo depth estimation based on spherical deep network,” Image and Vision Computing, vol. 114, p. 104264, 2021
work page 2021
-
[2]
N. Perraudin, M. Defferrard, T. Kacprzak, and R. Sgier, “Deepsphere: Efficient spherical convolutional neural network with healpix sampling for cosmological applications,” Astronomy and Computing, vol. 27, pp. 130–146, 2019
work page 2019
-
[4]
X. Zheng, J. Zhu, Y . Liu, Z. Cao, C. Fu, and L. Wang, “Both style and distortion matter: Dual-path unsupervised domain adaptation for panoramic semantic segmentation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 1285–1295
work page 2023
-
[5]
X. Zheng, T. Pan, Y . Luo, and L. Wang, “Look at the neighbor: Distortion-aware unsupervised domain adaptation for panoramic semantic segmentation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 18 687–18 698
work page 2023
-
[6]
Bending reality: Distortion-aware transformers for adapting to panoramic semantic segmentation,
J. Zhang, K. Yang, C. Ma, S. Reiß, K. Peng, and R. Stiefelhagen, “Bending reality: Distortion-aware transformers for adapting to panoramic semantic segmentation,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 16 917–16 927
work page 2022
-
[7]
Single frame semantic segmentation using multi-modal spherical images,
S. Guttikonda and J. Rambach, “Single frame semantic segmentation using multi-modal spherical images,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , 2024, pp. 3222–3231
work page 2024
-
[8]
Orientation-aware semantic segmentation on icosahedron spheres,
C. Zhang, S. Liwicki, W. Smith, and R. Cipolla, “Orientation-aware semantic segmentation on icosahedron spheres,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2019, pp. 3533–3541
work page 2019
-
[9]
Tangent images for mitigating spherical distortion,
M. Eder, M. Shvets, J. Lim, and J.-M. Frahm, “Tangent images for mitigating spherical distortion,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2020, pp. 12 426–12 434
work page 2020
Show all 29 references
-
[10]
Spherephd: Applying cnns on a spherical polyhedron representation of 360deg images,
Y . Lee, J. Jeong, J. Yun, W. Cho, and K.-J. Yoon, “Spherephd: Applying cnns on a spherical polyhedron representation of 360deg images,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019
2019
-
[11]
Complementary bi-directional feature compression for indoor 360deg semantic segmentation with self-distillation,
Z. Zheng, C. Lin, L. Nie, K. Liao, Z. Shen, and Y . Zhao, “Complementary bi-directional feature compression for indoor 360deg semantic segmentation with self-distillation,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , 2023, pp. 4501–4510
2023
-
[12]
Sgat4pass: spherical geometry-aware transformer for panoramic semantic segmentation,
X. Li, T. Wu, Z. Qi, G. Wang, Y . Shan, and X. Li, “Sgat4pass: spherical geometry-aware transformer for panoramic semantic segmentation,” arXiv preprint arXiv:2306.03403, 2023
2023 arXiv
-
[13]
Joint 2d-3d-semantic data for indoor scene understanding,
I. Armeni, S. Sax, A. R. Zamir, and S. Savarese, “Joint 2d-3d-semantic data for indoor scene understanding,” arXiv preprint arXiv:1702.01105, 2017
2017 arXiv
-
[14]
Hohonet: 360 indoor holistic understanding with latent horizontal features,
C. Sun, M. Sun, and H.-T. Chen, “Hohonet: 360 indoor holistic understanding with latent horizontal features,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021, pp. 2573–2582
2021
-
[15]
Panoformer: Panorama transformer for indoor 360 depth estimation,
Z. Shen, C. Lin, K. Liao, L. Nie, Z. Zheng, and Y . Zhao, “Panoformer: Panorama transformer for indoor 360 depth estimation,” in European Conference on Computer Vision. Springer, 2022, pp. 195–211
2022
-
[16]
Behind every domain there is a shift: Adapting distortion-aware vision transformers for panoramic semantic segmentation,
J. Zhang, K. Yang, H. Shi, S. Reiß, K. Peng, C. Ma, H. Fu, P. H. Torr, K. Wang, and R. Stiefelhagen, “Behind every domain there is a shift: Adapting distortion-aware vision transformers for panoramic semantic segmentation,” arXiv preprint arXiv:2207.11860, 2022. 8 Running Titl...
2022 arXiv
-
[17]
Distortion-aware convolutional filters for dense prediction in panoramic images,
K. Tateno, N. Navab, and F. Tombari, “Distortion-aware convolutional filters for dense prediction in panoramic images,” in Proceedings of the European Conference on Computer Vision (ECCV) , 2018, pp. 707–722
2018
-
[18]
Pass: Panoramic annular semantic segmentation,
K. Yang, X. Hu, L. M. Bergasa, E. Romera, and K. Wang, “Pass: Panoramic annular semantic segmentation,” IEEE Transactions on Intelligent Transportation Systems, vol. 21, no. 10, pp. 4171–4185, 2019
2019
-
[19]
Panelnet: Understanding 360 indoor environment via panel representation,
H. Yu, L. He, B. Jian, W. Feng, and S. Liu, “Panelnet: Understanding 360 indoor environment via panel representation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2023, pp. 878–887
2023
-
[20]
Osrt: Omnidirectional image super-resolution with distortion-aware transformer,
F. Yu, X. Wang, M. Cao, G. Li, Y . Shan, and C. Dong, “Osrt: Omnidirectional image super-resolution with distortion-aware transformer,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 13 283–13 292
2023
-
[21]
Spherical convolution empowered viewport prediction in 360 video multicast with limited fov feedback,
J. Li, L. Han, C. Zhang, Q. Li, and Z. Liu, “Spherical convolution empowered viewport prediction in 360 video multicast with limited fov feedback,” ACM Transactions on Multimedia Computing, Communications and Applications, vol. 19, no. 1, pp. 1–23, 2023
2023
-
[22]
Estimating depth of monocular panoramic image with teacher-student model fusing equirectangular and spherical representations,
J. Liu, Y . Xu, S. Li, and J. Li, “Estimating depth of monocular panoramic image with teacher-student model fusing equirectangular and spherical representations,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2024, ...
2024
-
[23]
Bifuse: Monocular 360 depth estimation via bi-projection fusion,
F.-E. Wang, Y .-H. Yeh, M. Sun, W.-C. Chiu, and Y .-H. Tsai, “Bifuse: Monocular 360 depth estimation via bi-projection fusion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2020, pp. 462–471
2020
-
[24]
Unifuse: Unidirectional fusion for 360 panorama depth estimation,
H. Jiang, Z. Sheng, S. Zhu, Z. Dong, and R. Huang, “Unifuse: Unidirectional fusion for 360 panorama depth estimation,” IEEE Robotics and Automation Letters , vol. 6, no. 2, pp. 1519–1526, 2021
2021
-
[25]
Hrdfuse: Monocular 360deg depth estimation by collaboratively learning holistic-with-regional depth distributions,
H. Ai, Z. Cao, Y .-P. Cao, Y . Shan, and L. Wang, “Hrdfuse: Monocular 360deg depth estimation by collaboratively learning holistic-with-regional depth distributions,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 13 273–13 282
2023
-
[26]
Deep residual learning for image recognition,
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2016
2016
-
[27]
An image is worth 16x16 words: Transformers for image recognition at scale,
A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929, 2020
2010 arXiv
-
[28]
A convnet for the 2020s,
Z. Liu, H. Mao, C.-Y . Wu, C. Feichtenhofer, T. Darrell, and S. Xie, “A convnet for the 2020s,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2022, pp. 11 976–11 986
2022
-
[29]
360bev: Panoramic semantic mapping for indoor bird’s-eye view,
Z. Teng, J. Zhang, K. Yang, K. Peng, H. Shi, S. Reiß, K. Cao, and R. Stiefelhagen, “360bev: Panoramic semantic mapping for indoor bird’s-eye view,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2024, pp. 373–382
2024
-
[30]
Spherical cnns,
T. S. Cohen, M. Geiger, J. Köhler, and M. Welling, “Spherical cnns,” arXiv preprint arXiv:1801.10130, 2018. 9
2018 arXiv
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.