REVIEW 4 major objections 5 minor 36 references
Super-resolution of Omnidirectional Images Using Adversarial Learning
T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read A GAN that accounts for sphere geometry sharpens 360-degree VR images
desk verdict A competent, incremental GAN SR paper for omnidirectional images whose central spherical-quality claim is weakened by the training loss doubling as the evaluation metric. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The two load-bearing components are the PatchGAN discriminator and the 360-SS loss. The PatchGAN discriminator classifies each overlapping 70 by 70 patch of the input separately and averages the results, so it has fewer parameters than a full-image discriminator and pushes the generator to make local, fine-scale structure convincing. The 360-SS loss is a structural-similarity term in which each pixel's SSIM contribution is multiplied by the cosine weight of the equirectangular projection, a weight derived from the stretching ratio of the planar-to-spherical mapping. That weighting is what redirects the optimization toward preserving detail where the sphere is actually observed, particularly near the poles.
What would settle it
Train the same generator with the cosine weight replaced by (a) no weight and (b) an area-based weight from a different projection, then compare WS-SSIM and WS-PSNR on polar-heavy test crops; if cosine weighting does not beat both, the specific mechanism claimed for polar detail is not confirmed.
Extended reading notes
Core claim
The paper's claim is that a generative adversarial network optimized for the spherical observation space can reconstruct high-resolution omnidirectional images from low-resolution equirectangular inputs more faithfully than either interpolation or a standard image super-resolution GAN. The model keeps the SRGAN generator, replaces its discriminator with a lighter PatchGAN, and adds a loss term L360-SS that measures error in spherical space by weighting SSIM with the cosine of the latitude-dependent stretch of the equirectangular projection. Reported on 500 held-out omnidirectional images, the method attains the highest WS-SSIM and WS-PSNR at all three scaling factors, and the margin over baselines grows as the scaling factor increases from 2x to 8x.
Load-bearing premise
The polar-detail claim rests on the assumption that the cosine weighting in Eq. (4) correctly measures how much the equirectangular projection distorts each pixel; if that weight misrepresents spherical geometry, the 360-SS loss would not preserve the regions the paper says it preserves.
Editorial extensions
If this is right
- At 2x, 4x, and 8x upscaling on 500 test panoramas, the 360-SS-augmented model reports the highest WS-SSIM and WS-PSNR scores among the compared methods.
- The margin over baselines grows with the scaling factor, so the spherical weighting pays off most when each output pixel covers a larger solid angle.
- Because the PatchGAN discriminator has fewer parameters than the full-image discriminator, the model trains faster and can still enforce fine-scale realism.
- The same weighting formula can be reused for other omnidirectional projections by replacing the cosine map in Eq. (4) with the appropriate stretching-ratio weights.
Reading between the lines
- If the cosine weighting is correct, this loss can be dropped into other GAN-based image enhancers for spherical video or streaming applications without retraining from scratch.
- The 8x result hints that projection-aware weighting becomes more important as per-pixel spherical distortion increases; testing at 16x or on very high-latitude crops would sharpen that picture.
- An additional test, comparing cosine weighting against unweighted SSIM and against alternative area-based weights, would isolate the contribution of the specific functional form in Eq. (4).
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper addresses single-image super-resolution of equirectangular omnidirectional images. The authors propose a GAN with a generator similar to SRGAN and a PatchGAN discriminator, and introduce a spherical-content loss '360-SS' that weights SSIM by cosine latitude weights to account for ERP distortion. They construct a 4500-image training/test set from SUN360 and compare against NN, bicubic, and SRGAN for 2x, 4x, and 8x super-resolution, reporting SSIM, PSNR, WS-SSIM, and WS-PSNR. The central claim is that the proposed model, particularly with the 360-SS loss, yields higher omnidirectional-image quality than the baselines.
Significance. If the technical issues are resolved, the paper would provide a useful engineering contribution: it is among the first deep-learning super-resolution studies for ODIs, it releases code and a dataset, and the PatchGAN design is simple and computationally efficient. The 360-SS loss idea of incorporating spherical weighting into a perceptual loss is plausible and worth testing. However, the extent of the claimed advantage depends on resolving the sign error and the metric/loss overlap; with the manuscript as printed, the main quantitative evidence for the 360-SS term is not independent of the training objective.
major comments (4)
- [§III-B, Eqs. (1)–(4)] Equation (3) defines d_360-SS as a weighted SSIM value, which is a similarity score where higher values mean better agreement with the ground truth, and Eq. (2) averages it. Equation (1) then lists L_360-SS as a term to be minimized. As written, minimizing Eq. (2) pushes the generated image away from the ground truth. If the implemented loss uses 1 - d_360-SS or -d_360-SS, that sign convention must be stated explicitly in Eqs. (1)–(3), and the experiments re-checked under the corrected objective. This is not a notational nit: it changes the meaning of every quantitative result involving the 360-SS term.
- [§IV-A, §IV-C, Tables I–III] The evaluation metric WS-SSIM [22] is the same weighted-SSIM quantity as the proposed loss L_360-SS (up to the sign ambiguity in Major Comment 1). Since γ=10, the 'Ours+360-SS' model is explicitly optimizing the headline metric. The reported WS-SSIM gains over 'Ours' are therefore partly self-certifying: they show that the network can maximize its training objective, not that it better preserves spherical structure. The WS-PSNR gains are less directly circular and are the strongest independent evidence, but they do not disentangle the effect of the PatchGAN from the metric-aligned loss. I request an ablation or an evaluation on a spherical quality metric not present in the training objective, and a discussion of the overlap.
- [§III-B, Eq. (5)] The text states that the feature loss is a distance between VGG feature maps, but Eq. (5) is written as a plain difference without any norm, squaring, or absolute value. As printed it is not a meaningful loss, because it can be negative and is not minimized by matching features. This should be corrected to an L1 or L2 distance, and the correction is material because this term carries weight β=10.
- [Tables I–III] The reported mean differences are small and standard deviations overlap across methods; for example, at r=8 the SSIM for 'Ours+360-SS' is 0.87±0.04 versus 0.86±0.02 for SRGAN, and WS-SSIM intervals overlap. The paper does not report paired significance tests or effect sizes. Given that the quantitative comparison is the basis for the 'highest performance' claim, the authors should add paired statistical comparisons (e.g., Wilcoxon signed-rank or bootstrap confidence intervals) over the 500 test ODIs.
minor comments (5)
- [§IV-A] The metrics paragraph refers to 'S-SSIM' and 'WS-PSNR'; the intended metric is WS-SSIM [22]. Please fix the typo so it does not look like a different metric.
- [§IV-B] The sentence 'Please not that the inference time...' should read 'Please note that the inference time...'.
- [§IV-B] The reported inference time of 0.030 milliseconds per ODI is surprising; please clarify whether this is per image, per patch, or at a particular resolution, and adjust the unit if it is a typo.
- [§III-B, Eq. (4)] The definition of q_{x,y}^r would benefit from a clean typeset and an explicit statement of the role of r in the weight; as printed the parentheses are ambiguous.
- [Figure 3] The rectilinear projection crops are small; enlarged crops with an error map would make the qualitative differences easier to verify.
Circularity Check
The WS-SSIM gains anchoring the strongest claim reduce to the 360-SS training objective, which is the same cosine-weighted SSIM quantity used for evaluation.
-
self definitional
[Section III-B-a, Eqs. (2)-(4); Section IV-A Metrics; Tables I-III]
"To account for the distortion of the spherical surface, we apply the weighted-to-spherically-uniform structural similarity (WS-SSIM) [22] quality score. ... we use a novel WS-SSIM-based loss function, called l360−SS."
Eq. (2)-(3) define L360-SS as a cosine-weighted SSIM average with weights from Eq. (4), which is the same quantity as the WS-SSIM metric reported in Tables I-III. Eq. (1) trains the generator to optimize this term with gamma = 10, so the WS-SSIM gains of Ours+360-SS over SRGAN and over the Ours ablation are the expected result of optimizing the evaluation metric, not independent evidence of spherical-structure preservation. The remaining PSNR/SSIM/WS-PSNR improvements are smaller and are not significance-tested, so the headline ODI-metric claim is partially self-certifying by construction.
full rationale
The paper's central empirical claim is supported mainly by the ODI-specific metrics in Tables I-III. Section III-B-a explicitly defines the proposed 360-SS loss as a cosine-weighted SSIM, i.e., the WS-SSIM quantity later used as an evaluation metric in Section IV-A. Because the generator is trained with this term in the objective, reporting higher WS-SSIM for Ours+360-SS is a consistency check of the training objective rather than an independent test of spherical-content preservation. The paper does provide some independent evidence via PSNR, SSIM, and WS-PSNR improvements, but these are modest, correlated with the same spherical weighting, and not significance-tested. No load-bearing self-citation chain or imported uniqueness theorem was found. Overall, this is partial circularity: one of the two headline ODI metrics reduces to the training loss by construction.
Assumptions & free parameters
free parameters (2)
- beta (feature loss weight) =
10
- gamma (360-SS loss weight) =
10
assumptions (3)
- domain assumption SUN 360 Panorama Database images are representative of real omnidirectional content for training and evaluation.
- domain assumption The cosine weighting function in Eq. (4), adopted from JVET [24], correctly compensates for the distortion in equirectangular projection.
- domain assumption Low-resolution ODIs are generated by Gaussian filtering followed by down-sampling, following SRGAN [14]; this matches real low-resolution capture.
Cite this review
Pith. "Pith review of Super-resolution of Omnidirectional Images Using Adversarial Learning." pith.science (2026). https://pith.science/paper/Z7HLHZGL
@misc{pith2026190804297,
author = {Pith},
title = {Pith review of: Super-resolution of Omnidirectional Images Using Adversarial Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/Z7HLHZGL}},
note = {Machine review of arXiv:1908.04297}
}
read the original abstract
An omnidirectional image (ODI) enables viewers to look in every direction from a fixed point through a head-mounted display providing an immersive experience compared to that of a standard image. Designing immersive virtual reality systems with ODIs is challenging as they require high resolution content. In this paper, we study super-resolution for ODIs and propose an improved generative adversarial network based model which is optimized to handle the artifacts obtained in the spherical observational space. Specifically, we propose to use a fast PatchGAN discriminator, as it needs fewer parameters and improves the super-resolution at a fine scale. We also explore the generative models with adversarial learning by introducing a spherical-content specific loss function, called 360-SS. To train and test the performance of our proposed model we prepare a dataset of 4500 ODIs. Our results demonstrate the efficacy of the proposed method and identify new challenges in ODI super-resolution for future investigations.
Figures
Reference graph
Works this paper leans on
-
[22]
Weighted- to-spherically-uniform ssim objective quality evaluation for panoramic video,
Y . Zhou, M. Yu, H. Ma, H. Shao, and G. Jiang, “Weighted- to-spherically-uniform ssim objective quality evaluation for panoramic video,” in 14th IEEE International Conference on Signal Processing (ICSP) , Aug 2018, pp. 54–57
work page 2018
-
[1]
Algorithm descriptions of pro- jection format conversion and video quality metrics in 360lib,
Y . Ye, E. Alshina, and J. Boyce, “Algorithm descriptions of pro- jection format conversion and video quality metrics in 360lib,” ISO/IEC JTC1/SC29/WG11/N16888, Hobart, AU, Tech. Rep. JVET-F1003, March 2017
work page 2017
-
[2]
Towards generating ambisonics using audio-visual cue for virtual reality,
A. Rana, C. Ozcinar, and A. Smolic, “Towards generating ambisonics using audio-visual cue for virtual reality,” in 44th International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2019
work page 2019
-
[3]
Visual attention-aware omnidirectional video streaming using optimal tiles for virtual reality,
C. Ozcinar, J. Cabrera, and A. Smolic, “Visual attention-aware omnidirectional video streaming using optimal tiles for virtual reality,” IEEE Journal on Emerging and Selected Topics in Circuits and Systems (JETCAS) , vol. 9, no. 1, pp. 217–230, March 2019
work page 2019
-
[4]
Viewport-aware adaptive 360◦ video streaming using tiles for virtual reality,
C. Ozcinar, A. De Abreu, and A. Smolic, “Viewport-aware adaptive 360◦ video streaming using tiles for virtual reality,” in International Conference on Image Processing (ICIP) , Beijing, China, Sep 2017
work page 2017
-
[5]
V oronoi-based objective quality metrics for omnidirectional video,
S. Croci, C. Ozcinar, E. Zerman, J. Cabrera, and A. Smolic, “V oronoi-based objective quality metrics for omnidirectional video,” in 11th International Conference on Quality of Mul- timedia Experience (QoMEX) , 2019
work page 2019
-
[6]
Toward low-latency and ultra-reliable virtual reality,
M. S. Elbamby, C. Perfecto, M. Bennis, and K. Doppler, “Toward low-latency and ultra-reliable virtual reality,” IEEE Network, vol. 32, no. 2, pp. 78–84, March 2018
work page 2018
-
[7]
Plenoptic based super-resolution for omnidirectional image sequences,
L. Bagnato, Y . Boursier, P. Frossard, and P. Vandergheynst, “Plenoptic based super-resolution for omnidirectional image sequences,” in IEEE International Conference on Image Pro- cessing (ICIP) , Sep. 2010, pp. 2829–2832
work page 2010
Show all 36 references
-
[8]
Joint registration and super- resolution with omnidirectional images,
Z. Arican and P. Frossard, “Joint registration and super- resolution with omnidirectional images,” IEEE Transactions on Image Processing (TIP) , vol. 20, no. 11, Nov 2011
2011
-
[9]
Learning-based tone mapping operator for efficient image matching,
A. Rana, G. Valenzise, and F. Dufaux, “Learning-based tone mapping operator for efficient image matching,” IEEE Transac- tions on Multimedia (TMM) , 2018
2018
-
[10]
Optimizing tone mapping operators for keypoint detection under illumination changes,
A. Rana, G. Valenzise, and F. Dufaux, “Optimizing tone mapping operators for keypoint detection under illumination changes,” in 2016 IEEE 18th International Workshop on Mul- timedia Signal Processing (MMSP) , Sep. 2016, pp. 1–6
2016
-
[11]
Learning-Based Tone Mapping Operator for Image Matching,
A. Rana, G. Valenzise, and F. Dufaux, “Learning-Based Tone Mapping Operator for Image Matching,” in IEEE International Conference on Image Processing (ICIP). Beijing, China: IEEE, 2017
2017
-
[12]
The 2018 PIRM challenge on perceptual image super- resolution,
Y . Blau, R. Mechrez, R. Timofte, T. Michaeli, and L. Zelnik- Manor, “The 2018 PIRM challenge on perceptual image super- resolution,” arXiv cs.CV 1809.07517 , 20 Sep. 2018
2018 arXiv
-
[13]
Learning a single convo- lutional super-resolution network for multiple degradations,
K. Zhang, W. Zuo, and L. Zhang, “Learning a single convo- lutional super-resolution network for multiple degradations,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2018, pp. 3262–3271
2018
-
[14]
Photo-realistic single image super-resolution using a generative adversarial network,
C. Ledig, L. Theis, F. Husz ´ar, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, and Others, “Photo-realistic single image super-resolution using a generative adversarial network,” in Proceedings of the IEEE conference on computer vision and pat...
2017
-
[15]
ESRGAN: Enhanced Super-Resolution generative adversarial networks,
X. Wang, K. Yu, S. Wu, J. Gu, Y . Liu, C. Dong, Y . Qiao, and C. C. Loy, “ESRGAN: Enhanced Super-Resolution generative adversarial networks,” in Proceedings of the European Confer- ence on Computer Vision (ECCV) , 2018
2018
-
[16]
The relativistic discriminator: a key el- ement missing from standard (GAN),
A. Jolicoeur-Martineau, “The relativistic discriminator: a key el- ement missing from standard (GAN),” arXiv cs.LG 1807.00734 , 2 Jul. 2018
2018 arXiv
-
[17]
Image-to- image translation with conditional adversarial networks,
P. Isola, J.-Y . Zhu, T. Zhou, and A. A. Efros, “Image-to- image translation with conditional adversarial networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) , 2017, pp. 1125–1134
2017
-
[18]
Super-resolution: a com- prehensive survey,
K. Nasrollahi and T. B. Moeslund, “Super-resolution: a com- prehensive survey,” Machine vision and applications (MVA) , vol. 25, no. 6, pp. 1423–1468, 2014
2014
-
[19]
Deep learning for image super-resolution: A survey,
Z. Wang, J. Chen, and S. C. H. Hoi, “Deep learning for image super-resolution: A survey,” arXiv cs.CV 1902.06068 , 16 Feb. 2019
1902 arXiv
-
[20]
Feature learning for the image retrieval task,
A. Rana, J. Zepeda, and P. Perez, “Feature learning for the image retrieval task,” in Asian Conference on Computer Vision (ACCV). Springer, 2014, pp. 152–165
2014
-
[21]
Colornet - estimating colorfulness in natural images,
E. Zerman, A. Rana, and A. Smolic., “Colornet - estimating colorfulness in natural images,” in The International Conference on Image Processing (ICIP) , 2019
2019
-
[23]
Image quality assessment: from error visibility to structural similarity,
Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Transactions on Image Processing (TIP) , vol. 13, no. 4, pp. 600–612, April 2004
2004
-
[24]
AHG8: WS-PSNR for 360 video objective quality evaluation,
JVET, “AHG8: WS-PSNR for 360 video objective quality evaluation,” JTC1/SC29/WG11, ISO/IEC, Chengdu, CN, Tech. Rep. JVET-D0040, Oct. 2016
2016
-
[25]
Very deep convolutional networks for large-scale image recognition,
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in International Conference on Learning Representations (ICLR) , 2015
2015
-
[26]
Generative adversarial nets,
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde- Farley, S. Ozair, A. Courville, and Y . Bengio, “Generative adversarial nets,” in Advances in neural information processing systems (NeurIPS) , 2014, pp. 2672–2680
2014
-
[27]
Recogniz- ing scene viewpoint using panoramic place representation,
J. Xiao, K. A. Ehinger, A. Oliva, and A. Torralba, “Recogniz- ing scene viewpoint using panoramic place representation,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2012, pp. 2695–2702
2012
-
[28]
Weighted-to-spherically-uniform quality evaluation for omnidirectional video,
Y . Sun, A. Lu, and L. Yu, “Weighted-to-spherically-uniform quality evaluation for omnidirectional video,” IEEE Signal Processing Letters (SPL) , vol. 24, no. 9, pp. 1408–1412, Sep. 2017
2017
-
[29]
Saliency-driven omnidirectional imaging adaptive coding: Modeling and assess- ment,
G. Luz, J. Ascenso, C. Brites, and F. Pereira, “Saliency-driven omnidirectional imaging adaptive coding: Modeling and assess- ment,” in IEEE 19th International Workshop on Multimedia Signal Processing (MMSP) , Oct. 2017, pp. 1–6
2017
-
[30]
Subjective panoramic video quality assessment database for coding applications,
Y . Zhang, Y . Wang, F. Liu, Z. Liu, Y . Li, D. Yang, and Z. Chen, “Subjective panoramic video quality assessment database for coding applications,” IEEE Transactions on Broad- casting (TBC) , vol. 64, no. 2, pp. 461–473, Jun. 2018
2018
-
[31]
A Large- Scale compressed 360-degree spherical image database: From subjective quality evaluation to objective model comparison,
W. Sun, K. Gu, S. Ma, W. Zhu, N. Liu, and G. Zhai, “A Large- Scale compressed 360-degree spherical image database: From subjective quality evaluation to objective model comparison,” in IEEE 20th International Workshop on Multimedia Signal Processing (MMSP), Aug. 2018, pp. 1–6
2018
-
[32]
Automatic differentiation in PyTorch,
A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. De- Vito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer, “Automatic differentiation in PyTorch,” in Workshop Autodiff Submission in Conference on Neural Information Processing Systems (NIPS- W), 2017
2017
-
[33]
Adam: A method for stochastic optimization,
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980 , 2014
2014 arXiv
-
[34]
Graph-cut-based model for spectral- spatial classification of hyperspectral images,
Y . Tarabalka and A. Rana, “Graph-cut-based model for spectral- spatial classification of hyperspectral images,” in IEEE Geo- science and Remote Sensing Symposium (IGARSS) , July 2014, pp. 3418–3421
2014
-
[35]
An evaluation of hdr image matching under extreme illumination changes,
A. Rana, G. Valenzise, and F. Dufaux, “An evaluation of hdr image matching under extreme illumination changes,” in The International Conference on Visual Communications and Image Processing (VCIP), Chengdu, China, Nov. 2016
2016
-
[36]
Learning-based Adap- tive Tone Mapping for Keypoint Detection,
A. Rana, G. Valenzise, and F. Dufaux, “Learning-based Adap- tive Tone Mapping for Keypoint Detection,” in IEEE Interna- tional Conference on Multimedia & Expo (ICME) , Hong Kong, China, Jul. 2017
2017
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.