REVIEW 2 major objections 5 minor 33 references
OmniDS replaces rigid fisheye-to-ERP aggregation with dynamic dual-stream context fusion and multi-view consensus volumes, delivering state-of-the-art omnidirectional depth.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.5
2026-07-12 05:16 UTC pith:2N5SLAOE
load-bearing objection Solid engineering SOTA on the standard synthetic multi-fisheye benchmarks; the visibility-conflict story is well-motivated and the ablations clean, but everything is still synthetic fixed-rig. the 2 major comments →
OmniDS: Dual-Stream Context Fusion for Omnidirectional Depth from Fisheye Cameras
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper shows that iterative omnidirectional depth from multi-fisheye images becomes more accurate when rigid fixed-projection aggregation is replaced by two complementary mechanisms: (1) dynamic ERP context that reprojects dual-stream (CNN + DINOv3) features at the current depth estimate via learned view weighting and distortion-aware deformable attention, and (2) a multi-view consensus volume that fuses group-wise correlation with cross-camera feature variance under 3D regularization. Together they resolve visibility-induced multi-modal evidence that simple down-weighting cannot.
What carries the argument
Dynamic ERP Context Fusion: at each refinement step the current inverse-depth estimate drives learned-view-weighted grid sampling of CNN features and deformable cross-attention (with radial distortion bias) of DINOv3 features into a shared ERP map, fused by residual pointwise convolution; this is paired with a Multi-View Consensus Volume that concatenates group-wise correlations and feature variance across the four cameras, regularized by a 3D U-Net pyramid.
Load-bearing premise
The claim rests on the premise that gains measured only on synthetic scenes with a fixed, perfectly calibrated four-camera 220° fisheye rig will carry over to real imagery, arbitrary camera counts, and calibration noise.
What would settle it
Train and evaluate the identical architecture on a real multi-fisheye dataset with measured calibration noise and varying camera counts; if the reported error reductions versus the same baselines disappear or reverse, the central claim fails.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. OmniDS is an iterative omnidirectional depth estimator for fixed four-fisheye rigs. It replaces rigid fisheye-to-ERP aggregation with (i) a dual-stream encoder (lightweight CNN + frozen DINOv3) whose features are reprojected into ERP at every refinement step via learned view weights and deformable cross-attention with radial distortion bias, and (ii) a multi-view consensus volume that concatenates group-wise correlation with cross-camera feature variance and is regularized by a 3D U-Net. A ConvGRU then refines inverse depth under a standard sequence loss. A MobileNet student is distilled from the dual-stream teachers for faster inference. On the synthetic OmniThings / OmniHouse / Sunny (and weather-variant) benchmarks the method reports the lowest inverse-index error metrics under both OmniThings-only and fine-tuned regimes, with measured latencies of 148 ms (full) and 102 ms (distilled) on an RTX 4070 SUPER.
Significance. If the reported gains hold, the paper supplies a concrete architectural answer to a well-motivated failure mode of multi-fisheye stereo: multi-modal ERP evidence caused by wide-baseline visibility conflicts and self-occlusion. The combination of dynamic dual-stream context fusion with an explicit multi-view consensus volume is a clear advance over prior down-weighting or center-view heuristics, and the distillation path makes the design practically usable. The ablations (Tables 3–4) isolate the contribution of each component, and the quantitative tables show consistent improvements across five datasets and two training regimes. The work is therefore a solid engineering contribution to omnidirectional multi-view stereo, even though its evaluation remains confined to synthetic, perfectly calibrated data.
major comments (2)
- §4.1 and Tables 1–2: every quantitative number, ablation and qualitative example is obtained on synthetic OmniThings/OmniHouse/Sunny data with a fixed, perfectly calibrated four-camera 220° rig. The central claim that dynamic fusion + MVC “resolves” visibility conflicts (Fig. 1, §1) therefore rests on an untested transfer assumption that the authors themselves list as a limitation in §5. Without at least one real-image experiment (or a controlled calibration-noise study) it is impossible to know whether the learned view weights (Eq. 1), distortion-biased deformable attention (Eqs. 3–4) or the feature-variance term inside the MVC remain effective under photometric non-idealities and residual calibration error. This is the single load-bearing gap for any claim that goes beyond “SOTA on these synthetic benchmarks.”
- §4.3, latency paragraph: the inference time of the strongest competing method (MDP-Omni) is not measured but “derived \ldots proportionally” from ratios reported in that paper. Because the SOTA claim is accompanied by a competitive-speed claim, the comparison should be re-run under identical hardware and software, or the estimated numbers should be clearly flagged as non-comparable.
minor comments (5)
- Eq. (4) defines the distortion bias b(r_i,k) but never states how it is added to the attention logits; a one-line clarification would remove ambiguity.
- Table 3 row (e) shows that adding MVC alone can raise RMS; a short discussion of when the consensus volume helps versus hurts would strengthen the ablation narrative.
- The abstract and introduction repeatedly claim that prior methods “do not resolve the underlying discrepancy”; a more precise statement that they only down-weight unreliable views would avoid over-claiming.
- Fig. 3 caption should state the exact inverse-depth visualization range so that error maps can be compared across methods.
- DINOv3 is cited as arXiv:2508.10104; confirm that the frozen weights used in the experiments match a publicly released checkpoint so that the dual-stream results are reproducible.
Circularity Check
No circularity: empirical SOTA claims rest on external synthetic benchmarks and standard supervised losses, not self-defined quantities.
full rationale
OmniDS is a standard engineering/architecture paper. The central claim is lower inverse-index error (>1/>3/>5, MAE, RMS) than prior methods on the held-out OmniThings/OmniHouse/Sunny (and weather) test sets (Tables 1–2). Those metrics are computed against external ground-truth depth maps supplied by the benchmarks; they are not algebraic rearrangements of any fitted parameter. The dual-stream encoder, learned view weights (Eq. 1), distortion-biased deformable cross-attention (Eqs. 3–4), multi-view consensus volume (group-wise correlation + feature variance + 3D U-Net), ConvGRU residual updates, sequence loss (Eq. 8 with fixed γ=0.9), and distillation MSE (Eq. 9 with fixed λ) are ordinary trainable components and hyper-parameters. Ablations (Tables 3–4) isolate each module by training under identical protocols and measuring the same external metrics. Self-citations (e.g., OmniMVS by overlapping authors) appear only as baselines or related work, never as uniqueness theorems that force the architecture. No step reduces a claimed “prediction” or “first-principles result” to its own inputs by construction. The acknowledged limitation (synthetic fixed-rig data) is a transfer/generalization concern, not circularity.
Axiom & Free-Parameter Ledger
free parameters (5)
- sequence-loss discount γ =
0.9
- distillation loss weights λ_cnn, λ_dino =
1.0 / 10.0
- number of inverse-depth bins (96 inside loop, 192 final) =
96 / 192
- deformable-attention heads and sampling points =
4×4
- training epochs and learning-rate schedule =
30+15+15 / 5e-4
axioms (4)
- domain assumption Camera models and extrinsics of the four fisheye views are known exactly.
- domain assumption Equirectangular projection is an adequate common coordinate system for multi-view fusion.
- domain assumption Frozen DINOv3 features supply useful semantic priors under severe fisheye distortion.
- ad hoc to paper Synthetic OmniThings/OmniHouse/Sunny distributions are representative of the target real-world deployment domain.
invented entities (2)
-
Multi-view consensus volume (group-wise correlation + cross-camera feature variance + 3D U-Net)
no independent evidence
-
Dynamic dual-stream ERP context fusion (learned view weights + deformable cross-attention with radial distortion bias)
no independent evidence
read the original abstract
Omnidirectional depth estimation from multi-fisheye camera rigs is complicated by visibility conflicts: wide baselines cause different cameras to observe different portions, or even different faces, of the same object, so aggregating their features into a unified equirectangular (ERP) representation under fixed projection produces ambiguous matching evidence near occlusion boundaries and thin structures. Although existing methods mitigate this by down-weighting unreliable views, they do not resolve the underlying discrepancy because context formation and cross-view fusion remain tied to rigid fisheye-to-ERP sampling. We present OmniDS, an iterative depth refinement framework that replaces rigid aggregation by combining dynamic context fusion with consensus-aware multi-view similarity. A dual-stream encoder pairs a lightweight CNN for geometric detail with a frozen DINOv3 for semantic priors; their features are reprojected into ERP space at each refinement step via learned view weighting and deformable cross-attention with geometric distortion bias. In parallel, a multi-view consensus volume captures global cross-camera agreement through group-wise correlation and feature variance, regularized by a 3D U-Net. For efficient deployment, we distill the dual-stream representation into a single MobileNet-based encoder. OmniDS achieves state-of-the-art performance on the OmniThings, OmniHouse, and Sunny benchmarks while maintaining competitive inference speed. Project page and codes are available at https://parkchaesong.github.io/omnids.
Figures
Reference graph
Works this paper leans on
-
[1]
In: European Conference on Computer Vision (ECCV) (2022),https://arxiv.org/abs/2204.009934
Bai, J., Yuan, L., Xia, S.T., Yan, S., Li, Z., Liu, W.: Improving vision transformers by revisiting high-frequency components. In: European Conference on Computer Vision (ECCV) (2022),https://arxiv.org/abs/2204.009934
Pith/arXiv arXiv 2022
-
[2]
In: International Conference on Computer Vision (ICCV) (2021),https://arxiv.org/abs/2104
Caron, M., Touvron, H., Misra, I., Jegou, H., Mairal, J., Bojanowski, P., Joulin, A.: Emerging properties in self-supervised vision transformers. In: International Conference on Computer Vision (ICCV) (2021),https://arxiv.org/abs/2104. 142945
2021
-
[3]
In: Proceedings of the 31st ACM International Conference on Multimedia
Chen, Z., Lin, C., Nie, L., Shen, Z., Liao, K., Cao, Y., Zhao, Y.: S-omnimvs: Incor- porating sphere geometry into omnidirectional stereo matching. In: Proceedings of the 31st ACM International Conference on Multimedia. pp. 1495–1503 (2023) 11, 12
2023
-
[4]
In: Proceedings of the Computer Vision and Pattern Recognition Conference
Deng,J.,Wang,Y.,Meng,H.,Hou,Z.,Chang,Y.,Chen,G.:Omnistereo:Real-time omnidireactional depth estimation with multiview fisheye cameras. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 1003–1012 (2025) 3, 4, 11, 12 16 C. Park et al
2025
-
[5]
In: CVPR (2024),https://openaccess.thecvf.com/ content/CVPR2024/papers/Edstedt_RoMa_Robust_Dense_Feature_Matching_ CVPR_2024_paper.pdf5
Edstedt, J., Sun, Q., Bokman, G., Wadenbäck, M., Felsberg, M.: Roma: Robust dense feature matching. In: CVPR (2024),https://openaccess.thecvf.com/ content/CVPR2024/papers/Edstedt_RoMa_Robust_Dense_Feature_Matching_ CVPR_2024_paper.pdf5
2024
-
[6]
In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition
Guo, X., Yang, K., Yang, W., Wang, X., Li, H.: Group-wise correlation stereo network. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 3273–3282 (2019) 5, 9
2019
-
[7]
In: 2022 International Conference on Robotics and Automation (ICRA)
Huang, H., Yeung, S.K.: 360vo: Visual odometry using a single 360 camera. In: 2022 International Conference on Robotics and Automation (ICRA). pp. 5594– 5600 (2022).https://doi.org/10.1109/ICRA46639.2022.98122031
-
[8]
IEEE Robotics and Automation Letters9(3), 2511–2518 (2024) 2, 3, 4, 8, 10, 13
Jiang, H., Xu, R., Tan, M., Jiang, W.: Romnistereo: Recurrent omnidirectional stereo matching. IEEE Robotics and Automation Letters9(3), 2511–2518 (2024) 2, 3, 4, 8, 10, 13
2024
-
[9]
In: CVPR (2025),https : / / openaccess
Jung, D., Choi, J., Lee, Y., Jeong, S., Lee, T., Manocha, D., Yeon, S.: Edm: Equirectangular projection-oriented dense kernelized feature matching. In: CVPR (2025),https : / / openaccess . thecvf . com / content / CVPR2025 / papers/Jung_EDM_Equirectangular_Projection-Oriented_Dense_Kernelized_ Feature_Matching_CVPR_2025_paper.pdf5
2025
-
[10]
In: Proceedings of the European Conference on Computer Vision (ECCV)
de La Garanderie, G.P., Abarghouei, A.A., Breckon, T.P.: Eliminating the blind spot: Adapting 3d object detection and monocular depth estimation to 360 panoramic imagery. In: Proceedings of the European Conference on Computer Vision (ECCV). pp. 789–807 (2018) 1
2018
-
[11]
Li, M., Jin, X., Hu, X., Dai, J., Du, S., Li, Y.: Mode: Multi-view omnidirectional depth estimation with 360◦ cameras. In: Computer Vision – ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXXIII. p. 197–213. Springer-Verlag, Berlin, Heidelberg (2022).https://doi. org/10.1007/978-3-031-19827-4_12,https://doi.o...
-
[12]
IEEE Transactions on Pattern Analysis and Machine Intelligence45(3), 3292–3310 (2022) 2
Liao, Y., Xie, J., Geiger, A.: Kitti-360: A novel dataset and benchmarks for urban scene understanding in 2d and 3d. IEEE Transactions on Pattern Analysis and Machine Intelligence45(3), 3292–3310 (2022) 2
2022
-
[13]
In: 2021 International conference on 3D vision (3DV)
Lipson, L., Teed, Z., Deng, J.: Raft-stereo: Multilevel recurrent field transforms for stereo matching. In: 2021 International conference on 3D vision (3DV). pp. 218–227. IEEE (2021) 5, 9
2021
-
[14]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Meuleman, A., Jang, H., Jeon, D.S., Kim, M.H.: Real-time sphere sweeping stereo from multiview fisheye images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 11423–11432 (2021) 4
2021
-
[15]
Naseer, M., Ranasinghe, K., Khan, S., Hayat, M., Khan, F.S., Yang, M.H.: Intrigu- ing properties of vision transformers. In: Advances in Neural Information Process- ing Systems (NeurIPS) (2021),https://arxiv.org/abs/2105.104975
Pith/arXiv arXiv 2021
-
[16]
Park, N., Kim, S.: How do vision transformers work? In: International Conference on Learning Representations (ICLR) (2022),https://arxiv.org/abs/2202.06709 4
Pith/arXiv arXiv 2022
-
[17]
arXiv preprint arXiv:2508.10104 (2025) 3
Siméoni, O., Vo, H.V., Seitzer, M., Baldassarre, F., Oquab, M., Jose, C., Khali- dov, V., Szafraniec, M., Yi, S., Ramamonjisoa, M., et al.: Dinov3. arXiv preprint arXiv:2508.10104 (2025) 3
Pith/arXiv arXiv 2025
-
[18]
In: Proceedings of the IEEE/CVF International Conference on Computer Vision
Son, E., Jo, H., Kwon, W., Lee, S.J.: Mdp-omni: Parameter-free multimodal depth prior-based sampling for omnidirectional stereo matching. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 26178–26187 (2025) 3, 4, 10, 11, 12 OmniDS 17
2025
-
[19]
In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition
Sun, J., Shen, Z., Wang, Y., Bao, H., Zhou, X.: Loftr: Detector-free local fea- ture matching with transformers. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 8922–8931 (2021) 5
2021
-
[20]
In: European conference on computer vision
Teed, Z., Deng, J.: Raft: Recurrent all-pairs field transforms for optical flow. In: European conference on computer vision. pp. 402–419. Springer (2020) 2, 5
2020
-
[21]
Applied Sciences14(2) (2024), https://www.mdpi.com/2076-3417/14/2/5172
Wang, P., Li, M., Cao, J., Du, S., Li, Y.: Casomnimvs: Cascade omnidirectional depth estimation with dynamic spherical sweeping. Applied Sciences14(2) (2024), https://www.mdpi.com/2076-3417/14/2/5172
2024
-
[22]
In: Proceedings of the Asian conference on computer vision
Wang, Q., Zhang, J., Yang, K., Peng, K., Stiefelhagen, R.: Matchformer: Interleav- ing attention in transformers for feature matching. In: Proceedings of the Asian conference on computer vision. pp. 2746–2762 (2022) 5
2022
-
[23]
In: CVPR (2025),https://openaccess
Wen, B., Trepte, M., Aribido, J., Kautz, J., Gallo, O., Birchfield, S.: Founda- tionstereo: Zero-shot stereo matching. In: CVPR (2025),https://openaccess. thecvf . com / content / CVPR2025 / papers / Wen _ FoundationStereo _ Zero - Shot _ Stereo_Matching_CVPR_2025_paper.pdf5
2025
-
[24]
In: Proceedings of the IEEE/CVF International Conference on Com- puter Vision
Won,C.,Ryu,J.,Lim,J.:Omnimvs:End-to-endlearningforomnidirectionalstereo matching. In: Proceedings of the IEEE/CVF International Conference on Com- puter Vision. pp. 8987–8996 (2019) 2, 4, 10, 11, 12, 13
2019
-
[25]
In: 2019 International Conference on Robotics and Automation (ICRA)
Won, C., Ryu, J., Lim, J.: Sweepnet: Wide-baseline omnidirectional depth estima- tion. In: 2019 International Conference on Robotics and Automation (ICRA). pp. 6073–6079. IEEE (2019) 4, 10
2019
-
[26]
IEEE transactions on pattern analysis and machine intelligence43(11), 3850–3862 (2020) 11, 12
Won, C., Ryu, J., Lim, J.: End-to-end learning for omnidirectional stereo match- ing with uncertainty prior. IEEE transactions on pattern analysis and machine intelligence43(11), 3850–3862 (2020) 11, 12
2020
-
[27]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Xie, S., Wang, D., Liu, Y.H.: Omnividar: Omnidirectional depth estimation from multi-fisheye images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 21529–21538 (2023) 3, 4
2023
-
[28]
In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition
Xu, G., Cheng, J., Guo, P., Yang, X.: Attention concatenation volume for accurate and efficient stereo matching. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 12981–12990 (2022) 5, 9
2022
-
[29]
In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition
Xu, G., Wang, X., Ding, X., Yang, X.: Iterative geometry encoding volume for stereo matching. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 21919–21928 (2023) 5
2023
-
[30]
In: Proceedings of the IEEE/CVF international conference on computer vision
Yogamani, S., Hughes, C., Horgan, J., Sistu, G., Varley, P., O’Dea, D., Uricár, M., Milz, S., Simon, M., Amende, K., et al.: Woodscape: A multi-task, multi- camera fisheye dataset for autonomous driving. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 9308–9318 (2019) 2
2019
-
[31]
In: Proceedings of the Computer Vision and Pattern Recognition Conference
Zayene,M.,Endres,J.,Havolli,A.,Corbière,C.,Cherkaoui,S.,Kontouli,A.,Alahi, A.: Helvipad: A real-world dataset for omnidirectional stereo depth estimation. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 26975–26984 (2025) 2
2025
-
[32]
Zhang, J., Herrmann, C., Hur, J., Polania Cabrera, L., Jampani, V., Sun, D., Yang, M.H.: A tale of two features: Stable diffusion complements dino for zero- shotsemanticcorrespondence.In:NeurIPS(2023),https://arxiv.org/abs/2305. 153475
2023
-
[33]
arXiv preprint arXiv:2010.04159 (2020) 8
Zhu, X., Su, W., Lu, L., Li, B., Wang, X., Dai, J.: Deformable detr: Deformable transformers for end-to-end object detection. arXiv preprint arXiv:2010.04159 (2020) 8
Pith/arXiv arXiv 2010
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.