Pith. sign in

REVIEW 2 major objections 5 minor 53 references

Pinhole bias in positional encodings is why 3D foundation models fail on fisheye images, and a tiny residual adapter can fix it online.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-12 07:33 UTC pith:ABG3KQF7

load-bearing objection Clean, targeted PE residual adapter that actually makes frozen 3D foundation models usable on real fisheye without multi-view cost. the 2 major comments →

arxiv 2607.02711 v1 pith:ABG3KQF7 submitted 2026-07-02 cs.CV

RayTun3R: Online Camera Adaptation in 3D Foundation Models

Daniil Sinitsyn , Nikita Araslanov , Daniel Cremers This is my paper
classification cs.CV
keywords 3D foundation modelsfisheye cameraspositional encodingsonline adaptationcamera geometrydepth and pose estimationparameter-efficient fine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Modern 3D foundation models recover depth and pose from a few images, but they collapse when the camera is a fisheye rather than a pinhole. The paper argues that a large part of this failure lives in the positional encodings: those encodings were learned under a constant local geometry that only a pinhole camera has, so the same image-grid step no longer means the same change of viewing direction under fisheye optics. RayTun3R freezes the backbone and corrects only the camera-dependent pieces—small residual lookup tables on absolute and rotary positional encodings, plus parameter-free fixes to patch tokenization and prediction-grid coordinates. The adapter has about eleven thousand parameters, is fit on a short self-supervised temporal segment, and then runs at the original single-pass cost on the rest of the sequence. Across outdoor, handheld, and indoor fisheye data from 110° to 200° field of view, and across several frozen backbones, it cuts rotation error by factors of two to twelve relative to the unadapted model, beats generic low-rank fine-tuning with far fewer parameters, and improves pose over multi-view cropping baselines without their extra inference cost while staying competitive on depth.

Core claim

The local Jacobian of pretrained positional embeddings is nearly radius-independent and therefore encodes a pinhole spatial prior; once that prior is corrected by lightweight residual PE/RoPE adapters (plus parameter-free tokenization and grid fixes), frozen 3D foundation models recover coherent fisheye depth and pose without retraining the backbone.

What carries the argument

RayTun3R: zero-initialized residual radial/angular lookup-table corrections to absolute and rotary positional encodings, trained online with geometric losses on a short multi-frame segment while the foundation model stays frozen.

Load-bearing premise

The method assumes that external matches and a once-computed relative pose from a short, non-static clip give clean enough geometric supervision to isolate the camera bias rather than scene-specific or matcher-specific errors.

What would settle it

On a high-FOV fisheye sequence with accurate ground-truth poses, replace the external matcher with perfect correspondences (or deliberately corrupt them) and check whether the PE residual still recovers the claimed rotation and translation gains; if the gains vanish under clean matches or survive under strongly biased ones, the supervision claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper diagnoses a pinhole bias in the positional encodings of recent 3D foundation models (DA3, VGGT, π³) via the local Jacobian of absolute PE (Fig. 2, Sec. 3), and proposes RayTun3R: a frozen-backbone adapter that learns small residual radial/angular lookup tables for absolute PE and RoPE (Eqs. 5–6), plus parameter-free patch undistortion, border-token handling, and DPT-grid coordinate correction. The adapter (10,752 parameters on DA3-Small) is fit online on ~30 three-frame windows with geometric losses that use fixed external UFM matches and a once-computed MAGSAC++ pose target (Sec. 4.3, Eqs. 8–13). Across five fisheye datasets (110°–200° FOV) and multiple backbones, it reduces rotation error by roughly 2–12× vs. the unadapted model, outperforms LoRA and CalTok with far fewer parameters, improves pose over Center-PH/Multi-PH without multi-view cost, and remains competitive on depth (Tables 1–3, 6).

Significance. If the results hold, the work offers a practical, low-cost route to reuse pinhole-trained 3D foundation models on fisheye imagery without retraining or multi-view projection. Strengths include a clear geometric diagnosis (PE Jacobian), extreme parameter efficiency, multi-backbone multi-dataset evaluation, component ablations isolating learned PE residuals (Table 4a, Sec. G), and sensitivity checks under predicted calibration (Table 5). The approach is falsifiable and immediately useful for robotics and mapping pipelines that already deploy these models. Code release is promised, which would further strengthen reproducibility.

major comments (2)
  1. The supervision pipeline in Sec. 4.3 (UFM matches + once-computed MAGSAC++ pose target, Eqs. 8–9) is the load-bearing soft spot. Gains are large and consistent, but the paper does not quantify how much residual matcher bias under strong fisheye distortion remains after the fixed-target design. A short controlled experiment—e.g., synthetic fisheye with known ground-truth matches, or deliberate corruption of UFM confidence—would make the isolation of PE bias more airtight. This does not overturn the empirical pattern, but it is the main correctness-risk concern for the central claim.
  2. Evaluation protocol (Sec. 5): adaptation uses 30 filtered windows (optical-flow ≥2 px) from the same sequence that is later evaluated. While this matches the intended online setting, there is no multi-seed variance, no held-out sequence transfer within a camera, and no error bars. For a journal claim of 2–12× rotation reduction, at least seed-level variance or a leave-one-sequence-out check on one multi-sequence dataset (e.g., KITTI-360 or TUM-VI) would strengthen confidence that the adapter is not over-fitting sequence-specific appearance.
minor comments (5)
  1. Table 4a: the full RayTun3R row is not uniformly best on every metric (some ablations have slightly lower R° or t°). A one-sentence clarification that the full model is selected for lowest dreproj / overall balance would avoid reader confusion.
  2. Fig. 2 caption and panels: the switch from “pretrained σ1 / det” to “adapted” is clear, but the analytical fisheye reference curves are not overlaid on the same axes; adding them would make the visual match more immediate.
  3. Notation: κ and κ⁻¹ are introduced in Sec. 3, but the precise fisheye model (KB vs. EUCM) used per dataset is only mentioned in passing; a short table or appendix note would help reproducibility.
  4. Related work: contemporaneous Fisheye3R and FishRoPE are cited; a slightly sharper sentence on the frozen-backbone / self-supervised distinction would further clarify novelty.
  5. Typos / polish: “π3” vs. “π³” inconsistency; “drive 0000, cam02” style labels in Table 4a could be standardized with the main tables.

Circularity Check

0 steps flagged

No significant circularity: empirical PE residual adaptation trained on external geometric pseudo-labels and scored against independent ground-truth pose/depth.

full rationale

RayTun3R is an empirical methods paper, not a first-principles derivation. The PE-Jacobian diagnosis (Fig. 2, Sec. 3) is an observational measurement of pretrained vs. adapted embeddings; it is not used as a tautological proof that the adapter must work. The adapter is fit with geometric losses that depend on fixed external UFM correspondences and a once-computed MAGSAC++ pose target (Sec. 4.3, Eqs. 8–9), so the model cannot rewrite its own pseudo-labels. Reported metrics (R°, t°, dreproj, AbsRel, δ1.25) are computed from dataset ground-truth poses/depth after adaptation on a short filtered segment, and are compared against independent baselines (LoRA, CalTok, Center-PH, Multi-PH) across multiple frozen backbones and FOVs. Self-citations (e.g. PRaDA, prior alignment work) are peripheral and not load-bearing for the central claim. Capacity is restricted to PE/RoPE residuals, so sequence-specific absorption is possible in principle but does not make the reported gains equal to the training inputs by construction. No self-definitional loop, fitted-input-as-prediction, uniqueness import, or renamed known result appears in the derivation chain.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 1 invented entities

The central empirical claim rests on standard camera geometry, the frozen pretrained backbones, an external dense matcher for self-supervision, and a handful of hand-chosen adapter hyperparameters. No new physical entities are postulated; the invented object is the adapter itself. Free parameters are the usual PEFT/test-time knobs rather than quantities fitted to hide the result.

free parameters (5)
  • PE radial/angular bins (Nr, Nθ) = Nr=20, Nθ=8
    Default Nr=20, Nθ=8 chosen by ablation; controls capacity of residual tables and is not derived from first principles.
  • RoPE radial bins = 20
    20 radial bins for rotary angle correction when the backbone uses RoPE.
  • Loss weights (w_pose, w_smooth, w_L2, w_TV) = 1 / 10 / 2 / 20
    Fixed to 1, 10, 2, 20 for all experiments; ablated but still free design choices that affect the reported optimum.
  • Adaptation set size and motion filter = 30 windows, ≥2 px
    30 three-frame windows with average optical-flow displacement ≥2 px; defines the online training distribution.
  • Adam learning rate and gradient clip = 1e-3, clip 1.0
    1e-3 with clip norm 1.0; standard but free.
axioms (5)
  • domain assumption Central camera model with known or estimated κ (principal point + mostly radial distortion) correctly maps pixels to rays.
    Invoked throughout Sec. 3–4 and Sec. B; AnyCalib is shown to be usable but the method still requires a camera model.
  • domain assumption Local Jacobian of pretrained absolute PE is approximately radius-independent, matching pinhole geometry.
    Empirical measurement in Fig. 2 on DA3; motivates targeting PE rather than generic features.
  • domain assumption UFM dense matches and MAGSAC++ relative poses provide usable geometric supervision on fisheye imagery.
    Sec. 4.3; the adapter never sees ground-truth depth/pose during training.
  • domain assumption Sufficient inter-frame displacement exists so that reprojection and pose losses constrain depth and relative motion.
    Stated as limitation (v) in Sec. 6; near-static windows are filtered.
  • standard math Standard ViT absolute PE and axial RoPE constructions as used by the frozen backbones.
    Sec. 3 preliminaries; residual corrections are added to these known mechanisms.
invented entities (1)
  • RayTun3R residual PE/RoPE lookup tables (radial + angular) independent evidence
    purpose: Correct pinhole-biased positional encodings for fisheye ray geometry while freezing the backbone.
    The only learned parameters; initialized at zero and fit online. Independent evidence is the multi-dataset pose improvement and the post-adaptation PE Jacobian bending toward the analytic fisheye curve (Fig. 2).

pith-pipeline@v1.1.0-grok45 · 23084 in / 3332 out tokens · 34418 ms · 2026-07-12T07:33:38.106123+00:00 · methodology

0 comments
read the original abstract

Recent 3D foundation models, such as DUSt3R, MASt3R, VGGT, $\pi^3$, and Depth Anything 3, provide strong feed-forward depth and pose estimates on pinhole imagery, but degrade sharply under fisheye camera geometry. We show that this failure is partly caused by a pinhole camera bias in the positional encodings of pretrained 3D foundation models, and propose RayTun3R, a lightweight camera adaptation approach. It keeps the pretrained network fixed and adapts only lightweight components tied to token position and camera geometry. RayTun3R learns parameter-efficient residual corrections to absolute and rotary positional encodings, together with parameter-free tokenization and corrections to prediction-grid coordinates that remove residual pinhole assumptions. The resulting adapter contains only 10,752 trainable parameters and can be learned from a short temporal segment using geometric losses. Once adapted, RayTun3R transfers effectively to the remaining frames of the sequence without incurring additional runtime costs. Across diverse fisheye datasets with fields of view from $110^\circ$ to $200^\circ$, our adapter reduces rotation error by $2$-$12\times$ relative to the unadapted model, outperforms LoRA while using $\sim\!14\times$ fewer trainable parameters, improves pose over adaptation-free baselines while avoiding their multi-view inference cost, and remains competitive on depth accuracy.

Figures

Figures reproduced from arXiv: 2607.02711 by Daniel Cremers, Daniil Sinitsyn, Nikita Araslanov.

Figure 1
Figure 1. Figure 1: RayTun3R overview. The figure shows fisheye input, the original and adapted positional encodings, the frozen 3D foundation model, and the resulting geometry before and after adaptation. RayTun3R keeps the backbone frozen and adapts lightweight components tied to token location and camera geometry to recover more coherent depth, pose, and 3D structure. prediction-grid coordinates and patch-tokenization corr… view at source ↗
Figure 2
Figure 2. Figure 2: Local Jacobian of the positional encoding in Depth Anything 3. Pretrained positional embeddings have nearly radius-independent local geometry, consistent with a pinhole-like spatial prior (a, c). After adaptation on KITTI-360, the PE Jacobian becomes radius-dependent, matching the fisheye camera geometry more closely (b, d). where g is a non-linear function of image radius (e.g. the pinhole model is a spec… view at source ↗
Figure 3
Figure 3. Figure 3: Adaptation-free baselines. posed DSLR fisheye scans (ScanNet++ [50], 115◦ ), multi-camera captures (ETH3D [51], 110◦ ), and dual￾fisheye indoor/outdoor scenes (FIORD [52], 200◦ ). All datasets provide reference poses; ETH3D and Scan￾Net++ also provide dense depth. Baselines. We compare against representative base￾lines from the two most relevant strategies for reusing pinhole-trained models on fisheye inpu… view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative reconstructions. Across datasets, the frozen model produces distorted or inconsistent geometry, while RayTun3R recovers more coherent structure after online adaptation. 110◦ to avoid severe center compression and peripheral stretching from wider rectilinear projections. Multi-PH fuses predictions from Center-PH and four additional virtual pinhole views back into the fisheye frame (Fig. 3b), imp… view at source ↗
Figure 5
Figure 5. Figure 5: Naive undistortion-based PE reuse. (a) Original DA3-small PE table, shown as ℓ2 distance to the center embedding. (b) Locations reached by undistorted and rescaled fisheye coordinates. (c) Resulting PE lookup. Many fisheye patches collapse onto a small central region of the table, producing nearly identical embeddings and losing positional resolution [PITH_FULL_IMAGE:figures/full_fig_p014_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

53 extracted references · 6 linked inside Pith

  1. [1]

    DUSt3R: Geometric 3D vision made easy

    Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. DUSt3R: Geometric 3D vision made easy. InCVPR, pages 20697–20709, 2024

    2024

  2. [2]

    MASt3R-SfM: A fully integrated solution for unconstrained structure- from-motion

    Bardienus Pieter Duisterhof, Lojze Zust, Philippe Weinzaepfel, Vincent Leroy, Yohann Cabon, and Jerome Revaud. MASt3R-SfM: A fully integrated solution for unconstrained structure- from-motion. In3DV, 2025

    2025

  3. [3]

    VGGT: Visual geometry grounded transformer

    Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. VGGT: Visual geometry grounded transformer. InCVPR, pages 5294–5306, 2025

    2025

  4. [4]

    π3: Permutation-equivariant visual geometry learning

    Yifan Wang, Jianjun Zhou, Haoyi Zhu, Wenzheng Chang, Yang Zhou, Zizun Li, Junyi Chen, Jiangmiao Pang, Chunhua Shen, and Tong He. π3: Permutation-equivariant visual geometry learning. InICLR, 2026

    2026

  5. [5]

    Chen, Zhenyu Li, Yang Zhao, Sida Peng, Hengkai Guo, Xiaowei Zhou, Guang Shi, Jiashi Feng, and Bingyi Kang

    Haotong Lin, Sili Chen, Junhao Liew, Donny Y . Chen, Zhenyu Li, Yang Zhao, Sida Peng, Hengkai Guo, Xiaowei Zhou, Guang Shi, Jiashi Feng, and Bingyi Kang. Depth Anything 3: Recovering the visual space from any views. InICLR, 2026

    2026

  6. [6]

    CubemapSLAM: A piecewise-pinhole monocular fisheye SLAM system

    Yahui Wang, Shaojun Cai, Shi-Jie Li, Yun Liu, Yangyan Guo, Tao Li, and Ming-Ming Cheng. CubemapSLAM: A piecewise-pinhole monocular fisheye SLAM system. InACCV, pages 34–49, 2018

    2018

  7. [7]

    SDGE: Stereo guided depth estimation for 360 camera sets

    Jialei Xu, Wei Yin, Dong Gong, Junjun Jiang, and Xianming Liu. SDGE: Stereo guided depth estimation for 360 camera sets. InIROS, pages 11179–11186, 2024

    2024

  8. [8]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Liang Wang, Weizhu Chen, et al

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Liang Wang, Weizhu Chen, et al. LoRA: Low-rank adaptation of large language models. InICLR, 2022

    2022

  9. [9]

    Parameter-efficient transfer learning for NLP

    Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for NLP. InICML, pages 2790–2799, 2019

    2019

  10. [10]

    Scaling and shifting your features: A new baseline for efficient model tuning

    Dongze Lian, Daquan Zhou, Jiashi Feng, and Xinchao Wang. Scaling and shifting your features: A new baseline for efficient model tuning. InNeurIPS, 2022

    2022

  11. [11]

    Extending foundational monocular depth estimators to fisheye cameras with calibration tokens

    Suchisrit Gangopadhyay, Jung-Hee Kim, Xien Chen, Patrick Rim, Hyoungseob Park, and Alex Wong. Extending foundational monocular depth estimators to fisheye cameras with calibration tokens. InICCV, pages 5198–5209, 2025

    2025

  12. [12]

    Structure-from-motion revisited

    Johannes Lutz Schönberger and Jan-Michael Frahm. Structure-from-motion revisited. InCVPR, pages 4104–4113, 2016

    2016

  13. [13]

    Back to the feature: Learning robust camera localization from pixels to pose

    Paul-Edouard Sarlin, Ajaykumar Unagar, Mans Larsson, Hugo Germain, Carl Toft, Viktor Larsson, Marc Pollefeys, Vincent Lepetit, Lars Hammarstrand, Fredrik Kahl, et al. Back to the feature: Learning robust camera localization from pixels to pose. InCVPR, pages 3247–3257, 2021

    2021

  14. [14]

    An analytical solution to gauss-newton loss for direct image alignment

    Sergei Solonets, Daniil Sinitsyn, Lukas V on Stumberg, Nikita Araslanov, and Daniel Cremers. An analytical solution to gauss-newton loss for direct image alignment. InICLR, 2024

    2024

  15. [15]

    Monocular depth estimation with self-supervised instance adaptation.arXiv:2004.05821 [cs.CV], 2020

    Robert McCraith, Lukas Neumann, Andrew Zisserman, and Andrea Vedaldi. Monocular depth estimation with self-supervised instance adaptation.arXiv:2004.05821 [cs.CV], 2020

    Pith/arXiv arXiv 2004

  16. [16]

    OmniViDAR: Omnidirectional depth estimation from multi-fisheye images

    Sheng Xie, Daochuan Wang, and Yun-Hui Liu. OmniViDAR: Omnidirectional depth estimation from multi-fisheye images. InCVPR, pages 21529–21538, 2023

    2023

  17. [17]

    OmniStereo: Real-time omnidirectional depth estimation with multi-view fisheye cameras

    Jiaxi Deng, Yushen Wang, Haitao Meng, Zuoxun Hou, Yi Chang, and Gang Chen. OmniStereo: Real-time omnidirectional depth estimation with multi-view fisheye cameras. InCVPR, pages 1003–1012, 2025. 10

    2025

  18. [18]

    Yuliang Guo, Sparsh Garg, S. Mahdi H. Miangoleh, Xinyu Huang, and Liu Ren. Depth any camera: Zero-shot metric depth estimation from any camera. InCVPR, pages 26996–27006, 2025

    2025

  19. [19]

    FisheyeDepth: A real-scale self-supervised depth estimation model for fisheye camera

    Guoyang Zhao, Yuxuan Liu, Weiqing Qi, Fulong Ma, Ming Liu, and Jun Ma. FisheyeDepth: A real-scale self-supervised depth estimation model for fisheye camera. InICRA, pages 3780–3787, 2025

    2025

  20. [20]

    UniDepth: Universal monocular metric depth estimation

    Luigi Piccinelli, Yung-Hsu Yang, Christos Sakaridis, Mattia Segu, Siyuan Li, Luc Van Gool, and Fisher Yu. UniDepth: Universal monocular metric depth estimation. InCVPR, pages 10106–10116, 2024

    2024

  21. [21]

    UniK3D: Universal camera monocular 3D estimation

    Luigi Piccinelli, Christos Sakaridis, Mattia Segu, Yung-Hsu Yang, Siyuan Li, Wim Abbeloos, and Luc Van Gool. UniK3D: Universal camera monocular 3D estimation. InCVPR, pages 1028–1039, 2025

    2025

  22. [22]

    Mu Hu, Wei Yin, Chi Zhang, Zhipeng Cai, Xiaoxiao Long, Hao Chen, Kaixuan Wang, Gang Yu, Chunhua Shen, and Shaojie Shen. Metric3D v2: A versatile monocular geometric foundation model for zero-shot metric depth and surface normal estimation.IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12):10579–10596, 2024

    2024

  23. [23]

    AnyCalib: On-manifold learning for model-agnostic single-view camera calibration

    Javier Tirado-Garín and Javier Civera. AnyCalib: On-manifold learning for model-agnostic single-view camera calibration. InICCV, pages 8044–8055, 2025

    2025

  24. [24]

    PRaDA: Projective radial distortion averaging

    Daniil Sinitsyn, Linus Härenstam-Nielsen, and Daniel Cremers. PRaDA: Projective radial distortion averaging. InCVPR, pages 21902–21912, 2025

    2025

  25. [25]

    DarSwin: Distortion-aware radial swin transformer

    Akshaya Athwale, Arman Afrasiyabi, Justin Lagüe, Ichrak Shili, Ola Ahmad, and Jean-François Lalonde. DarSwin: Distortion-aware radial swin transformer. InICCV, pages 5929–5938, 2023

    2023

  26. [26]

    DarSwin-UNet: Distortion-aware architecture

    Akshaya Athwale, Ichrak Shili, Émile Bergeron, Ola Ahmad, and Jean-François Lalonde. DarSwin-UNet: Distortion-aware architecture. InWACV, pages 8670–8680, 2025

    2025

  27. [27]

    Sector patch embedding: An embedding module conforming to the distortion pattern of fisheye image.arXiv:2303.14645 [cs.CV], 2023

    Dianyi Yang, Jiadong Tang, Yu Gao, Yi Yang, and Mengyin Fu. Sector patch embedding: An embedding module conforming to the distortion pattern of fisheye image.arXiv:2303.14645 [cs.CV], 2023

    Pith/arXiv arXiv 2023

  28. [28]

    Cameras as relative positional encoding

    Ruilong Li, Brent Yi, Junchen Liu, Hang Gao, Yi Ma, and Angjoo Kanazawa. Cameras as relative positional encoding. InNeurIPS, 2025

    2025

  29. [29]

    Fisheye3R: Adapting unified 3D feed-forward foundation models to fisheye lenses.arXiv:2603.28896 [cs.CV], 2026

    Ruxiao Duan, Erin Hong, Dongxu Zhao, Eric Turner, Alex Wong, and Yunwen Zhou. Fisheye3R: Adapting unified 3D feed-forward foundation models to fisheye lenses.arXiv:2603.28896 [cs.CV], 2026

    Pith/arXiv arXiv 2026

  30. [30]

    FishRoPE: Projective rotary position embeddings for omnidirectional visual perception.arXiv:2604.10391 [cs.CV], 2026

    Rahul Ahuja, Mudit Jain, Bala Murali Manoghar Sai Sudhakar, Venkatraman Narayanan, Pratik Likhar, Varun Ravi Kumar, and Senthil Yogamani. FishRoPE: Projective rotary position embeddings for omnidirectional visual perception.arXiv:2604.10391 [cs.CV], 2026

    Pith/arXiv arXiv 2026

  31. [31]

    Depth Anywhere: Enhancing 360 monocular depth estimation via perspective distillation and unlabeled data augmentation

    Ning-Hsu Wang and Yu-Lun Liu. Depth Anywhere: Enhancing 360 monocular depth estimation via perspective distillation and unlabeled data augmentation. InNeurIPS, 2024

    2024

  32. [32]

    VGGT-360: Geometry-consistent zero-shot panoramic depth estimation.arXiv:2603.18943 [cs.CV], 2026

    Jiayi Yuan, Haobo Jiang, De Wen Soh, and Na Zhao. VGGT-360: Geometry-consistent zero-shot panoramic depth estimation.arXiv:2603.18943 [cs.CV], 2026

    Pith/arXiv arXiv 2026

  33. [33]

    RPG360: Robust 360 depth estimation with perspective foundation models and graph optimization

    Dongki Jung, Jaehoon Choi, Yonghan Lee, and Dinesh Manocha. RPG360: Robust 360 depth estimation with perspective foundation models and graph optimization. InNeurIPS, 2025

    2025

  34. [34]

    Test-time training with self-supervision for generalization under distribution shifts

    Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt. Test-time training with self-supervision for generalization under distribution shifts. InICML, pages 9229–9248, 2020

    2020

  35. [35]

    Freeman, and Hao Tan

    Tianyuan Zhang, Sai Bi, Yicong Hong, Kai Zhang, Fujun Luan, Songlin Yang, Kalyan Sunkavalli, William T. Freeman, and Hao Tan. Test-time training done right. InICLR, 2026. 11

    2026

  36. [36]

    TTT3R: 3D recon- struction as test-time training

    Xingyu Chen, Yue Chen, Yuliang Xiu, Andreas Geiger, and Anpei Chen. TTT3R: 3D recon- struction as test-time training. InICLR, 2026

    2026

  37. [37]

    Scal3R: Scalable test-time training for large-scale 3D reconstruction.arXiv:2604.08542 [cs.CV], 2026

    Tao Xie, Peishan Yang, Yudong Jin, Yingfeng Cai, Wei Yin, Weiqiang Ren, Qian Zhang, Wei Hua, Sida Peng, Xiaoyang Guo, and Xiaowei Zhou. Scal3R: Scalable test-time training for large-scale 3D reconstruction.arXiv:2604.08542 [cs.CV], 2026

    Pith/arXiv arXiv 2026

  38. [38]

    Juho Kannala and Sami S. Brandt. A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses.IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(8):1335–1340, 2006

    2006

  39. [39]

    An enhanced unified camera model.IEEE Robotics and Automation Letters, 1(1):137–144, 2015

    Bogdan Khomutenko, Gaëtan Garcia, and Philippe Martinet. An enhanced unified camera model.IEEE Robotics and Automation Letters, 1(1):137–144, 2015

    2015

  40. [40]

    DINOv2: Learning robust visual features without supervision.TMLR, 2024

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Mahmoud Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Hervé Jegou, Julien Mairal, Patrick La...

    2024

  41. [41]

    Rotary position embedding for vision transformer

    Byeongho Heo, Song Park, Dongyoon Han, and Sangdoo Yun. Rotary position embedding for vision transformer. InECCV, pages 289–305, 2024

    2024

  42. [42]

    Vision transformers for dense prediction

    René Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vision transformers for dense prediction. InICCV, pages 12179–12188, 2021

    2021

  43. [43]

    Tracking feature points of fisheye full-view image by normalized image patch.IEEJ Transactions on Electronics, Information and Systems, 132(9):1516–1523, 2012

    Xuebin Qin and Shigang Li. Tracking feature points of fisheye full-view image by normalized image patch.IEEJ Transactions on Electronics, Information and Systems, 132(9):1516–1523, 2012

    2012

  44. [44]

    UFM: A simple path towards unified dense correspondence with flow

    Yuchen Zhang, Nikhil Keetha, Chenwei Lyu, Bhuvan Jhamb, Yutian Chen, Yuheng Qiu, Jay Karhade, Shreyas Jha, Yaoyu Hu, Deva Ramanan, Sebastian Scherer, and Wenshan Wang. UFM: A simple path towards unified dense correspondence with flow. InNeurIPS, 2025

    2025

  45. [45]

    MAGSAC++: A fast, reliable, and accurate robust estimator

    Daniel Barath, Jana Noskova, Maksym Ivashechkin, and Jiri Matas. MAGSAC++: A fast, reliable, and accurate robust estimator. InCVPR, pages 1304–1312, 2020

    2020

  46. [46]

    Clément Godard, Oisin Mac Aodha, and Gabriel J. Brostow. Unsupervised monocular depth estimation with left-right consistency. InCVPR, pages 270–279, 2017

    2017

  47. [47]

    AnyCam: Learning to recover camera poses and intrinsics from casual videos

    Felix Wimbauer, Weirong Chen, Dominik Muhle, Christian Rupprecht, and Daniel Cremers. AnyCam: Learning to recover camera poses and intrinsics from casual videos. InCVPR, pages 16717–16727, 2025

    2025

  48. [48]

    KITTI-360: A novel dataset and benchmarks for urban scene understanding in 2D and 3D.IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(3):3292–3310, 2023

    Yiyi Liao, Jun Xie, and Andreas Geiger. KITTI-360: A novel dataset and benchmarks for urban scene understanding in 2D and 3D.IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(3):3292–3310, 2023

    2023

  49. [49]

    The TUM VI benchmark for evaluating visual-inertial odometry

    David Schubert, Thore Goll, Nikolaus Demmel, Vladyslav Usenko, Jorg Stuckler, and Daniel Cremers. The TUM VI benchmark for evaluating visual-inertial odometry. InIROS, pages 1680–1687, 2018

    2018

  50. [50]

    ScanNet++: A high-fidelity dataset of 3D indoor scenes

    Chandan Yeshwanth, Yueh-Cheng Liu, Matthias Nießner, and Angela Dai. ScanNet++: A high-fidelity dataset of 3D indoor scenes. InICCV, pages 12–22, 2023

    2023

  51. [51]

    Schonberger, Silvano Galliani, Torsten Sattler, Konrad Schindler, Marc Pollefeys, and Andreas Geiger

    Thomas Schops, Johannes L. Schonberger, Silvano Galliani, Torsten Sattler, Konrad Schindler, Marc Pollefeys, and Andreas Geiger. A multi-view stereo benchmark with high-resolution images and multi-camera videos. InCVPR, pages 3260–3269, 2017

    2017

  52. [52]

    FIORD: A fisheye indoor-outdoor dataset with LiDAR ground truth for 3D scene reconstruction and benchmarking

    Ulas Gunes, Matias Turkulainen, Xuqian Ren, Arno Solin, Juho Kannala, and Esa Rahtu. FIORD: A fisheye indoor-outdoor dataset with LiDAR ground truth for 3D scene reconstruction and benchmarking. InSCIA, pages 3–17, 2025

    2025

  53. [53]

    Depth map prediction from a single image using a multi-scale deep network

    David Eigen, Christian Puhrsch, and Rob Fergus. Depth map prediction from a single image using a multi-scale deep network. InNeurIPS, 2014. 12 A Evaluation metrics For each frame pair (i, j), let ( ˆRij, ˆtij) be the predicted relative pose and (R⋆ ij, t⋆ ij) the ground-truth pose. We report angular rotation and translation-direction errors in degrees: Re...

    2014