The reviewed record of science sign in
Pith

arxiv: 2607.06165 · v1 · pith:BOXU4CYM · submitted 2026-07-07 · cs.RO

EAGOR: Embodied Reasoning in Omni-direction

Shriram Damodaran , Soumyaratna Debnath , Yan Wu , Wei-Yun Yau , Addison Lin Wang This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-07-08 14:29 UTCglm-5.2pith:BOXU4CYMrecord.jsonopen to challenge →

classification cs.RO
keywords directionaleagorreasoningagentsphericaldirectionembodiednavigation
0
0 comments X

The pith

Spherical belief field lets robots reason in 360° without training

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

EAGOR reframes embodied directional reasoning with 360° cameras as a recursive Bayesian estimation problem on the sphere rather than pixel-coordinate prediction in a flat equirectangular projection (ERP) image. The central object is the Spherical Harmonic Belief Field (SH-BF), which represents the agent's continuous belief over possible target directions using spherical harmonic coefficients. At each timestep, a frozen vision-language model (VLM) produces a spatial response map over the panorama; EAGOR lifts this map onto the sphere as a directional log-likelihood, projects it into the spherical harmonic basis, and fuses it additively with a rotated copy of the previous belief. When the agent moves, the belief is propagated via Wigner-D rotation matrices that act directly on the harmonic coefficients, maintaining geometric consistency without re-querying the VLM. The final target direction is decoded analytically from the degree-1 coefficients as a spherical Fréchet mean. Because the belief lives on the sphere throughout, the framework eliminates the seam discontinuities, latitude-dependent distortions, and interpolation errors inherent to ERP representations. The entire pipeline requires no task-specific training of the backbone VLM. The paper claims that this geometric decoupling—VLM for semantic evidence, SH-BF for directional state estimation—yields substantial gains: +34.4% and +45.6% relative improvement on the HOS and OSR-Bench benchmarks respectively, a 14.6% navigation success increase, 17.7% fewer steps, and 24.5% lower angular error, with a 3B-parameter model outperforming a standalone 72B model on spatial reasoning.

Core claim

The paper's central discovery is that treating VLM spatial attention as a directional likelihood on the sphere and accumulating it through a spherical-harmonic Bayesian filter produces geometrically consistent, motion-equivariant directional estimates that substantially outperform direct pixel-coordinate prediction from ERP images. The Spherical Harmonic Belief Field is the mechanism that makes this work: it provides a continuous, globally defined, rotation-aware representation that supports additive evidence accumulation in coefficient space and analytical direction decoding, all without training the VLM backbone.

What carries the argument

The Spherical Harmonic Belief Field (SH-BF) is a continuous belief representation over target directions on the unit sphere, expressed in the real spherical harmonic basis up to bandlimit L=7. It supports three operations in coefficient space: (1) projection of per-frame VLM response maps as directional log-likelihood observations, (2) Wigner-D rotation of the prior belief under agent ego-motion, and (3) additive Bayesian fusion of prior and observation. The MAP direction is decoded analytically from the degree-1 coefficients as a spherical Fréchet mean, avoiding grid search.

Load-bearing premise

The framework treats the VLM's spatially distributed attention patch as a calibrated directional likelihood field proportional to the log-probability of the target existing in each viewing direction. VLM attention scores are uncalibrated heuristics; if they systematically misrepresent true target probability, the recursive Bayesian update will propagate and amplify that bias rather than converge to the correct direction.

What would settle it

If the VLM's target-conditioned response map does not correlate monotonically with true target direction probability—e.g., if attention peaks on distractors or is distorted by ERP artifacts before lifting—then the SH-BF Bayesian update will accumulate biased evidence and the belief will diverge from the true target direction over time, producing worse estimates than single-frame prediction.

Figures

Figures reproduced from arXiv: 2607.06165 by Addison Lin Wang, Shriram Damodaran, Soumyaratna Debnath, Wei-Yun Yau, Yan Wu.

Figure 1
Figure 1. Figure 1: EAGOR for geometry-aware embodied omni-directional reasoning (⃝ = Target) Abstract: Omni-directional (360◦ ) cameras enable embodied agents with a wide and holistic view of their surroundings, making them advantageous for directional reasoning in embodied tasks, such as navigation, and object search. Existing Vi￾sion Language Models (VLMs) project 360◦ data to 2D planar images via com￾monly used equirectan… view at source ↗
Figure 2
Figure 2. Figure 2: Qualitative results for Waypoint Following task. (a) Initial target-direction estimate. (b) Directional estimates after viewpoint changes during waypoint following. Qwen2.5-VL + EAGOR (⋄) consistently preserves the target direction under agent motion, whereas the centroid (⃝) and grid-based (□) baselines drift due to accumulated ERP interpolation errors. 2 Related Works Omni-directional Perception for Embo… view at source ↗
Figure 3
Figure 3. Figure 3: Overview of EAGOR framework. See main texts of Sec.3 for details. we introduce the Spherical Harmonic Belief Field (SH-BF), which represents belief on S 2 and supports consistent evidence accumulation, equivariant propagation under agent rotation, and MAP direction estimation. In short, the VLM identifies what to look for, while SH-BF maintains where the target lies relative to the agent at time t. 3.1 Fro… view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative results for EAGOR using Qwen2.5-VL. Direction reasoning during seam crossing (target: bed) where → indicates the EAGOR predicted direction, while → shows the erro￾neous direction produced by the centroid-based baseline. ⃝ marks the belief centroid. To support dynamic belief updates, the representation must satisfy two requirements: 1) it must accumulate evidence additively over time and 2) tran… view at source ↗
Figure 5
Figure 5. Figure 5: Direction Estimation under Stationary and Mobile Operation. Top: stationary robot with moving target, (· · ·) is the target trajectory. Bottom: moving robot with stationary target. In each frame, (→) denotes the spherical directional prediction, (→) the ERP-based VLM baseline, and (□) the accumulated directional belief field. ⃝ = Target, and (→) denotes the Ground Truth. the posterior belief only for the c… view at source ↗
Figure 6
Figure 6. Figure 6: Embodied omni-directional reasoning in Habitat-Sim: (a) Waypoint Following (b) Map-Free Navigation. We evaluate EAGOR as a training-free, model￾agnostic framework for embodied omni￾directional reasoning. Our evaluation is de￾signed to validate two core claims: that SH-BF maintains geometrically consistent directional belief under egocentric motion, and that this geometric correctness translates to improved… view at source ↗
Figure 7
Figure 7. Figure 7: Effect of SH bandlimit. The SH bandlimit L balances angular precision and side￾lobe ringing. As shown in [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
read the original abstract

Omni-directional (360{\deg}) cameras provide embodied agents with a holistic view of their surroundings, making them suited for directional reasoning in tasks such as navigation and object search. Existing Vision Language Models (VLMs) project 360{\deg} observations to 2D equirectangular projection (ERP) images and process them using architectures designed for perspective images. However, they ignore the spherical nature of 360{\deg} observations, where each pixel represents a viewing direction relative to the agent. Consequently, their direction estimates often become inconsistent under camera view transformations caused by agent motion. This limitation is particularly critical for map-free navigation, where the agent must continuously estimate the target direction in its egocentric frame. We propose EAGOR, a training-free, geometry-aware framework for embodied 360{\deg} directional reasoning. Instead of predicting target directions as ERP image coordinates, EAGOR formulates directional reasoning as recursive Bayesian estimation directly on the sphere. It maintains a continuous belief over target directions and propagates it equivariantly under agent motion without training the backbone VLMs. To achieve this, we introduce the Spherical Harmonic Belief Field (SH-BF), whose spherical harmonic representation provides a globally defined, rotation-aware basis for directional estimation on the spherical manifold. This formulation eliminates ERP seam discontinuities, latitude distortions, and interpolation errors. We evaluate EAGOR on two benchmark datasets and real-world experiments with a legged robot across directional reasoning tasks. EAGOR consistently outperforms existing methods, achieving average relative gains of +34.4% and +45.6% on HOS and OSR-Bench, respectively, while improving navigation success by +14.6%, reducing step count by 17.7%, and lowering mean angular error by 24.5%.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 6 minor

Summary. The paper introduces EAGOR, a training-free framework for embodied omnidirectional reasoning that formulates target-direction estimation as recursive Bayesian filtering on the sphere. The core technical contribution is the Spherical Harmonic Belief Field (SH-BF), which represents directional belief using spherical harmonics, propagates it equivariantly under agent rotation via Wigner-D matrices, and decodes the target direction via the Fréchet mean. The approach is evaluated on waypoint following, map-free navigation (Habitat-Sim), active visual search (HOS, OSR-Bench), and real-world dynamic tracking on a legged robot. The central claim is that maintaining belief directly on the sphere avoids ERP seam discontinuities, latitude distortions, and interpolation errors, yielding consistent directional estimates under ego-motion.

Significance. The paper addresses a genuine representation gap in embodied 360° reasoning: treating VLM outputs as directional likelihoods on S² rather than as ERP pixel coordinates. The SH-BF formulation is clean and parameter-free in its core derivation (Eqs. 2–6), grounding the belief representation in well-established mathematical frameworks (spherical harmonics, Wigner-D rotations, Bayesian filtering). The equivariant propagation via Wigner-D matrices in coefficient space is a principled solution to the motion-consistency problem. The real-world deployment on a legged robot and the evaluation across multiple VLM backbones (Qwen2.5-VL, Gemma-3) strengthen the practical relevance. The framework is training-free and model-agnostic, which is a notable strength for adoptability. However, the experimental attribution of headline gains to the spherical harmonic representation specifically—rather than to temporal evidence accumulation more generally—is not fully established on the benchmarks producing the headline numbers.

major comments (2)
  1. §4.2, Table 1 (HOS and OSR-Bench): The headline gains (+34.4% on HOS, +45.6% on OSR-Bench) compare EAGOR—which combines (a) spherical harmonic belief representation, (b) equivariant rotation propagation, and (c) temporal evidence accumulation—against standalone single-frame VLM baselines that lack any temporal accumulation. For the navigation tasks (Tables 2–3), an ERP-space temporal accumulation baseline ('Grid') is included, and EAGOR outperforms it. However, for Active Visual Search (Table 1), no ERP-space temporal accumulation baseline is included. This means the gains on HOS and OSR-Bench could be largely attributable to temporal evidence accumulation (which any simple scheme could provide) rather than to the spherical harmonic representation specifically. The paper's core novelty claim—that maintaining belief on the sphere via SH is superior to ERP-based approaches—is not directly
  2. §3.1: The interpretation of the VLM's target-conditioned response map ℓ_t(u,v) as a directional likelihood field proportional to the log-likelihood of the target existing in that viewing direction is the foundational assumption of the entire framework. VLM attention scores are uncalibrated heuristics, and the paper does not validate that ℓ_t(ω) behaves as a proper likelihood (e.g., that its peaks correspond to higher target probability, that its relative magnitudes are meaningful). While empirical results show the system works in practice, the paper would benefit from either (i) a calibration analysis showing that VLM response maps correlate with target presence probability, or (ii) an explicit acknowledgment that the log-likelihood interpretation is an approximation and a discussion of conditions under which it may fail. This is load-bearing because the Bayesian update (Eq. 5) and theFr
minor comments (6)
  1. §3.2, Eq. (5): The additive update c^(t) = c̃^(t) + b^(t) corresponds to log-space Bayesian fusion with equal weighting of prior and observation. This implicitly assumes that the observation noise is stationary and uniform across directions. A brief discussion of why equal weighting is appropriate (or whether a discount factor on the prior would be beneficial) would strengthen the presentation.
  2. §3.2: The statement 'L=7 is the SH bandlimit' is introduced without justification in the main text. Fig. 7 provides the ablation, but the choice of L=7 should be cross-referenced when first introduced.
  3. §4.1: The baselines for HOS and OSR-Bench are described as 'fine-tuned and zero-shot VLM baselines,' but Table 1 only shows zero-shot VLM results. If fine-tuned baselines were evaluated, their results should be reported; if not, the description should be corrected.
  4. Fig. 7: The x-axis label 'Angular Separation δ' and the curve labels 'L=12 (azimuth. min)' are unclear. Clarifying what 'azimuth. min' refers to would help the reader.
  5. §4.2, Table 2: The 'Grid' baseline shows a MAE of 138.9° on segment L2, which is extremely high and suggests a systematic failure (e.g., 180° ambiguity). A brief note explaining this failure mode would help the reader interpret the comparison fairly.
  6. The abstract states 'reducing step count by 17.7%,' but Table 3 shows steps reduced from 61.0 (Centroid) to 50.2 (EAGOR), which is a 17.7% reduction relative to Centroid. Clarifying which baseline is the reference for each percentage would improve precision.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for a careful and constructive review. The referee correctly identifies two important gaps in our experimental design and theoretical framing. We address both below and commit to revisions.

read point-by-point responses

  1. Referee: §4.2, Table 1 (HOS and OSR-Bench): No ERP-space temporal accumulation baseline is included for Active Visual Search, so headline gains could be attributable to temporal evidence accumulation rather than the spherical harmonic representation specifically.

    Authors: The referee is correct. For the navigation tasks (Tables 2–3), we included the 'Grid' baseline, which performs temporal accumulation in ERP pixel space, and EAGOR outperforms it. However, for Active Visual Search (Table 1), we did not include an ERP-space temporal accumulation baseline, which means the attribution of the +34.4% and +45.6% gains to the spherical harmonic representation specifically is not directly supported by the current experiments on those benchmarks. This is a valid gap in our experimental design. We will address it in the revision by adding an ERP-space temporal accumulation baseline (analogous to the 'Grid' baseline used in Tables 2–3) to the HOS and OSR-Bench experiments in Table 1. This will allow a direct comparison between temporal accumulation in ERP space versus temporal accumulation on the sphere via SH-BF, isolating the contribution of the spherical representation from the contribution of evidence accumulation per se. We expect the spherical representation to retain an advantage—particularly under seam crossings and rotation, as demonstrated in Tables 2–3—but we agree the reader should be able to see this directly for the active visual search benchmarks as well. We will also temper the language in the abstract and main text to clarify that the gains reflect the combination of (a) spherical belief representation, (b) equivariant propagation, and (c) temporal accumulation, and that the relative contribution of each component is partially—but not fully—disentangled across all benchmarks. revision: yes

  2. Referee: §3.1: The interpretation of VLM response maps as directional likelihood fields is unvalidated. VLM attention scores are uncalibrated heuristics; the paper should either provide a calibration analysis or explicitly acknowledge the approximation and discuss failure conditions.

    Authors: The referee raises a legitimate concern. We do not claim that VLM response maps are calibrated probabilities, and our use of the log-likelihood formulation is best understood as a modeling assumption: we treat the VLM's spatial response as a directional evidence signal that, when accumulated recursively, yields a useful posterior over target directions. We agree that this should be stated more explicitly rather than left implicit. In the revision, we will: (i) add an explicit statement in §3.1 that the log-likelihood interpretation is an approximation, not a claim of calibrated probability; (ii) add a brief calibration analysis showing the empirical correlation between VLM response map peaks and target presence on a subset of HOS episodes, reporting rank correlation between response intensity and ground-truth target direction proximity; and (iii) expand the limitations discussion (currently in §5, Table 4) to explicitly address conditions under which the likelihood assumption breaks down—namely, multi-instance confusion (where multiple peaks of similar intensity exist), rare targets (where the response map may be flat or dominated by false positives), and fine-grained text/OCR tasks (where spatial attention may not reflect directional likelihood at all). Table 4 already provides some evidence for these failure modes; we will connect it more directly to the likelihood assumption. We note that the recursive Bayesian formulation is somewhat robust to miscalibration because it accumulates evidence over multiple views, which partially mitigates single-frame noise—a point we will also make explicit. revision: yes

Circularity Check

0 steps flagged

No significant circularity; SH-BF derivation is parameter-free and grounded in external mathematics

full rationale

The paper's core derivation chain—VLM response map → log-likelihood → SH projection (Eq. 3) → Wigner-D rotation (Eq. 4) → additive belief update (Eq. 5) → Fréchet mean decoding (Eq. 6)—is a self-contained application of standard spherical harmonics, Wigner-D matrices, and recursive Bayesian filtering. No step reduces to its own inputs by construction. The SH bandlimit L=7 is a design parameter chosen via ablation (Fig. 7), not a fitted parameter renamed as a prediction. The VLM attention map is treated as an observation likelihood by assumption (Sec. 3.1), but this is a modeling assumption stated openly, not a circular definition where the output is defined in terms of the input. The paper does cite prior work by some of the same authors (Ref [3], [47], [48], [49]), but these citations are tangential to the central derivation—they concern neuroscience-inspired perspectives and bio-inspired representations, not the SH-BF formulation itself. The mathematical framework (spherical harmonics, Wigner-D rotations, Bayesian filtering) is standard and externally grounded. The experimental results compare against external baselines (Centroid, Cent-Circ, Grid, standalone VLMs) on public benchmarks (HOS, OSR-Bench, Habitat). The reader's concern about VLM likelihood calibration is a validity/assumption concern, not a circularity concern—the paper does not define its prediction in terms of the thing it claims to predict. The skeptic's concern about missing ERP-accumulation baselines on HOS/OSR-Bench is an experimental attribution concern, not circularity. No fitted parameter is renamed as a prediction, no self-citation chain forces the conclusion, and no ansatz is smuggled in via self-citation. The derivation is self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

2 free parameters · 2 axioms · 1 invented entities

The framework relies on two core domain assumptions: that VLM attention maps serve as valid probabilistic likelihoods, and that agent motion can be approximated as pure rotation. The SH bandlimit L=7 is a hand-chosen hyperparameter. The SH-BF is a newly introduced mathematical representation, not a physical entity, and its properties are well-defined.

free parameters (2)
  • SH bandlimit L = 7
    Chosen by hand to balance angular precision and side-lobe ringing (Sec 4.3). Not fitted to target data, but a hyperparameter selected ad hoc.
  • epsilon (epsilon)
    Introduced in the log-likelihood field r_t(omega) = log(l_t(omega) + epsilon) to avoid numerical instability (Sec 3.1). Value not specified in main text.
axioms (2)
  • domain assumption VLM target-conditioned response maps can be validly interpreted as directional likelihood fields proportional to the log-likelihood of the target's existence in that viewing direction.
    Invoked in Sec 3.1 when converting the image response l_t(u,v) to a directional likelihood field l_t(omega). This is the foundational assumption enabling the Bayesian formulation.
  • domain assumption Agent motion between timesteps can be approximated by a pure egocentric rotation R_t in SO(3), ignoring translational parallax effects.
    Invoked in Sec 3.2 when propagating the prior belief using Wigner-D rotation. The paper notes this limitation for future work, acknowledging it assumes parallax remains within single-frame observation noise.
invented entities (1)
  • Spherical Harmonic Belief Field (SH-BF) independent evidence
    purpose: Represents continuous directional belief on S^2 and supports consistent evidence accumulation and equivariant propagation under agent rotation.
    The SH-BF is a mathematical construct rather than a physical entity. Its existence and properties are derived from standard spherical harmonics. It provides falsifiable predictions (e.g., lower angular error under rotation) that are tested in the experiments.

pith-pipeline@v1.1.0-glm · 16928 in / 2392 out tokens · 371367 ms · 2026-07-08T14:29:51.925407+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

55 extracted references · 55 canonical work pages · 29 internal anchors

  1. [1]

    A. Das, S. Datta, G. Gkioxari, S. Lee, D. Parikh, and D. Batra. Embodied question answering,

    work page

  2. [2]

    URLhttps://arxiv.org/abs/1711.11543

    work page internal anchor Pith review Pith/arXiv arXiv

  3. [3]

    Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environments

    J. Krantz, E. Wijmans, A. Majumdar, D. Batra, and S. Lee. Beyond the nav-graph: Vision-and- language navigation in continuous environments, 2020. URLhttps://arxiv.org/abs/ 2004.02857

    work page internal anchor Pith review Pith/arXiv arXiv 2020

  4. [4]

    B. D. Manh, S. Debnath, Z. Zhang, S. Damodaran, A. Kumar, Y . Zhang, L. Mi, E. Cambria, and L. Wang. Mind meets space: Rethinking agentic spatial intelligence from a neuroscience- inspired perspective.arXiv preprint arXiv:2509.09154, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  5. [5]

    Winters, J

    N. Winters, J. Gaspar, G. Lacey, and J. Santos-Victor. Omni-directional vision for robot navi- gation. InProceedings IEEE Workshop on Omnidirectional Vision (Cat. No.PR00704), pages 21–28, 2000. doi:10.1109/OMNVIS.2000.853799

    work page doi:10.1109/omnvis.2000.853799 2000

  6. [6]

    Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments

    P. Anderson, Q. Wu, D. Teney, J. Bruce, M. Johnson, N. S ¨underhauf, I. Reid, S. Gould, and A. van den Hengel. Vision-and-language navigation: Interpreting visually-grounded naviga- tion instructions in real environments, 2018. URLhttps://arxiv.org/abs/1711.07280

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  7. [7]

    J. Bai, S. Bai, Y . Chu, Z. Cui, K. Dang, X. Deng, Y . Fan, W. Ge, Y . Han, F. Huang, B. Hui, L. Ji, M. Li, J. Lin, R. Lin, D. Liu, G. Liu, C. Lu, K. Lu, J. Ma, R. Men, X. Ren, X. Ren, C. Tan, S. Tan, J. Tu, P. Wang, S. Wang, W. Wang, S. Wu, B. Xu, J. Xu, A. Yang, H. Yang, J. Yang, S. Yang, Y . Yao, B. Yu, H. Yuan, Z. Yuan, J. Zhang, X. Zhang, Y . Zhang, ...

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  8. [8]

    Y . Dong, C. Fang, L. Bo, Z. Dong, and P. Tan. Panocontext-former: Panoramic total scene understanding with a transformer, 2023. URLhttps://arxiv.org/abs/2305.12497

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  9. [9]

    Tateno, N

    K. Tateno, N. Navab, and F. Tombari. Distortion-aware convolutional filters for dense predic- tion in panoramic images. In V . Ferrari, M. Hebert, C. Sminchisescu, and Y . Weiss, editors, Computer Vision – ECCV 2018, pages 732–750, Cham, 2018. Springer International Publish- ing. ISBN 978-3-030-01270-0

    work page 2018

  10. [10]

    Coors, A

    B. Coors, A. P. Condurache, and A. Geiger. Spherenet: Learning spherical representations for detection and classification in omnidirectional images. In V . Ferrari, M. Hebert, C. Smin- chisescu, and Y . Weiss, editors,Computer Vision – ECCV 2018, pages 525–541, Cham, 2018. Springer International Publishing

    work page 2018

  11. [11]

    Su and K

    Y .-C. Su and K. Grauman. Learning spherical convolution for fast features from 360 imagery,

    work page

  12. [12]

    URLhttps://arxiv.org/abs/1708.00919

    work page internal anchor Pith review Pith/arXiv arXiv

  13. [13]

    Visual Question Answering on 360{\deg} Images

    S.-H. Chou, W.-L. Chao, W.-S. Lai, M. Sun, and M.-H. Yang. Visual question answering on 360 images, 2020. URLhttps://arxiv.org/abs/2001.03339

    work page internal anchor Pith review Pith/arXiv arXiv 2020

  14. [14]

    D. S. Chaplot, D. Gandhi, A. Gupta, and R. Salakhutdinov. Object goal navigation using goal-oriented semantic exploration, 2020. URLhttps://arxiv.org/abs/2007.00643

    work page internal anchor Pith review Pith/arXiv arXiv 2020

  15. [15]

    H. Yun, Y . Yu, W. Yang, K. Lee, and G. Kim. Pano-avqa: Grounded audio-visual question answering on 360◦ videos, 2021. URLhttps://arxiv.org/abs/2110.05122

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  16. [16]

    M. Eder, M. Shvets, J. Lim, and J.-M. Frahm. Tangent images for mitigating spherical distor- tion, 2020. URLhttps://arxiv.org/abs/1912.09390

    work page internal anchor Pith review Pith/arXiv arXiv 2020

  17. [17]

    Benny and L

    Y . Benny and L. Wolf. Sphereuformer: A u-shaped transformer for spherical 360 perception,

    work page

  18. [18]

    URLhttps://arxiv.org/abs/2412.06968. 9

    work page internal anchor Pith review Pith/arXiv arXiv

  19. [19]

    Bending Reality: Distortion-aware Transformers for Adapting to Panoramic Semantic Segmentation

    J. Zhang, K. Yang, C. Ma, S. Reiß, K. Peng, and R. Stiefelhagen. Bending reality: Distortion- aware transformers for adapting to panoramic semantic segmentation, 2022. URLhttps: //arxiv.org/abs/2203.01452

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  20. [20]

    E. Unlu. Spherical position encoding for transformers, 2023. URLhttps://arxiv.org/ abs/2310.04454

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  21. [21]

    H. Li, W. Zheng, J. He, Y . Liu, X. Lin, X. Yang, Y .-C. Chen, and C. Guo. Da2: Depth anything in any direction, 2025. URLhttps://arxiv.org/abs/2509.26618

    work page arXiv 2025

  22. [22]

    C. Wang, X. Lin, J. Liu, Y . Liu, Z. Wang, D. Qi, Y . Yan, and X. Chen. Panoworld: Towards spatial supersensing in 360◦ panorama world, 2026. URLhttps://arxiv.org/abs/2605. 13169

    work page 2026

  23. [23]

    L. Yang, H. Duan, R. Tao, J. Cheng, S. Wu, Y . Li, J. Liu, X. Min, and G. Zhai. Odi-bench: Can mllms understand immersive omnidirectional environments?, 2026. URLhttps://arxiv. org/abs/2510.11549

    work page arXiv 2026

  24. [24]

    Are Multimodal Large Language Models Ready for Omnidirectional Spatial Reasoning?

    Z. Dongfang, X. Zheng, Z. Weng, Y . Lyu, D. P. Paudel, L. V . Gool, K. Yang, and X. Hu. Are multimodal large language models ready for omnidirectional spatial reasoning?, 2025. URL https://arxiv.org/abs/2505.11907

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  25. [25]

    W. Liu, Q. Xue, H. Wang, X. Yin, B. Yang, and W. Gao. Spatial reasoning in multimodal large language models: A survey of tasks, benchmarks and methods, 2025. URLhttps: //arxiv.org/abs/2511.15722

    work page arXiv 2025

  26. [26]

    Zheng, Z

    X. Zheng, Z. Dongfang, L. Jiang, B. Zheng, Y . Guo, Z. Zhang, G. Albanese, R. Yang, M. Ma, Z. Zhang, C. Liao, D. Zhen, Y . Lyu, Y . Fu, B. Ren, L. Zhang, D. P. Paudel, N. Sebe, L. V . Gool, and X. Hu. Multimodal spatial reasoning in the large model era: A survey and benchmarks,

    work page

  27. [27]

    URLhttps://arxiv.org/abs/2510.25760

    work page arXiv

  28. [28]

    X. Zhao, W. Cai, L. Tang, and T. Wang. Imaginenav: Prompting vision-language models as embodied navigator through scene imagination, 2024. URLhttps://arxiv.org/abs/ 2410.09874

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  29. [29]

    Q. Jin, Y . Wu, and C. Chen. Panonav: Mapless zero-shot object navigation with panoramic scene parsing and dynamic memory, 2025. URLhttps://arxiv.org/abs/2511.06840

    work page arXiv 2025

  30. [30]

    VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigation

    N. Yokoyama, S. Ha, D. Batra, J. Wang, and B. Bucher. Vlfm: Vision-language frontier maps for zero-shot semantic navigation, 2023. URLhttps://arxiv.org/abs/2312.03275

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  31. [31]

    Z. Wang, X. Li, J. Yang, Y . Liu, and S. Jiang. Gridmm: Grid memory map for vision-and- language navigation, 2023. URLhttps://arxiv.org/abs/2307.12907

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  32. [32]

    H. Wang, W. Liang, L. V . Gool, and W. Wang. Dreamwalker: Mental planning for continuous vision-language navigation, 2023. URLhttps://arxiv.org/abs/2308.07498

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  33. [33]

    Zhang, K

    J. Zhang, K. Wang, S. Wang, M. Li, H. Liu, S. Wei, Z. Wang, Z. Zhang, and H. Wang. Uni- navid: A video-based vision-language-action model for unifying embodied navigation tasks,

    work page

  34. [34]

    URLhttps://arxiv.org/abs/2412.06224

    work page internal anchor Pith review Pith/arXiv arXiv

  35. [35]

    H. Yu, Y . Han, X. Zhang, B. Yin, B. Chang, X. Han, X. Liu, J. Zhang, M. Pavone, C. Feng, S. Xie, and Y . Li. Thinking in 360: Humanoid visual search in the wild, 2025. URLhttps: //arxiv.org/abs/2511.20351

    work page arXiv 2025

  36. [36]

    Kendall and R

    A. Kendall and R. Cipolla. Modelling uncertainty in deep learning for camera relocalization,

    work page

  37. [37]

    URLhttps://arxiv.org/abs/1509.05909. 10

    work page internal anchor Pith review Pith/arXiv arXiv

  38. [38]

    Baumann, R

    A. Baumann, R. Li, M. Klasson, S. Mentu, S. Karthik, Z. Akata, A. Solin, and M. Trapp. Post- hoc probabilistic vision-language models, 2026. URLhttps://arxiv.org/abs/2412. 06014

    work page 2026

  39. [39]

    G. Kurz, I. Gilitschenski, and U. D. Hanebeck. Recursive bayesian filtering in circular state spaces.IEEE Aerospace and Electronic Systems Magazine, 31(3):70–87, Mar. 2016. ISSN 0885-8985. doi:10.1109/maes.2016.150083. URLhttp://dx.doi.org/10.1109/MAES. 2016.150083

    work page doi:10.1109/maes.2016.150083 2016

  40. [40]

    Peretroukhin, M

    V . Peretroukhin, M. Giamou, W. Nicholas Greene, D. Rosen, J. Kelly, and N. Roy. A smooth representation of belief over so(3) for deep rotation learning with uncertainty. InRobotics: Science and Systems XVI, RSS2020. Robotics: Science and Systems Foundation, 2020. doi: 10.15607/rss.2020.xvi.007. URLhttp://dx.doi.org/10.15607/RSS.2020.XVI.007

    work page doi:10.15607/rss.2020.xvi.007 2020

  41. [41]

    X. Shao, Y . Tang, P. Xie, K. Zhou, Y . Zhuang, X. Quan, J. Hao, L. Zeng, and X. Li. More than a point: Capturing uncertainty with adaptive affordance heatmaps for spatial grounding in robotic tasks, 2025. URLhttps://arxiv.org/abs/2510.10912

    work page arXiv 2025

  42. [42]

    Zangeneh, L

    F. Zangeneh, L. Bruns, A. Dekel, A. Pieropan, and P. Jensfelt. A probabilistic framework for visual localization in ambiguous scenes, 2023. URLhttps://arxiv.org/abs/2301. 02086

    work page 2023

  43. [43]

    Learning SO(3) Equivariant Representations with Spherical CNNs

    C. Esteves, C. Allen-Blanchette, A. Makadia, and K. Daniilidis. Learning so(3) equivariant representations with spherical cnns, 2018. URLhttps://arxiv.org/abs/1711.06721

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  44. [44]

    Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs

    Y .-L. Liao and T. Smidt. Equiformer: Equivariant graph attention transformer for 3d atomistic graphs, 2023. URLhttps://arxiv.org/abs/2206.11990

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  45. [45]

    J. Lee, H. Park, B.-U. Lee, and K. Joo. Hush: Holistic panoramic 3d scene understanding using spherical harmonics. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16599–16608, June 2025

    work page 2025

  46. [46]

    Zhang, I

    C. Zhang, I. Budvytis, S. Liwicki, and R. Cipolla. Rotation equivariant orientation estimation for omnidirectional localization. In H. Ishikawa, C.-L. Liu, T. Pajdla, and J. Shi, editors,Com- puter Vision – ACCV 2020, pages 334–350, Cham, 2021. Springer International Publishing. ISBN 978-3-030-69538-5

    work page 2020

  47. [47]

    Habitat: A Platform for Embodied AI Research

    M. Savva, A. Kadian, O. Maksymets, Y . Zhao, E. Wijmans, B. Jain, J. Straub, J. Liu, V . Koltun, J. Malik, D. Parikh, and D. Batra. Habitat: A platform for embodied ai research, 2019. URL https://arxiv.org/abs/1904.01201

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  48. [48]

    S. K. Ramakrishnan, A. Gokaslan, E. Wijmans, O. Maksymets, A. Clegg, J. Turner, E. Un- dersander, W. Galuba, A. Westbury, A. X. Chang, M. Savva, Y . Zhao, and D. Batra. Habitat- matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai, 2021. URL https://arxiv.org/abs/2109.08238

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  49. [49]

    H. Liu, C. Li, Q. Wu, and Y . J. Lee. Visual instruction tuning, 2023. URLhttps://arxiv. org/abs/2304.08485

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  50. [50]

    X. Chen, Z. Wu, X. Liu, Z. Pan, W. Liu, Z. Xie, X. Yu, and C. Ruan. Janus-pro: Unified multimodal understanding and generation with data and model scaling, 2025. URLhttps: //arxiv.org/abs/2501.17811

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  51. [51]

    G. Team, A. Kamath, J. Ferret, S. Pathak, N. Vieillard, R. Merhej, S. Perrin, T. Matejovi- cova, A. Ram´e, M. Rivi`ere, L. Rouillard, T. Mesnard, G. Cideron, J. bastien Grill, S. Ramos, E. Yvinec, M. Casbon, E. Pot, I. Penchev, G. Liu, F. Visin, K. Kenealy, L. Beyer, X. Zhai, A. Tsitsulin, R. Busa-Fekete, A. Feng, N. Sachdeva, B. Coleman, Y . Gao, B. Must...

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  52. [52]

    M. Wu, X. Cai, J. Ji, J. Li, O. Huang, H. Fei, G. Jiang, X. Sun, and R. Ji. Controlmllm: Training-free visual prompt learning for multimodal large language models.Advances in Neu- ral Information Processing Systems, 37:45206–45234, 2024

    work page 2024

  53. [53]

    LLMind: Bio-inspired Training-free Adaptive Visual Representations for Vision-Language Models

    S. Debnath, B. D. Manh, Z. Liu, and L. Wang. Llmind: Bio-inspired training-free adaptive visual representations for vision-language models, 2026. URLhttps://arxiv.org/abs/ 2603.14882

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  54. [54]

    Debnath, B

    S. Debnath, B. D. Manh, Z. Liu, and L. Wang. Llmind: Bio-inspired training-free adaptive vi- sual representations for vision-language models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3133–3142, 2026

    work page 2026

  55. [55]

    Z. Liu, E. Zheng, S. Debnath, H. Shi, L. Xiao, and L. Wang. Vl2spike: Spike-driven distillation from vlms for low-power visual perception in embodied ai.arXiv preprint arXiv:2606.15898, 2026. 12

    work page arXiv 2026