pith. sign in

arxiv: 2605.21032 · v1 · pith:5WCBEM3Qnew · submitted 2026-05-20 · 💻 cs.CV

Towards Physically Consistent 4D Scene Reconstruction for Closed-loop Autonomous Driving Simulation

Bowyn Tan , Yutong Xie , Bai Huang , Fan Luo , Xiao Li , Naizheng Wang , Yang Guan , Shengbo Eben Li This is my paper

Pith reviewed 2026-05-21 05:43 UTC · model grok-4.3

classification 💻 cs.CV
keywords 4D scene reconstructionGaussian Splattingautonomous driving simulationnovel view synthesistemporal regularizationorthogonal projection
0
0 comments X

The pith

Orthogonal Projected Gradient secures spatial representations first to resolve null-space ambiguity in 4D scene reconstruction.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper identifies a credit assignment problem in existing 3D Gaussian Splatting methods and their 4D extensions, where the coupling of viewpoint and time in single observations creates low-rank structures that let temporal changes overwhelm spatial cues and cause spatial parameters to become unidentifiable. It introduces Orthogonal Projected Gradient as a hierarchical training approach that first locks in spatial integrity and then confines all temporal updates to the spatial null space, followed by a Temporal Regularization Strategy that adds a smoothness constraint drawn from the physical expectation of consistent appearance over time. This combination restores the ability to perform stable novel-view synthesis while modeling dynamics, producing scenes that remain physically consistent for closed-loop autonomous driving simulation.

Core claim

The core discovery is that the deterministic coupling between viewpoint and time in single-source data induces massive null-space ambiguity between static view-dependent and dynamic time-varying components; Orthogonal Projected Gradient restores spatial identifiability by securing spatial parameters in an initial stage and restricting subsequent temporal updates to the spatial null space, while the Temporal Regularization Strategy imposes a smoothness constraint based on the physical prior of consistent appearance evolution.

What carries the argument

Orthogonal Projected Gradient (OPG), a hierarchical training procedure that first secures spatial representations and then algebraically restricts temporal updates to the spatial null space.

If this is right

  • Stable novel-view synthesis becomes compatible with explicit modeling of temporal dynamics in the same representation.
  • Reconstructed scenes satisfy physical consistency priors, making them suitable for closed-loop simulation without drift in appearance over time.
  • Observation-reproducing metrics improve because temporal updates no longer degrade the underlying spatial structure.
  • Credit assignment between spatial and temporal parameters is performed proactively rather than reactively during optimization.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same hierarchical separation could be tested on multi-camera or multi-sensor driving datasets to check whether the null-space ambiguity shrinks when more viewpoints are available.
  • Replacing the appearance-evolution smoothness prior with other physical constraints such as rigid-body motion or lighting consistency might further tighten the temporal solution space.
  • The method's emphasis on algebraic isolation of updates suggests it could transfer to other dynamic reconstruction problems where spatial and temporal parameters compete for the same degrees of freedom.

Load-bearing premise

The assumption that single-source observations always produce a low-rank coupling between viewpoint and time that creates irresolvable ambiguity between static and dynamic scene components unless spatial parameters are secured first.

What would settle it

An experiment in which the orthogonal projection step is removed and spatial parameter estimation variance is measured across training; if the variance remains bounded and novel-view synthesis quality stays stable, the claimed necessity of the hierarchical separation would be contradicted.

Figures

Figures reproduced from arXiv: 2605.21032 by Bai Huang, Bowyn Tan, Fan Luo, Naizheng Wang, Shengbo Eben Li, Xiao Li, Yang Guan, Yutong Xie.

Figure 1
Figure 1. Figure 1 [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Spatiotemporal Credit Assignment under SOF. 4D reconstruction aims to recover the appearance surface c = F(d, t) (shorthand for ck). While multi-source observation allows perfect surface solving, SOF restricts observations to a 1D trajectory c = F(γ(t), t), necessitating proactive credit assignment via physical priors to infer the full surface manifold: (1) Handling Occlusions (B-C): During observation gap… view at source ↗
Figure 3
Figure 3. Figure 3: Information-Geometric Diagnostic. In SOF, the observed radiance c(γ(t), t) can be modeled within the span of temporal bases R(Jτ ). Unconstrained temporal component lead to solution ambiguity, while omitting them causes spatial underfitting. OPG ensures unique identifiability via proactive assignment. To handle observation gaps (cf [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Ablation study on temporal bases and model components. Top and bottom rows show results using Fourier and B-spline bases, respectively. In each row, the leftmost image displays the full model, followed by its ablated versions [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
read the original abstract

High-fidelity street scene reconstruction is pivotal for end-to-end autonomous driving simulation, where novel-view synthesis (NVS) and time-varying information modeling are two fundamental capabilities to facilitate closed-loop training. However, existing 3DGS methods and their 4D extensions fail to simultaneously achieve both. To bridge this gap, we establish an information-geometric diagnostic framework, revealing that this limitation stems from a credit assignment dilemma between spatial and temporal parameters. Specifically, the deterministic coupling between viewpoint and time in single-source observation creates a low-rank structure that induces massive null-space ambiguity between static view-dependent and dynamic time-varying components. Temporal information overshadows spatial cues, causing the estimation variance of spatial parameters to diverge. To address this issue, we propose Orthogonal Projected Gradient (OPG), a hierarchical training method designed to restore spatial identifiability. OPG prioritizes the integrity of spatial representations by securing them in an initial stage, then restricts temporal updates to the spatial null space, enabling proactive credit assignment. While OPG isolates temporal updates algebraically, Temporal Regularization Strategy is proposed to further refine the temporal solution space by imposing a smoothness constraint based on the physical prior of consistent appearance evolution, ensuring that the reconstructed scene remains physically consistent in closed-loop simulation. Extensive experiments demonstrate that our method not only maintains stable NVS capabilities but also demonstrates superior performance in traditional observation-reproducing metrics, which indirectly reflect the capability of modeling temporal dynamics.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript diagnoses a credit assignment dilemma in 3D Gaussian Splatting-based 4D reconstruction for street scenes: deterministic viewpoint-time coupling in single-source data creates a low-rank structure with null-space ambiguity between static view-dependent and dynamic time-varying components, allowing temporal signals to dominate and spatial parameter variance to diverge. It proposes Orthogonal Projected Gradient (OPG) as a hierarchical optimizer that first secures spatial representations and then projects temporal gradients onto the spatial null space, followed by a Temporal Regularization Strategy that imposes a physical smoothness prior on appearance evolution. Experiments are reported to show preserved novel-view synthesis quality alongside improved performance on observation-reproducing metrics that indirectly indicate better temporal modeling for closed-loop autonomous driving simulation.

Significance. If the OPG projection remains effective and the claimed isolation of spatial and temporal credit assignment holds, the work would offer a practical route to physically consistent 4D reconstructions from monocular driving sequences. The information-geometric framing and explicit use of a physical prior distinguish it from purely data-driven 4D extensions of 3DGS and could inform future simulation pipelines that require stable geometry under viewpoint and time variation.

major comments (2)
  1. [OPG Method] The central technical claim of OPG—that temporal updates can be algebraically restricted to the spatial null space after an initial spatial-securing stage—rests on the assumption that this projection remains stable. In §3 (OPG description) the rendering map from 3DGS parameters (means, covariances, SH coefficients, opacities) to pixels is nonlinear and the overall loss is non-convex; therefore the linear-algebraic null-space argument does not automatically guarantee that subsequent gradient steps preserve the isolation. A stability analysis, drift bound, or ablation that measures spatial-parameter variance before and after the projection stage is required to substantiate the proactive credit assignment.
  2. [Temporal Regularization Strategy] The manuscript states that the Temporal Regularization Strategy further refines the temporal solution space via a smoothness constraint derived from consistent appearance evolution. However, no derivation or explicit loss term is supplied that shows how this prior interacts with the OPG projection without re-introducing spatial contamination. If the regularization is applied after the projection, its effect on the already-isolated temporal subspace should be quantified (e.g., via an ablation that disables the prior while keeping OPG).
minor comments (2)
  1. Notation for the spatial null-space projector (e.g., the orthogonal complement operator) should be introduced with a short equation block so that the projection step can be reproduced from the text alone.
  2. Figure captions for the qualitative results should explicitly label which rows correspond to OPG-only versus OPG+regularization to allow direct visual assessment of each component.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments on our work. We address each major comment below and have revised the manuscript accordingly to strengthen the technical claims.

read point-by-point responses

  1. Referee: [OPG Method] The central technical claim of OPG—that temporal updates can be algebraically restricted to the spatial null space after an initial spatial-securing stage—rests on the assumption that this projection remains stable. In §3 (OPG description) the rendering map from 3DGS parameters (means, covariances, SH coefficients, opacities) to pixels is nonlinear and the overall loss is non-convex; therefore the linear-algebraic null-space argument does not automatically guarantee that subsequent gradient steps preserve the isolation. A stability analysis, drift bound, or ablation that measures spatial-parameter variance before and after the projection stage is required to substantiate the proactive credit assignment.

    Authors: We acknowledge that the nonlinearity of the rendering function and non-convexity of the loss imply that the algebraic projection alone does not provide a strict theoretical guarantee of isolation across all optimization steps. In practice, our initial spatial-securing stage followed by repeated projection reduces spatial variance, as indirectly supported by maintained novel-view synthesis quality. To substantiate this, we will add an ablation measuring spatial-parameter variance (e.g., on means and covariances) before and after the projection stage, along with a short discussion of observed empirical stability. revision: yes

  2. Referee: [Temporal Regularization Strategy] The manuscript states that the Temporal Regularization Strategy further refines the temporal solution space via a smoothness constraint derived from consistent appearance evolution. However, no derivation or explicit loss term is supplied that shows how this prior interacts with the OPG projection without re-introducing spatial contamination. If the regularization is applied after the projection, its effect on the already-isolated temporal subspace should be quantified (e.g., via an ablation that disables the prior while keeping OPG).

    Authors: We agree that an explicit derivation and loss term would better demonstrate the interaction. The smoothness prior is applied exclusively to temporal parameters after each OPG projection step, preserving the spatial null-space isolation. In the revised manuscript we will include the explicit loss formulation and an ablation that disables the regularization while retaining OPG, quantifying its effect on temporal consistency metrics without degrading spatial representations. revision: yes

Circularity Check

0 steps flagged

No circularity: derivation rests on independent geometric analysis and external physical prior

full rationale

The paper first establishes an information-geometric diagnostic framework from the deterministic viewpoint-time coupling in single-source observations, identifying low-rank structure and null-space ambiguity. It then introduces OPG as a hierarchical training procedure that secures spatial parameters initially and projects temporal updates onto the spatial null space. The Temporal Regularization Strategy adds a smoothness constraint drawn from the stated physical prior of consistent appearance evolution. None of these steps reduce the target result to a fitted parameter or self-citation defined inside the same equations; the diagnostic and algorithmic choices remain externally grounded rather than self-referential.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Based on abstract only; the central claim rests on one domain assumption about physical appearance evolution and on the geometric claim of low-rank coupling. No explicit free parameters or new invented entities are named.

axioms (1)
  • domain assumption Physical prior of consistent appearance evolution
    Invoked to justify the smoothness constraint in the Temporal Regularization Strategy.

pith-pipeline@v0.9.0 · 5811 in / 1349 out tokens · 43568 ms · 2026-05-21T05:43:43.002155+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

37 extracted references · 37 canonical work pages · 2 internal anchors

  1. [1]

    SimScale: Learning to Drive via Real-World Simulation at Scale

    Haochen Tian, Tianyu Li, Haochen Liu, Jiazhi Yang, Yihang Qiu, Guang Li, Junli Wang, Yinfeng Gao, Zhang Zhang, Liang Wang, et al. Simscale: Learning to drive via real-world simulation at scale.arXiv preprint arXiv:2511.23369, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  2. [2]

    Carplanner: Consistent auto-regressive trajectory planning for large-scale reinforcement learning in autonomous driving

    Dongkun Zhang, Jiaming Liang, Ke Guo, Sha Lu, Qi Wang, Rong Xiong, Zhenwei Miao, and Yue Wang. Carplanner: Consistent auto-regressive trajectory planning for large-scale reinforcement learning in autonomous driving. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 17239–17248, 2025

    work page 2025

  3. [3]

    Hgsim: High-fidelity and generalizable simulation frame-work for autonomous driving scenes.Neurocomputing, page 131784, 2025

    Yue Tian, Wenbo Chu, Wei Zhou, Xiaolin Tang, and Keqiang Li. Hgsim: High-fidelity and generalizable simulation frame-work for autonomous driving scenes.Neurocomputing, page 131784, 2025

    work page 2025

  4. [4]

    Model-based imitation learning for urban driving.Advances in Neural Information Processing Systems, 35:20703–20716, 2022

    Anthony Hu, Gianluca Corrado, Nicolas Griffiths, Zachary Murez, Corina Gurau, Hudson Yeo, Alex Kendall, Roberto Cipolla, and Jamie Shotton. Model-based imitation learning for urban driving.Advances in Neural Information Processing Systems, 35:20703–20716, 2022

    work page 2022

  5. [5]

    Sem2: Enhance sample efficiency and robustness of end-to-end urban autonomous driving via semantic masked world model

    Zeyu Gao, Yao Mu, Ruoyan Shen, Chen Chen, Yangang Ren, Jianyu Chen, Shengbo Eben Li, Ping Luo, and Yanfeng Lu. Sem2: Enhance sample efficiency and robustness of end-to-end urban autonomous driving via semantic masked world model. InDeep Reinforcement Learning Workshop NeurIPS 2022, 2022

    work page 2022

  6. [6]

    Hydra-MDP: End-to-end Multimodal Planning with Multi-target Hydra-Distillation

    Zhenxin Li, Kailin Li, Shihao Wang, Shiyi Lan, Zhiding Yu, Yishen Ji, Zhiqi Li, Ziyue Zhu, Jan Kautz, Zuxuan Wu, et al. Hydra-mdp: End-to-end multimodal planning with multi-target hydra-distillation.arXiv preprint arXiv:2406.06978, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  7. [7]

    Drivinggaussian: Composite gaussian splatting for surrounding dynamic autonomous driving scenes

    Xiaoyu Zhou, Zhiwei Lin, Xiaojun Shan, Yongtao Wang, Deqing Sun, and Ming-Hsuan Yang. Drivinggaussian: Composite gaussian splatting for surrounding dynamic autonomous driving scenes. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 21634–21643, 2024

    work page 2024

  8. [8]

    Protocar: Learning 3d vehicle prototypes from single-view and unconstrained driving scene images

    Hongyuan Liu, Haochen Yu, Bochao Zou, Juntao Lyu, Qi Mei, Jiansheng Chen, and Huimin Ma. Protocar: Learning 3d vehicle prototypes from single-view and unconstrained driving scene images. InProceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 5460–5468, 2025

    work page 2025

  9. [9]

    Hugs: Holistic urban 3d scene understanding via gaussian splatting

    Hongyu Zhou, Jiahao Shao, Lu Xu, Dongfeng Bai, Weichao Qiu, Bingbing Liu, Yue Wang, Andreas Geiger, and Yiyi Liao. Hugs: Holistic urban 3d scene understanding via gaussian splatting. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21336–21345, 2024

    work page 2024

  10. [10]

    Street gaussians: Modeling dynamic urban scenes with gaussian splatting

    Yunzhi Yan, Haotong Lin, Chenxu Zhou, Weijie Wang, Haiyang Sun, Kun Zhan, Xianpeng Lang, Xiaowei Zhou, and Sida Peng. Street gaussians: Modeling dynamic urban scenes with gaussian splatting. InEuropean Conference on Computer Vision, pages 156–173. Springer, 2024

    work page 2024

  11. [11]

    Splatflow: Self- supervised dynamic gaussian splatting in neural motion flow field for autonomous driving

    Su Sun, Cheng Zhao, Zhuoyang Sun, Yingjie Victor Chen, and Mei Chen. Splatflow: Self- supervised dynamic gaussian splatting in neural motion flow field for autonomous driving. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 27487–27496, 2025. 10

    work page 2025

  12. [12]

    Coda-4dgs: Dynamic gaussian splatting with context and deformation awareness for autonomous driving

    Rui Song, Chenwei Liang, Yan Xia, Walter Zimmer, Hu Cao, Holger Caesar, Andreas Festag, and Alois Knoll. Coda-4dgs: Dynamic gaussian splatting with context and deformation awareness for autonomous driving. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 28031–28041, 2025

    work page 2025

  13. [13]

    Neural scene graphs for dynamic scenes

    Julian Ost, Fahim Mannan, Nils Thuerey, Julian Knodt, and Felix Heide. Neural scene graphs for dynamic scenes. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2856–2865, 2021

    work page 2021

  14. [14]

    S3Gaussian: Self-supervised street gaussians for autonomous driving.arXiv preprint arXiv:2405.20323, 2024

    Nan Huang, Xiaobao Wei, Wenzhao Zheng, Pengju An, Ming Lu, Wei Zhan, Masayoshi Tomizuka, Kurt Keutzer, and Shanghang Zhang. S3Gaussian: Self-supervised street gaussians for autonomous driving.arXiv preprint arXiv:2405.20323, 2024

    work page arXiv 2024

  15. [15]

    Periodic vibration gaussian: Dynamic urban scene reconstruction and real-time rendering.International Journal of Computer Vision, 134(3):83, 2026

    Yurui Chen, Chun Gu, Junzhe Jiang, Xiatian Zhu, and Li Zhang. Periodic vibration gaussian: Dynamic urban scene reconstruction and real-time rendering.International Journal of Computer Vision, 134(3):83, 2026

    work page 2026

  16. [16]

    Mtgs: Multi-traversal gaussian splatting.arXiv preprint arXiv:2503.12552, 2025

    Tianyu Li, Yihang Qiu, Zhenhua Wu, Carl Lindström, Peng Su, Matthias Nießner, and Hongyang Li. Mtgs: Multi-traversal gaussian splatting.arXiv preprint arXiv:2503.12552, 2025

    work page arXiv 2025

  17. [17]

    3d gaussian splatting for real-time radiance field rendering.ACM Trans

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, George Drettakis, et al. 3d gaussian splatting for real-time radiance field rendering.ACM Trans. Graph., 42(4):139–1, 2023

    work page 2023

  18. [18]

    Colmap- free 3d gaussian splatting

    Yang Fu, Sifei Liu, Amey Kulkarni, Jan Kautz, Alexei A Efros, and Xiaolong Wang. Colmap- free 3d gaussian splatting. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 20796–20805, 2024

    work page 2024

  19. [19]

    4d gaussian splatting for real-time dynamic scene rendering

    Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4d gaussian splatting for real-time dynamic scene rendering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 20310–20320, 2024

    work page 2024

  20. [20]

    Mega: Memory-efficient 4d gaussian splatting for dynamic scenes

    Xinjie Zhang, Zhening Liu, Yifan Zhang, Xingtong Ge, Dailan He, Tongda Xu, Yan Wang, Zehong Lin, Shuicheng Yan, and Jun Zhang. Mega: Memory-efficient 4d gaussian splatting for dynamic scenes. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 27828–27838, 2025

    work page 2025

  21. [21]

    K-planes: Explicit radiance fields in space, time, and appearance

    Sara Fridovich-Keil, Giacomo Meanti, Frederik Rahbæk Warburg, Benjamin Recht, and Angjoo Kanazawa. K-planes: Explicit radiance fields in space, time, and appearance. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12479–12488, 2023

    work page 2023

  22. [22]

    V4d: V oxel for 4d novel view synthesis.IEEE Transactions on Visualization and Computer Graphics, 30(2):1579– 1591, 2023

    Wanshui Gan, Hongbin Xu, Yi Huang, Shifeng Chen, and Naoto Yokoya. V4d: V oxel for 4d novel view synthesis.IEEE Transactions on Visualization and Computer Graphics, 30(2):1579– 1591, 2023

    work page 2023

  23. [23]

    High- fidelity and real-time novel view synthesis for dynamic scenes

    Haotong Lin, Sida Peng, Zhen Xu, Tao Xie, Xingyi He, Hujun Bao, and Xiaowei Zhou. High- fidelity and real-time novel view synthesis for dynamic scenes. InSIGGRAPH Asia 2023 Conference Papers, pages 1–9, 2023

    work page 2023

  24. [24]

    Real-time photorealistic dynamic scene rep- resentation and rendering with 4D gaussian splatting

    Zeyu Yang, Hongye Yang, Zijie Pan, and Li Zhang. Real-time photorealistic dynamic scene rep- resentation and rendering with 4D gaussian splatting. InThe Twelfth International Conference on Learning Representations (ICLR), 2024

    work page 2024

  25. [25]

    Spacetime gaussian feature splatting for real-time dynamic view synthesis

    Zhan Li, Zhang Chen, Zhong Li, and Yi Xu. Spacetime gaussian feature splatting for real-time dynamic view synthesis. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8508–8520, 2024

    work page 2024

  26. [26]

    Evolsplat4d: Efficient volume-based gaussian splatting for 4d urban scene synthesis

    Sheng Miao, Sijin Li, Pan Wang, Dongfeng Bai, Bingbing Liu, Yue Wang, Andreas Geiger, and Yiyi Liao. Evolsplat4d: Efficient volume-based gaussian splatting for 4d urban scene synthesis. arXiv preprint arXiv:2601.15951, 2026

    work page arXiv 2026

  27. [27]

    Springer Science & Business Media, 2012

    Shun-ichi Amari.Differential-geometrical methods in statistics. Springer Science & Business Media, 2012. 11

    work page 2012

  28. [28]

    Theory of statistical estimation

    Ronald Aylmer Fisher. Theory of statistical estimation. InMathematical proceedings of the Cambridge philosophical society, volume 22, pages 700–725. Cambridge University Press, 1925

    work page 1925

  29. [29]

    Information and the accuracy attainable in the estimation of statistical parameters.Bull

    C Radhakrishna Rao et al. Information and the accuracy attainable in the estimation of statistical parameters.Bull. Calcutta Math. Soc, 37(3):81–91, 1945

    work page 1945

  30. [30]

    Scalability in perception for autonomous driving: Waymo open dataset

    Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, et al. Scalability in perception for autonomous driving: Waymo open dataset. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2446–2454, 2020

    work page 2020

  31. [31]

    EmerNeRF: Emergent spatial- temporal scene decomposition via self-supervision

    Jiawei Yang, Boris Ivanovic, Or Litany, Xinshuo Weng, Seung Wook Kim, Boyi Li, Tong Che, Danfei Xu, Sanja Fidler, Marco Pavone, and Yue Wang. EmerNeRF: Emergent spatial- temporal scene decomposition via self-supervision. InInternational Conference on Learning Representations, 2024. A Detailed formulation of 3D Gaussian Splatting This section provides a co...

    work page 2024

  32. [32]

    Each temporal gradient component is a modulation of the basisϕ n(t)by the aggregate spatial signalB(t)

    Temporal gradient formulationApplying the chain rule to the total color C with respect to the n-th temporal coefficientτ n: g(k) τ,n(t) = ∂C ∂τ (k) n =ω k ∂ck(t,d(t)) ∂τ (k) n =ω k    X l,m s(k) lm Y m l (d(t))   ϕn(t)   =ϕ n(t)·  ωk X l,m s(k) lm Y m l (d(t))   | {z } B(t) (25) where B(t) represents the total projected spatial contribution of...

    work page

  33. [33]

    Spatial gradient formulationFor a specific spatial coefficient s(k) lm of the k-th Gaussian, the gradient ofCis: g(k) s,lm(t) = ∂C ∂s(k) lm =ω k ∂ck(t,d(t)) ∂s(k) lm =ω k " Y m l (d(t))· X n τ (k) n ϕn(t) # =ω kY m l (d(t))·T(t) (26) whereT(t) = P n τ (k) n ϕn(t)is the shared temporal modulation function

    work page

  34. [34]

    True Scene Parameter

    Proof of subspace inclusionWe seek a set of coefficients {γn} such that the spatial gradient g(k) s,lm(t)is a linear combination of the temporal gradients{g τ,n(t)}. This requires: ωkY m l (d(t))T(t) = X n γn[B(t)ϕn(t)] ωkY m l (d(t))T(t) B(t) = X n γnϕn(t) (27) Since d(t) is a continuous trajectory, the term H(t) = ωkY m l (d(t))T(t) B(t) is a well-defin...

    work page

  35. [35]

    Orthogonality of the purified Jacobian.The OPG scheme defines the purified temporal Jacobian as ˜Jτ =P ⊥ s Jτ , where P⊥ s =I−J s(J⊤ s Js)−1J⊤ s is the projector onto the null-space of Js. We first show thatJ s and ˜Jτ are strictly orthogonal: J⊤ s ˜Jτ =J ⊤ s (I−J s(J⊤ s Js)−1J⊤ s )Jτ (63) = (J⊤ s −J ⊤ s Js(J⊤ s Js)−1J⊤ s )Jτ (64) = (J⊤ s −J ⊤ s )Jτ =0(65...

    work page

  36. [36]

    Block-diagonalization of the FIM.The joint FIM under OPG is constructed as: FOP G = 1 σ2 Js ˜Jτ ⊤ Js ˜Jτ = 1 σ2 J⊤ s Js J⊤ s ˜Jτ ˜J⊤ τ Js ˜J⊤ τ ˜Jτ (66) Substituting the orthogonality resultJ ⊤ s ˜Jτ =0, we obtain a block-diagonal matrix: FOP G = 1 σ2 J⊤ s Js 0 0 ˜J⊤ τ ˜Jτ = Fss 0 0 F ˜τ˜τ (67)

    work page

  37. [37]

    Derivation of the decoupled CRB and temporal variance.For a block-diagonal FIM, the Schur complements simplify significantly. The effective information for the spatial parameterssis: Ss =F ss −F s˜τF−1 ˜τ˜τF˜τ s=F ss −0=F ss.(68) The resulting lower bound for the estimation covariance is strictly bounded: Cov(ˆs)⪰S −1 s =σ 2(J⊤ s Js)−1.(69) This confirms ...

    work page