Pith. sign in

REVIEW 2 major objections 6 minor 3 cited by

STORM: Spatio-Temporal Reconstruction Model for Large-Scale Outdoor Scenes

T0 review · 2 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read One feed-forward pass can rebuild a moving street scene from sparse camera frames in 0.2 seconds, with motion estimated from reconstruction losses alone.

desk verdict Real feed-forward dynamic 3DGS result with strong numbers, but the 'self-supervised' claim doesn't survive the appendix: external depth and sky masks are in the training loss. read the letter →

arxiv 2501.00602 v1 pith:TZNS5JZO submitted 2024-12-31 cs.CV cs.LG

Jiawei Yang , Jiahui Huang , Yuxiao Chen , Yan Wang , Boyi Li , Yurong You , Apoorva Sharma , Maximilian Igl
show 5 more authors
Peter Karkus Danfei Xu Boris Ivanovic Yue Wang Marco Pavone
This is my paper · ORCID
classification cs.CVcs.LG
keywords dynamicscenereconstruction3DGaussiansplattingfeed-forwardflowestimationself-supervisedlearningmotionsegmentationautonomousdrivingnovelviewsynthesis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Normally, reconstructing a dynamic outdoor scene from sparse cameras means fitting every scene separately for minutes to hours, often guided by masks, optical flow, or LiDAR. STORM attempts to replace that with one feed-forward pass: given four posed frames from a two-second clip, a Transformer emits a 3D Gaussian cloud for each frame and a velocity for every Gaussian. Those velocities carry the clouds to a common time, where they merge into an amodal scene (one that includes regions no single frame saw), and the only training signal is the error between rendered and observed images. If the paper's results hold, dynamic street scenes can be reconstructed in about 0.2 seconds per clip, their motion estimated without LiDAR or pseudo-labels, and the learned motion tokens can even segment moving objects for editing.

What carries the argument

The load-bearing object is the per-frame, pixel-aligned 3D Gaussian Splat: a cloud of anisotropic 3D Gaussians, one per image patch, each carrying center, rotation, scale, opacity, color, and forward/backward velocity vectors. Motion is transported by Eq. (1), a constant-velocity translation, and Eq. (2) takes the union of all transported clouds at the target time, yielding the amodal representation that is rendered and supervised. To keep motion low-dimensional, $M=16$ learnable motion tokens are prepended to the Transformer; a mask decoder computes softmax weights over $M$ velocity bases, so every Gaussian's velocity is a convex combination of shared motion primitives (Eqs. (3)--(4)). The training loss is reconstruction, sky-opacity, and velocity-regularization only; no optical flow, masks, trajectories, or LiDAR enter the loss.

What would settle it

On clips with hard braking or sharp turns, measure object acceleration from LiDAR ground truth and compare STORM's camera-only scene flow against that ground truth separately for high-acceleration and low-acceleration objects; if constant velocity is load-bearing, EPE3D should rise with acceleration and dynamic-region PSNR should drop on the accelerating frames.

Watch

Extended reading notes

Core claim

The central claim is that dynamic reconstruction and scene-flow estimation are one self-supervised problem: if the per-frame 3D Gaussians are carried to a target timestep by the correct velocities, the union of all transported clouds must re-render the observed frames, so reconstruction error alone can supervise motion. On the Waymo Open Dataset, STORM reports dynamic-region PSNR of 22.10, surpassing per-scene optimization methods by 4.3 to 6.6 dB and feed-forward baselines by 2.1 to 4.7 dB, with 0.18 s inference per clip; it also reports camera-only scene flow with 0.276 m EPE3D and 81.12% Acc5, above LiDAR-based baselines on the same metrics. An emergent property, the weights that combine shared motion bases form instance-like masks with no segmentation labels, and the latent variant reconstructs fine human motion and edits scenes by adding or removing Gaussians.

Load-bearing premise

The whole pipeline assumes each 3D point glides at constant velocity for the roughly two-second clip (Eq. (1)), so vehicles that brake or swerve and humans who swing limbs are treated as piecewise-constant translations, and the reported reconstruction and flow gains inherit that assumption.

Editorial extensions

If this is right

  • Dynamic outdoor scenes can be reconstructed in about 0.2 s per two-second clip on a single A100 GPU, versus tens of minutes for per-scene fitting, making onboard or interactive use possible.
  • Scene flow is obtainable from cameras alone and is reported to beat LiDAR-fed flow methods in EPE3D and Acc5, so depth sensors would not be needed at test time.
  • Motion-token assignments give unsupervised motion masks, so objects can be removed, cloned, replaced, or added directly in the recovered Gaussian scene.
  • Because the model is a sequence-to-sequence Transformer, it transfers to fewer or more input timesteps without retraining and can be run iteratively to cover 20-second videos in under a second.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because Eq. (1) fixes velocity for the whole clip, the approach is likely to degrade on hard braking, sharp turns, or articulated motion; a variant that predicts acceleration or piecewise trajectories would be the natural stress test.
  • With only $M=16$ shared motion bases, we would expect STORM to merge or drop distinct movers when a single clip contains more than about sixteen independently moving groups.
  • If this model scales like other feed-forward reconstruction Transformers, dynamic-region PSNR should improve monotonically with training-set size; a training-fraction sweep would settle that, and the paper does not run it.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper presents STORM, a feed-forward Transformer that takes sparse, posed, multi-view images at several timesteps and outputs per-frame 3D Gaussians plus velocity bases. Predicted Gaussians are transported to a target timestep using piecewise-constant velocities and aggregated into an amodal representation; training combines color, depth, perceptual, sky, and velocity-regularization losses. Experiments on Waymo Open, NuScenes, and Argoverse2 report higher dynamic-region PSNR/SSIM than per-scene optimization and feed-forward baselines, lower depth RMSE, and improved scene-flow EPE3D/Acc5 at 0.18s inference. The paper also claims self-supervision and emergent motion masks using only reconstruction losses.

Significance. If the results withstand scrutiny, STORM is a meaningful advance: it is the first feed-forward model to reconstruct dynamic outdoor scenes and estimate camera-only scene flow in a single pass, with strong cross-dataset generalization and real-time speed. The use of public benchmarks, the ablations for velocity regularization, motion-token count, and input timesteps, and the conservative comparison against LiDAR-based flow baselines are notable strengths. The central 'self-supervised / only reconstruction losses' claim, however, is contradicted by the actual loss function, which includes externally provided depth maps and pre-trained sky masks. The significance of the emergent segmentation and scene-flow results therefore depends on additional ablations that separate those external cues from pure reconstruction.

major comments (2)
  1. [Abstract, §3.3 Eq. (7), Appendix A.2 Eqs. (A2)-(A3), Appendix B.2] The paper repeatedly claims self-supervision 'using only reconstruction losses' (abstract and §3.1), but Eq. (A2) includes a depth term ||(Dhat-D)/max(D)||_1 against observed depth maps D, and Eq. (A3) includes Lsky = ||Ohat-(1-M)||_1 with M supplied by a pre-trained segmentation model. These are external geometric and semantic supervision signals, not reconstruction losses. Since Appendix A.3 states that all models are trained with 'the same color, depth, perceptual and sky supervision,' the reported gains and the 'emergent' motion-mask and scene-flow claims are confounded with supervised depth and sky cues. Please add an ablation that removes the depth term and the sky term (or replaces them with self-supervised monocular estimates) and reports dynamic-region PSNR/SSIM and EPE3D/Acc5. Without such an ablation, the central self-supervision claim is not established.
  2. [§3.1 Eq. (1), §4.2] The motion model assumes piecewise constant velocity over the clip, as stated in Eq. (1): mu_{t->t'} = mu_t +/- (t'-t)v_t. This assumption is described as an empirical finding but is not ablated; no experiment considers accelerated motion, turning, or non-rigid deformation. Because the amodal aggregation in Eq. (2) and the scene-flow evaluation in §4.2 both build on Eq. (1), the reported dynamic-region PSNR and EPE3D gains may partly reflect a favorable test distribution (gentle longitudinal motion over 0.5s intervals) rather than a general motion model. Please add an evaluation on high-acceleration or turning segments, or compare against a quadratic or rigid-body motion variant, to quantify the impact of this assumption. At minimum, the limitation should be stated explicitly in the main text rather than only in passing.
minor comments (6)
  1. [Appendix B.2, Figures B.1 and B.2] Figures B.1 and B.2 each appear twice in the compiled PDF with identical content; please remove the duplicates.
  2. [§4.1] The definition of 'dynamic regions' is not specified; state whether the evaluation mask comes from ground-truth bounding boxes, from the motion-token assignments, or from an external detector.
  3. [§4.2] For the scene-flow evaluation, clarify whether EPE3D/Acc5 are computed at 0.5s intervals and how the ground-truth flows are interpolated to the 1st, 5th, 10th, and 15th context frames; also state which STORM variant (STORM or Latent-STORM) produces the 'Ours' row in Table 3.
  4. [Appendix A.2, Eq. (A2)] The denominator max(D) in the depth loss is ambiguous: is it a per-image maximum, a per-batch maximum, or a global constant? Please clarify and add a small epsilon for numerical stability.
  5. [Eq. (1)] The sign convention in Eq. (1) is stated as '+/-'; consider writing the backward and forward cases with explicit v^- and v^+ to avoid confusion with the two-velocity representation introduced earlier.
  6. [§3.2, Figures 2 and 3] The sky token, affine token, and latent decoder are evaluated only qualitatively; a small quantitative table (e.g., PSNR/SSIM with and without each component) would strengthen the claims in §3.2.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation: STORM's motion and rendering claims are evaluated against held-out data, though the 'self-supervised / only reconstruction losses' wording is overstated by external depth and sky-mask supervision.

full rationale

I walked the paper's derivation chain from Eq. (1)-(4) through the loss definitions in Eq. (A1)-(A3) and the evaluations in Tables 1-3. The predicted velocities are not constructed from the reported evaluation targets: they are produced by a Transformer from posed images, combined through the amodal aggregation in Eq. (2), and optimized only by the rendering, depth, sky, and velocity-regularization losses. Benchmarks on Waymo, NuScenes, and Argoverse2 are held-out public data, so the rendering and scene-flow numbers are independent of the training objective. No load-bearing self-citation chain is present: EmerNeRF and OmniRE are baselines or related work, not justifications of STORM's central representation. The constant-velocity assumption in Eq. (1) is an unablated modeling choice, but it is not a circular reduction because the model's outputs are not defined in terms of the evaluation quantities. The one flagged issue is an overclaim rather than circularity: Eq. (A2) includes a depth term against observed depth maps D and Eq. (A3) includes a sky term against a pre-trained mask M, while the abstract claims masks emerge 'using only reconstruction losses' and the paper describes the method as 'self-supervised.' No ablation removes D and M, so the self-supervision claim lacks support; however, no fitted parameter is renamed as a prediction and no output equals an input by construction. Score 1 reflects this minor claim-support inconsistency, not circularity.

Assumptions & free parameters 6 free parameters · 4 assumptions · 3 invented entities

The method rests on standard learned-representation assumptions plus hyperparameters tuned by ablation; no new physical entities are postulated. The main unstated load is external depth and sky-mask supervision, which the main text does not disclose, and the constant-velocity motion model.

free parameters (6)
  • velocity regularization weight lambda_reg = 5e-3
    Ablation Fig. B.1(a) shows optimal dynamic PSNR at 5e-3; required to prevent gradient explosion, so the flow and reconstruction results depend on this hand-set coefficient.
  • temperature tau in soft assignment = 0.5
    Controls sharpness of the soft assignment weights in Eq. (3); fixed in all experiments.
  • number of motion tokens M = 16
    Ablation Fig. B.1(b) finds M=16 best for rendering and flow; central to the low-rank motion representation.
  • sky loss weight lambda_sky = 0.1
    Balances the sky sparsity loss; chosen by hand and used in all experiments.
  • LPIPS and depth loss weights = 0.05 for LPIPS, 1.0 for depth
    Loss balance in Eq. (A2); no sensitivity analysis is reported.
  • near/far depth bounds = 0.1 / 400.0
    Defines the depth range for the sigmoid mapping; affects geometry and sky modeling, chosen for driving scenes.
assumptions (4)
  • domain assumption Each 3D Gaussian moves with constant velocity over the 2-second clip (Eq. 1).
    Load-bearing for amodal aggregation and scene flow; stated as an empirical finding in Section 3.1, not proven or extensively ablated.
  • domain assumption Ground-truth depth maps D and pre-trained sky masks M are available for training (Appendix A.2).
    Underlies the claim of self-supervision; the main text emphasizes camera images only and does not disclose these supervision signals.
  • domain assumption Renderer and losses (RGB, depth, LPIPS) provide gradients sufficient to learn geometry and motion without explicit optical flow or masks.
    The core learning hypothesis; supported indirectly by ablations but not formally established.
  • domain assumption A Transformer with patch embedding of Plucker ray maps generalizes from Waymo to NuScenes and Argoverse2 with unchanged hyperparameters.
    Cross-dataset results in Table 2 support this, but no failure analysis for domain shift is given.
invented entities (3)
  • Motion tokens with velocity bases
    purpose: Learnable tokens that interact with image tokens via self-attention and decode into shared velocity bases to regularize per-Gaussian motion and enable motion group segmentation.
    Internal network components; evidence is the reported behavior (masks, flow accuracy) but no independent falsifiable handle outside the trained model.
  • Sky token and affine tokens
    purpose: Auxiliary input tokens that condition sky color prediction and per-camera exposure correction.
    Internal model components; their effect is shown in qualitative examples only.
  • Latent Gaussians with inpainting token
    purpose: Patch-aligned latent-coded Gaussians and an inpainting token that let a decoder synthesize occluded or extrapolated regions and fine human motion.
    Internal representation; qualitative claims only, with no quantitative evaluation for hallucination quality.

how reviews work

0 comments
Cite this review

Pith. "Pith review of STORM: Spatio-Temporal Reconstruction Model for Large-Scale Outdoor Scenes." pith.science (2026). https://pith.science/paper/TZNS5JZO

@misc{pith2026250100602,
  author       = {Pith},
  title        = {Pith review of: STORM: Spatio-Temporal Reconstruction Model for Large-Scale Outdoor Scenes},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TZNS5JZO}},
  note         = {Machine review of arXiv:2501.00602}
}
read the original abstract

We present STORM, a spatio-temporal reconstruction model designed for reconstructing dynamic outdoor scenes from sparse observations. Existing dynamic reconstruction methods often rely on per-scene optimization, dense observations across space and time, and strong motion supervision, resulting in lengthy optimization times, limited generalization to novel views or scenes, and degenerated quality caused by noisy pseudo-labels for dynamics. To address these challenges, STORM leverages a data-driven Transformer architecture that directly infers dynamic 3D scene representations--parameterized by 3D Gaussians and their velocities--in a single forward pass. Our key design is to aggregate 3D Gaussians from all frames using self-supervised scene flows, transforming them to the target timestep to enable complete (i.e., "amodal") reconstructions from arbitrary viewpoints at any moment in time. As an emergent property, STORM automatically captures dynamic instances and generates high-quality masks using only reconstruction losses. Extensive experiments on public datasets show that STORM achieves precise dynamic scene reconstruction, surpassing state-of-the-art per-scene optimization methods (+4.3 to 6.6 PSNR) and existing feed-forward approaches (+2.1 to 4.7 PSNR) in dynamic regions. STORM reconstructs large-scale outdoor scenes in 200ms, supports real-time rendering, and outperforms competitors in scene flow estimation, improving 3D EPE by 0.422m and Acc5 by 28.02%. Beyond reconstruction, we showcase four additional applications of our model, illustrating the potential of self-supervised learning for broader dynamic scene understanding.

Figures

Figures reproduced from arXiv: 2501.00602 by the authors.

Figure 1
Figure 1. STORM Overview. From sparsely observed context frames, STORM reconstructs per-frame 3D Gaussian splats (3DGS) and predicts their scene flows using prepended learnable motion tokens and a dynamic mask decoder. The mask decoder computes weights for combining motion bases, derived from the motion tokens, to obtain scene flows. These predicted scene flows enable the ag￾gregation and transformation of 3DGS over time, whi… view at source ↗
Figure 2
Figure 2. Effect of affine and sky tokens. (a) The affine token handles exposure mismatches between cameras, eliminating artifacts like the black foggy floaters caused by exposure differences (orange arrows). (b) The sky token enables our method to predict sky colors for every pixel during rendering, even when they are not observed in any context frames. the Gaussians Gt ′ at an arbitrary target timestep t ′ are defined as th… view at source ↗
Figure 3
Figure 3. Latent-STORM Examples. Using latent Gaussians and a decoder, Latent-STORM can photorealistically reconstruct human leg movements. Note how the leg angles change over time in Latent-STORM, while they remain static in regular STORM (blue arrows). Please refer to “Human Modeling with Latent-STORM” section in our project page for video comparisons. Auxiliary tokens for sky and exposure mismatches. In in-the-wild video c… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Iterative reconstruction of static scenes. STORM reconstructs 20-second-long videos within 1 second in an iterative manner, which can serve as initialization for per-scene optimization methods for further refinement. (a) Sample-1, Viewer results (b) Sample-2, Viewer re…
Figure 5
Figure 5. Figure 5: Iterative reconstruction of dynamic scenes. Top: STORM reconstructs 20-second-long videos within 1 second in an iterative manner. Bottom: Furthermore, by chaining scene flows, we obtain point trajectories for dynamic Gaussians. mation with dense space-time observations…
Figure 6
Figure 6. Figure 6: Self-supervised scene flow estimation and motion segmentation. For each sample, we show the rendered camera images (top), scene flows (middle), and motion assignments (bottom). (a) Sample-1, original (b) Sample-1, remove dynamic objects (c) Sample-2, original (d) Sampl…
Figure 7
Figure 7. Figure 7: STORM editing examples. We present examples of removing or cloning vehicles (a-d), as well as adding or replacing pedestrians and vehicles (e-h). Notice how STORM harmonizes the edited images due to the use of the decoder. More examples can be found at our project page…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. LSD-3D: Large-Scale 3D Driving Scene Generation with Geometry Grounding

    cs.CV 2025-08 conditional novelty 7.0 of 10

    LSD-3D generates explicit, 3D-consistent driving scenes by combining a generated proxy mesh with geometry-grounded distillation from a 2D diffusion model.

  2. FFAvatar: Feed-Forward 4D Head Avatar Reconstruction from Sparse Portrait Images

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    FFAvatar uses a Transformer-based 3D Gaussian model with alternating attention and sparse-to-dense learning to enable feed-forward, incremental reconstruction of animatable 4D head avatars from sparse portrait images.

  3. E3D-Bench: A Benchmark for End-to-End 3D Geometric Foundation Models

    cs.CV 2025-06 conditional novelty 6.0 of 10

    E3D-Bench compares 16 3D geometric foundation models on depth, reconstruction, pose, and view-synthesis tasks with a unified evaluation toolkit.

Reference graph

Works this paper leans on

34 extracted references · 9 canonical work pages · cited by 3 Pith papers

  1. [1]

    When computing PSNR, we map color back to [0, 1]. Sky MLP. The modulated sky MLP predicts sky color from view directions by conditioning a sky token csky through a modulated linear layer. Specifically, frequency-embedded viewing directional vectors γ(d) are linearly projected to 64 dimensions, then normalized using LayerNorm without affine parameters. The...

    work page 2023

  2. [3]

    Vision transformer adapter for dense predictions

    Zhe Chen, Yuchen Duan, Wenhai Wang, Junjun He, Tong Lu, Jifeng Dai, and Yu Qiao. Vision transformer adapter for dense predictions. arXiv preprint arXiv:2205.08534,

    arXiv

  3. [5]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929,

    arXiv 2010

  4. [6]

    Dynamic 3d gaussian fields for urban areas

    Tobias Fischer, Jonas Kulhanek, Samuel Rota Bul `o, Lorenzo Porzi, Marc Pollefeys, and Peter Kontschieder. Dynamic 3d gaussian fields for urban areas. arXiv preprint arXiv:2406.03175 , 2024a. Tobias Fischer, Lorenzo Porzi, Samuel Rota Bulo, Marc Pollefeys, and Peter Kontschieder. Multi- level neural scene graphs for dynamic urban environments. In Proceedi...

    arXiv

  5. [8]

    s3 gaussian: Self-supervised street gaussians for au- tonomous driving

    Nan Huang, Xiaobao Wei, Wenzhao Zheng, Pengju An, Ming Lu, Wei Zhan, Masayoshi Tomizuka, Kurt Keutzer, and Shanghang Zhang. s3 gaussian: Self-supervised street gaussians for au- tonomous driving. arXiv preprint arXiv:2405.20323,

    arXiv

  6. [9]

    Cotracker: It is better to track together

    Nikita Karaev, Ignacio Rocco, Benjamin Graham, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Cotracker: It is better to track together. arXiv preprint arXiv:2307.07635,

    arXiv

  7. [10]

    Dynmf: Neural motion factorization for real-time dynamic view synthesis with 3d gaussian splatting

    Agelos Kratimenos, Jiahui Lei, and Kostas Daniilidis. Dynmf: Neural motion factorization for real-time dynamic view synthesis with 3d gaussian splatting. arXiv preprint arXiv:2312.00112,

    arXiv

  8. [11]

    Mosca: Dynamic gaussian fusion from casual videos via 4d motion scaffolds

    Jiahui Lei, Yijia Weng, Adam Harley, Leonidas Guibas, and Kostas Daniilidis. Mosca: Dynamic gaussian fusion from casual videos via 4d motion scaffolds. arXiv preprint arXiv:2405.17421,

    arXiv

Show all 34 references
  1. [12]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 . OpenReview.net,

    2019

  2. [14]

    Hypernerf: A higher-dimensional representation for topologically varying neural radiance fields

    Keunhong Park, Utkarsh Sinha, Peter Hedman, Jonathan T Barron, Sofien Bouaziz, Dan B Goldman, Ricardo Martin-Brualla, and Steven M Seitz. Hypernerf: A higher-dimensional representation for topologically varying neural radiance fields. arXiv preprint arXiv:2106.13228,

    arXiv

  3. [15]

    Very deep convolutional networks for large-scale image recognition

    Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556,

    arXiv

  4. [17]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V . N. Vishwanathan, and Roman Garnett ...

    2017

  5. [19]

    Shape of motion: 4d reconstruction from a single video

    Qianqian Wang, Vickie Ye, Hang Gao, Jake Austin, Zhengqi Li, and Angjoo Kanazawa. Shape of motion: 4d reconstruction from a single video. 2024b. 14 Preprint Christopher Wewer, Kevin Raj, Eddy Ilg, Bernt Schiele, and Jan Eric Lenssen. latentsplat: Autoencoding variational gauss...

    arXiv

  6. [20]

    Argoverse 2: Next generation datasets for self-driving perception and forecasting

    Benjamin Wilson, William Qi, Tanmay Agarwal, John Lambert, Jagjeet Singh, Siddhesh Khandel- wal, Bowen Pan, Ratnesh Kumar, Andrew Hartnett, Jhony Kaesemodel Pontes, et al. Argoverse 2: Next generation datasets for self-driving perception and forecasting. In Thirty-fifth Confer...

    arXiv

  7. [21]

    S-nerf: Neural radiance fields for street views

    Ziyang Xie, Junge Zhang, Wenye Li, Feihu Zhang, and Li Zhang. S-nerf: Neural radiance fields for street views. arXiv preprint arXiv:2303.00749,

    arXiv

  8. [22]

    Grm: Large gaussian reconstruction model for efficient 3d reconstruction and generation

    Yinghao Xu, Zifan Shi, Wang Yifan, Sida Peng, Ceyuan Yang, Yujun Shen, and Wetzstein Gordon. Grm: Large gaussian reconstruction model for efficient 3d reconstruction and generation. arxiv: 2403.14621,

    arXiv

  9. [23]

    Street gaussians for modeling dynamic urban scenes

    Yunzhi Yan, Haotong Lin, Chenxu Zhou, Weijie Wang, Haiyang Sun, Kun Zhan, Xianpeng Lang, Xiaowei Zhou, and Sida Peng. Street gaussians for modeling dynamic urban scenes. arXiv preprint arXiv:2401.01339,

    arXiv

  10. [24]

    Emernerf: Emergent spatial-temporal scene decomposition via self-supervision

    Jiawei Yang, Boris Ivanovic, Or Litany, Xinshuo Weng, Seung Wook Kim, Boyi Li, Tong Che, Danfei Xu, Sanja Fidler, Marco Pavone, and Yue Wang. Emernerf: Emergent spatial-temporal scene decomposition via self-supervision. In The Twelfth International Conference on Learning Repre...

    2024 arXiv

  11. [25]

    Gs-lrm: Large reconstruction model for 3d gaussian splatting

    Kai Zhang, Sai Bi, Hao Tan, Yuanbo Xiangli, Nanxuan Zhao, Kalyan Sunkavalli, and Zexiang Xu. Gs-lrm: Large reconstruction model for 3d gaussian splatting. arXiv preprint arXiv:2404.19702,

    arXiv

  12. [27]

    Driv- inggaussian: Composite gaussian splatting for surrounding dynamic autonomous driving scenes

    Xiaoyu Zhou, Zhiwei Lin, Xiaojun Shan, Yongtao Wang, Deqing Sun, and Ming-Hsuan Yang. Driv- inggaussian: Composite gaussian splatting for surrounding dynamic autonomous driving scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 2...

    2024

  13. [28]

    For scale s, we compute s = min (exp(s′ − 2.3), 0.5), following Zhang et al

    that applies L2 normalization to ensure quaternion vectors are unit vectors. For scale s, we compute s = min (exp(s′ − 2.3), 0.5), following Zhang et al. (2024), where s′ rep- resents the outputs before normalization. This regularization limits the maximum size of Gaussians an...

    2024

  14. [30]

    Gradient checkpointing is enabled by default to reduce memory usage

    only 17 Preprint after 5,000 iterations, as we find this approach stabilizes training. Gradient checkpointing is enabled by default to reduce memory usage. Behind the scene, we observe that STORM benefits from longer training durations and larger model sizes. We maintain the d...

    2022

  15. [32]

    However, LGM is originally trained on an object-centric synthetic dataset, which has a significant domain gap compared to our problem

    has not. However, LGM is originally trained on an object-centric synthetic dataset, which has a significant domain gap compared to our problem. Therefore, we followed their official code to reimplement their model within our codebase to elimi- nate potential misalignments due ...

    2022

  16. [33]

    These scenes are divided into 700, 150, and 150 scenes for training, validation, and testing, respectively

    contains 1000 driving scenes, each lasting 20 seconds, captured at 12Hz frame rate. These scenes are divided into 700, 150, and 150 scenes for training, validation, and testing, respectively. Similar to our evaluation protocol for the Waymo Open Dataset (Sun et al., 2020), we ...

    2020

  17. [288]

    Models are trained on the training set and evaluated on the validation set with unchanged hyperparameters

    and leverage both sample (key frames) and sweep data. Models are trained on the training set and evaluated on the validation set with unchanged hyperparameters. Argoverse2. The Argoverse2 dataset (Wilson et al.) contains 1,000 driving scenes, split into 700 for training, 150 f...

  18. [2014]

    We setλlpips to 0.05, λsky to 0.1, and λreg to 5e-3 in all experiments

    implementation. We setλlpips to 0.05, λsky to 0.1, and λreg to 5e-3 in all experiments. A.3 B ASELINE IMPLEMENTATIONS For per-scene optimization 3DGS-based methods, we use the recently open-sourced codebase DriveStudio from Chen et al. (2024), which includes implementations fo...

    2024

  19. [2017]

    Letian Wang, Seung Wook Kim, Jiawei Yang, Cunjun Yu, Boris Ivanovic, Steven L Waslander, Yue Wang, Sanja Fidler, Marco Pavone, and Peter Karkus

    URL https://proceedings.neurips.cc/paper/2017/hash/ 3f5ee243547dee91fbd053c1c4a845aa-Abstract.html. Letian Wang, Seung Wook Kim, Jiawei Yang, Cunjun Yu, Boris Ivanovic, Steven L Waslander, Yue Wang, Sanja Fidler, Marco Pavone, and Peter Karkus. Distillnerf: Perceiving 3d scene...

    2017 arXiv

  20. [2018]

    Hugs: Holistic urban 3d scene understanding via gaussian splatting

    Hongyu Zhou, Jiahao Shao, Lu Xu, Dongfeng Bai, Weichao Qiu, Bingbing Liu, Yue Wang, Andreas Geiger, and Yiyi Liao. Hugs: Holistic urban 3d scene understanding via gaussian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 213...

    arXiv

  21. [2019]

    Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan

    URL https://openreview.net/forum?id=Bkg6RiCqY7. Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis. arXiv preprint arXiv:2308.09713,

    arXiv

  22. [2020]

    Flash3d: Feed-forward generalisable 3d scene recon- struction from a single image

    Stanislaw Szymanowicz, Eldar Insafutdinov, Chuanxia Zheng, Dylan Campbell, Jo˜ao F Henriques, Christian Rupprecht, and Andrea Vedaldi. Flash3d: Feed-forward generalisable 3d scene recon- struction from a single image. arXiv preprint arXiv:2406.04343, 2024a. Stanislaw Szymanowi...

    arXiv

  23. [2021]

    Gaussianflow: Splatting gaussian dynamics for 4d content creation

    Quankai Gao, Qiangeng Xu, Zhe Cao, Ben Mildenhall, Wenchao Ma, Le Chen, Danhang Tang, and Ulrich Neumann. Gaussianflow: Splatting gaussian dynamics for 4d content creation. arXiv preprint arXiv:2403.12365, 2024a. Ruiqi Gao, Aleksander Holynski, Philipp Henzler, Arthur Brussee,...

    arXiv

  24. [2022]

    Omnire: Omni urban scene reconstruc- tion

    Ziyu Chen, Jiawei Yang, Jiahui Huang, Riccardo de Lutio, Janick Martinez Esturo, Boris Ivanovic, Or Litany, Zan Gojcic, Sanja Fidler, Marco Pavone, et al. Omnire: Omni urban scene reconstruc- tion. arXiv preprint arXiv:2408.16760,

    arXiv

  25. [2023]

    Shapenet: An information-rich 3d model repository

    Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012,

    arXiv

  26. [2024]

    Periodic vibration gaussian: Dynamic urban scene reconstruction and real-time rendering

    Yurui Chen, Chun Gu, Junzhe Jiang, Xiatian Zhu, and Li Zhang. Periodic vibration gaussian: Dynamic urban scene reconstruction and real-time rendering. arXiv preprint arXiv:2311.18561,

    arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.