TRIG factorizes multi-camera poses into ego-trajectory and static rig geometry, with decoupled supervision and sparse temporal-spatial attention, claiming SOTA metric depth, pose, and 3D reconstruction on five driving benchmarks.
Driv3R: Learn- ing dense 4d reconstruction for autonomous driving
8 Pith papers cite this work. Polarity classification is still indexing.
8
Pith papers citing it
abstract
Realtime 4D reconstruction for dynamic scenes remains a crucial challenge for autonomous driving perception. Most existing methods rely on depth estimation through self-supervision or multi-modality sensor fusion. In this paper, we propose Driv3R, a DUSt3R-based framework that directly regresses per-frame point maps from multi-view image sequences. To achieve streaming dense reconstruction, we maintain a memory pool to reason both spatial relationships across sensors and dynamic temporal contexts to enhance multi-view 3D consistency and temporal integration. Furthermore, we employ a 4D flow predictor to identify moving objects within the scene to direct our network focus more on reconstructing these dynamic regions. Finally, we align all per-frame pointmaps consistently to the world coordinate system in an optimization-free manner. We conduct extensive experiments on the large-scale nuScenes dataset to evaluate the effectiveness of our method. Driv3R outperforms previous frameworks in 4D dynamic scene reconstruction, achieving 15x faster inference speed compared to methods requiring global alignment. Code: https://github.com/Barrybarry-Smith/Driv3R.
citation-role summary
baseline 1
citation-polarity summary
fields
cs.CV 8roles
baseline 1polarities
baseline 1representative citing papers
Argus plus Realsee3D deliver state-of-the-art metric camera pose, depth, and point-cloud reconstruction from unordered indoor panoramas via learned covisibility anchoring and geometric factorization.
Envision4D presents a feed-forward 4D Gaussian Splatting framework with future pose prediction, temporal attention, and conditioned motion lifting for pose-free extrapolation in autonomous driving scenes.
Sensor2Sensor uses 4D Gaussian Splatting to create synthetic training pairs and a diffusion model to convert monocular dashcam videos into high-fidelity multi-modal AV sensor data.
DVGT-2 is a streaming vision-geometry-action model that jointly reconstructs dense 3D geometry and plans trajectories online, achieving better reconstruction than prior batch methods while transferring directly to planning benchmarks without fine-tuning.
CylinderDepth uses cylindrical spatial attention with non-learned weights to enforce cross-view consistency in self-supervised surround depth estimation.
A causal transformer with key-value caching and distillation from a bidirectional VGGT model enables efficient online 4D geometry reconstruction from videos.
Geometry Forcing aligns video diffusion representations with geometric foundation model features via angular cosine and scale regression objectives to improve 3D consistency in generated videos.
citing papers explorer
-
TRIG: Trajectory-Rig Decoupled Metric Geometry Learning
TRIG factorizes multi-camera poses into ego-trajectory and static rig geometry, with decoupled supervision and sparse temporal-spatial attention, claiming SOTA metric depth, pose, and 3D reconstruction on five driving benchmarks.
-
Argus: Metric Panoramic 3D Reconstruction for Indoor Scenes
Argus plus Realsee3D deliver state-of-the-art metric camera pose, depth, and point-cloud reconstruction from unordered indoor panoramas via learned covisibility anchoring and geometric factorization.
-
Envision4D: Envisioning Visual Futures via Feed-forward 4D Gaussian Splatting for Autonomous Driving
Envision4D presents a feed-forward 4D Gaussian Splatting framework with future pose prediction, temporal attention, and conditioned motion lifting for pose-free extrapolation in autonomous driving scenes.
-
Sensor2Sensor: Cross-Embodiment Sensor Conversion for Autonomous Driving
Sensor2Sensor uses 4D Gaussian Splatting to create synthetic training pairs and a diffusion model to convert monocular dashcam videos into high-fidelity multi-modal AV sensor data.
-
DVGT-2: Vision-Geometry-Action Model for Autonomous Driving at Scale
DVGT-2 is a streaming vision-geometry-action model that jointly reconstructs dense 3D geometry and plans trajectories online, achieving better reconstruction than prior batch methods while transferring directly to planning benchmarks without fine-tuning.
-
CylinderDepth: Cylindrical Spatial Attention for Multi-View Consistent Self-Supervised Surround Depth Estimation
CylinderDepth uses cylindrical spatial attention with non-learned weights to enforce cross-view consistency in self-supervised surround depth estimation.
-
Streaming 4D Visual Geometry Transformer
A causal transformer with key-value caching and distillation from a bidirectional VGGT model enables efficient online 4D geometry reconstruction from videos.
-
Geometry Forcing: Marrying Video Diffusion and 3D Representation for Consistent World Modeling
Geometry Forcing aligns video diffusion representations with geometric foundation model features via angular cosine and scale regression objectives to improve 3D consistency in generated videos.