Drone-assisted Road Gaussian Splatting with Cross-view Uncertainty
read the original abstract
Robust and realistic rendering for large-scale road scenes is essential in autonomous driving simulation. Recently, 3D Gaussian Splatting (3D-GS) has made groundbreaking progress in neural rendering, but the general fidelity of large-scale road scene renderings is often limited by the input imagery, which usually has a narrow field of view and focuses mainly on the street-level local area. Intuitively, the data from the drone's perspective can provide a complementary viewpoint for the data from the ground vehicle's perspective, enhancing the completeness of scene reconstruction and rendering. However, training naively with aerial and ground images, which exhibit large view disparity, poses a significant convergence challenge for 3D-GS, and does not demonstrate remarkable improvements in performance on road views. In order to enhance the novel view synthesis of road views and to effectively use the aerial information, we design an uncertainty-aware training method that allows aerial images to assist in the synthesis of areas where ground images have poor learning outcomes instead of weighting all pixels equally in 3D-GS training like prior work did. We are the first to introduce the cross-view uncertainty to 3D-GS by matching the car-view ensemble-based rendering uncertainty to aerial images, weighting the contribution of each pixel to the training process. Additionally, to systematically quantify evaluation metrics, we assemble a high-quality synthesized dataset comprising both aerial and ground images for road scenes.
This paper has not been read by Pith yet.
Forward citations
Cited by 5 Pith papers
-
Sky2Ground: A Benchmark for Site Modeling under Varying Altitude
Sky2Ground provides a new three-view multi-altitude dataset and SkyNet model that improves absolute pose estimation performance by 9.6% on RRA@5 and 18.1% on RTA@5 over prior methods when satellite imagery is included.
-
TideGS: Scalable Training of Over One Billion 3D Gaussian Splatting Primitives via Out-of-Core Optimization
TideGS scales 3D Gaussian Splatting training to over one billion primitives on a single 24GB GPU via block-virtualized geometry, asynchronous I/O pipelines, and trajectory-adaptive differential streaming.
-
TideGS: Scalable Training of Over One Billion 3D Gaussian Splatting Primitives via Out-of-Core Optimization
TideGS scales 3D Gaussian Splatting to over one billion primitives on a single 24 GB GPU by using block-virtualized geometry, asynchronous I/O pipelines, and trajectory-adaptive differential streaming to exploit train...
-
Visually-grounded Humanoid Agents
A coupled world-agent framework uses 3D Gaussian reconstruction and first-person RGB-D perception with iterative planning to enable goal-directed, collision-avoiding humanoid behavior in novel reconstructed scenes.
-
ABot-Earth 0.5: Generative 3D Earth Model
ABot-Earth 0.5 is a 3DGS-based generative model trained on real-world urban reconstructions that synthesizes novel 3D scenes from satellite imagery in under 10 minutes per square kilometer.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.