Pith. sign in

REVIEW 8 cited by

DreamDrive: Generative 4D Scene Modeling from Street View Images

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2501.00601 v2 pith:74PRVYY2 submitted 2024-12-31 cs.CV cs.AIcs.GR

Jiageng Mao , Boyi Li , Boris Ivanovic , Yuxiao Chen , Yan Wang , Yurong You , Chaowei Xiao , Danfei Xu
show 2 more authors
Marco Pavone Yue Wang
This is my paper
classification cs.CVcs.AIcs.GR
keywords drivingscenessynthesizevideosgenerativeconsistencydreamdrivegeneralizable
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Synthesizing photo-realistic visual observations from an ego vehicle's driving trajectory is a critical step towards scalable training of self-driving models. Reconstruction-based methods create 3D scenes from driving logs and synthesize geometry-consistent driving videos through neural rendering, but their dependence on costly object annotations limits their ability to generalize to in-the-wild driving scenarios. On the other hand, generative models can synthesize action-conditioned driving videos in a more generalizable way but often struggle with maintaining 3D visual consistency. In this paper, we present DreamDrive, a 4D spatial-temporal scene generation approach that combines the merits of generation and reconstruction, to synthesize generalizable 4D driving scenes and dynamic driving videos with 3D consistency. Specifically, we leverage the generative power of video diffusion models to synthesize a sequence of visual references and further elevate them to 4D with a novel hybrid Gaussian representation. Given a driving trajectory, we then render 3D-consistent driving videos via Gaussian splatting. The use of generative priors allows our method to produce high-quality 4D scenes from in-the-wild driving data, while neural rendering ensures 3D-consistent video generation from the 4D scenes. Extensive experiments on nuScenes and street view images demonstrate that DreamDrive can generate controllable and generalizable 4D driving scenes, synthesize novel views of driving videos with high fidelity and 3D consistency, decompose static and dynamic elements in a self-supervised manner, and enhance perception and planning tasks for autonomous driving.

Discussion (0). Sign in to comment.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Instant NuRec: Feed-Forward 3D Gaussian Reconstruction for Driving Scene Simulation

    cs.GR 2026-07 conditional novelty 6.0 of 10

    A feed-forward model reconstructs a layered, simulation-ready 3D Gaussian world from multi-view driving video in ~1.5 s, with quality approaching per-scene optimized reconstruction.

  2. Pixel-to-4D: Camera-Controlled Image-to-Video Generation with Dynamic 3D Gaussians

    cs.CV 2026-01 unverdicted novelty 6.0 of 10

    A unified single-pass framework using dynamic 3D Gaussians generates temporally consistent camera-controlled videos from a single image.

  3. Pixel-to-4D: Camera-Controlled Image-to-Video Generation with Dynamic 3D Gaussians

    cs.CV 2026-01 unverdicted novelty 6.0 of 10

    Pixel-to-4D builds a dynamic 3D Gaussian representation from one image and samples object motion in a single forward pass to produce camera-controlled videos with claimed state-of-the-art quality and speed on KITTI, W...

  4. OmniNWM: Omniscient Driving Navigation World Models

    cs.CV 2025-10 conditional novelty 6.0 of 10

    OmniNWM jointly generates long panoramic multi-modal driving videos, controls them precisely via normalized Plücker ray-maps, and derives dense driving rewards from generated 3D occupancy.

  5. GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Driving

    cs.CV 2025-03 unverdicted novelty 6.0 of 10

    GAIA-2 is a controllable latent diffusion world model that produces spatiotemporally consistent multi-view videos for autonomous driving simulation across diverse geographies.

  6. DriveCtrl: Conditioned Sim-to-Real Driving Video Generation

    cs.CV 2026-05 unverdicted novelty 5.0 of 10

    DriveCtrl is a depth-conditioned controllable framework that generates realistic driving videos from simulation while preserving annotations and scene dynamics.

  7. Scaling Up Occupancy-centric Driving Scene Generation: Dataset and Method

    cs.CV 2025-10 conditional novelty 5.0 of 10

    UniScenev2 scales occupancy-centric driving-scene generation to NuPlan scale, releasing a 3.6M-frame semantic-occupancy dataset and jointly generating occupancy, video, and LiDAR that beats published baselines on its ...

  8. 3D and 4D World Modeling: A Survey

    cs.CV 2025-09 conditional novelty 5.0 of 10

    A survey that defines 3D/4D world modeling, organizes methods into VideoGen, OccGen, and LiDARGen categories, and compiles datasets, metrics, and benchmark numbers.

Pith tools