4D Gaussian Splatting for Real-Time Dynamic Scene Rendering
read the original abstract
Representing and rendering dynamic scenes has been an important but challenging task. Especially, to accurately model complex motions, high efficiency is usually hard to guarantee. To achieve real-time dynamic scene rendering while also enjoying high training and storage efficiency, we propose 4D Gaussian Splatting (4D-GS) as a holistic representation for dynamic scenes rather than applying 3D-GS for each individual frame. In 4D-GS, a novel explicit representation containing both 3D Gaussians and 4D neural voxels is proposed. A decomposed neural voxel encoding algorithm inspired by HexPlane is proposed to efficiently build Gaussian features from 4D neural voxels and then a lightweight MLP is applied to predict Gaussian deformations at novel timestamps. Our 4D-GS method achieves real-time rendering under high resolutions, 82 FPS at an 800$\times$800 resolution on an RTX 3090 GPU while maintaining comparable or better quality than previous state-of-the-art methods. More demos and code are available at https://guanjunwu.github.io/4dgs/.
This paper has not been read by Pith yet.
Forward citations
Cited by 15 Pith papers
-
No Pose, No Problem in 4D: Feed-Forward Dynamic Gaussians from Unposed Multi-View Videos
NoPo4D is the first feed-forward system for dynamic 4D Gaussian splatting from unposed multi-view videos, using velocity decomposition supervised by optical flow and a bidirectional motion encoder.
-
Splats in Splats++: Robust and Generalizable 3D Gaussian Splatting Steganography
Splats in Splats++ embeds messages into 3DGS via importance-graded SH encryption, hash-grid opacity mapping, and a gradient-gated consistency loss, achieving higher fidelity and robustness than prior methods.
-
GS-Surrogate: Deformable Gaussian Splatting for Parameter Space Exploration of Ensemble Simulations
GS-Surrogate creates a canonical Gaussian field that is sequentially deformed by simulation parameters to enable real-time, controllable 3D exploration of ensemble data while separating simulation variations from visu...
-
Can Protective Watermarking Safeguard the Copyright of 3D Gaussian Splatting?
GSPure removes watermarks from 3D Gaussian Splatting by isolating watermark Gaussians via view-dependent contributions and feature clustering, cutting watermark PSNR by up to 16.34 dB with under 1 dB scene quality loss.
-
Unified Panoramic-Gaussian Representation for Monocular 4D Scene Synthesis
PanoGaussian distills panoramic representations into explicit dynamic Gaussians for consistent monocular 4D scene synthesis under large viewpoint variations.
-
Learning Video Dynamics with Predictive Differentiable Rendering
PDR integrates a plug-and-play 2D Gaussian representation and CUDA renderer into pixel-based video predictors, replacing MSE loss with L1+SSIM to improve detail preservation on benchmarks like TaxiBJ and Human3.6M.
-
HAT-4D: Lifting Monocular Video for 4D Multi-Object Interactions via Human-Agent Collaboration
HAT-4D presents an agentic VLM-plus-human-in-the-loop pipeline for monocular 4D multi-object interaction reconstruction and releases the MVOIK-4D benchmark.
-
Feed-forward Motion In-betweening for Any 4D
Proposes a feed-forward keyframe-conditioned in-betweening method for arbitrary 4D meshes using a topology-agnostic VAE and MMDiT-based rectified flow model.
-
Robust Dreamer: Deviation-Aware Latent Gaussian Memory for Action-Controlled AR Video Generation
Robust Dreamer uses Latent Gaussian Memory anchored to diffusion latents and Deviation Learning with a Dynamic Deviation Archive to reduce drift in long-horizon action-controlled image-to-video generation, reporting S...
-
R5DGS: Semantic-Aware 4D Gaussian Splatting with Rigid Body Constraints for Efficient Dynamic Scene Reconstruction
R5DGS augments physics-driven 4D Gaussian splatting with identity encodings and centroid-only rigid-body dynamics to enable semantic open-vocabulary retrieval and 11 FPS faster extrapolation.
-
LagrangianSplats: Divergence-Free Transport of Gaussian Primitives for Fluid Reconstruction
A framework that structurally enforces divergence-free velocity and long-range transport coherence in 3D fluid reconstruction from 2D videos via divergence-free kernels advecting Lagrangian Gaussian splats.
-
WARPED: Wrist-Aligned Rendering for Robot Policy Learning from Egocentric Human Demonstrations
WARPED synthesizes realistic wrist-view observations from monocular egocentric human videos via foundation models, hand-object tracking, retargeting, and Gaussian Splatting to train visuomotor policies that match tele...
-
POTR: Post-Training 3DGS Compression
POTR introduces simultaneous-effect pruning via a modified 3DGS rasterizer and entropy-reducing lighting coefficient recomputation to outperform prior post-training 3DGS compression methods in rate-distortion and infe...
-
WebSpline: Structure-Informed Splines for Real-Time 3D Gaussians from Monocular Videos
WebSpline uses learnable cubic Hermite splines guided by a Structural Proxy Graph to deliver state-of-the-art quality dynamic 3D Gaussian rendering from monocular videos at over 10x the speed of prior methods on iPhon...
-
4D Vessel Reconstruction for Benchtop Thrombectomy Analysis
A nine-camera multi-view workflow with 4D Gaussian Splatting reconstructs dynamic vessel surfaces in thrombectomy phantoms to enable standardized comparative displacement and stress-proxy tracking.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.