D-NeRF: Neural Radiance Fields for Dynamic Scenes
read the original abstract
Neural rendering techniques combining machine learning with geometric reasoning have arisen as one of the most promising approaches for synthesizing novel views of a scene from a sparse set of images. Among these, stands out the Neural radiance fields (NeRF), which trains a deep network to map 5D input coordinates (representing spatial location and viewing direction) into a volume density and view-dependent emitted radiance. However, despite achieving an unprecedented level of photorealism on the generated images, NeRF is only applicable to static scenes, where the same spatial location can be queried from different images. In this paper we introduce D-NeRF, a method that extends neural radiance fields to a dynamic domain, allowing to reconstruct and render novel images of objects under rigid and non-rigid motions from a \emph{single} camera moving around the scene. For this purpose we consider time as an additional input to the system, and split the learning process in two main stages: one that encodes the scene into a canonical space and another that maps this canonical representation into the deformed scene at a particular time. Both mappings are simultaneously learned using fully-connected networks. Once the networks are trained, D-NeRF can render novel images, controlling both the camera view and the time variable, and thus, the object movement. We demonstrate the effectiveness of our approach on scenes with objects under rigid, articulated and non-rigid motions. Code, model weights and the dynamic scenes dataset will be released.
This paper has not been read by Pith yet.
Forward citations
Cited by 10 Pith papers
-
Breaking the Rigid Prior: Towards Articulated 3D Anomaly Detection
The paper releases the ArtiAD benchmark with 15k+ articulated point clouds and shows that a pose-conditioned SDF baseline reaches 0.884/0.874 AUROC on seen/unseen joint configurations, outperforming rigid priors.
-
No Pose, No Problem in 4D: Feed-Forward Dynamic Gaussians from Unposed Multi-View Videos
NoPo4D is the first feed-forward system for dynamic 4D Gaussian splatting from unposed multi-view videos, using velocity decomposition supervised by optical flow and a bidirectional motion encoder.
-
GSDeformer: Direct, Real-time and Extensible Cage-based Deformation for 3D Gaussian Splatting
GSDeformer enables direct, real-time cage-based deformation on 3D Gaussian Splatting via a proxy point-cloud representation and automated cage construction, without modifying the core 3DGS architecture.
-
Self-Learning Expression Deformations for Data-Efficient Gaussian Avatars
SAGE self-learns Gaussian expression deformations via joint surfel-SDF optimization and self-supervised consistency, enabling comparable avatar quality from single frames, monocular rotations, or one-shot inputs.
-
R5DGS: Semantic-Aware 4D Gaussian Splatting with Rigid Body Constraints for Efficient Dynamic Scene Reconstruction
R5DGS augments physics-driven 4D Gaussian splatting with identity encodings and centroid-only rigid-body dynamics to enable semantic open-vocabulary retrieval and 11 FPS faster extrapolation.
-
RiGS: Rigid-aware 4D Gaussian Splatting from a Single Monocular Video
RiGS decomposes scenes into static, rigid, and transient 4D Gaussians with an object-wise dynamic mask and scene flow guidance to model multi-scale motions and achieve SOTA novel view synthesis.
-
MatPhys: Learning Material-Aware Physics Parameters for Deformable Object Simulation from Videos
MatPhys is a feed-forward framework that predicts consistent part-level spring-mass parameters for deformable object simulation from monocular videos using semantic decomposition and a material embedding codebook.
-
Velox: Learning Representations of 4D Geometry and Appearance
Velox compresses dynamic point clouds into latent tokens that support geometry via 4D surface modeling and appearance via 3D Gaussians, showing strong results on video-to-4D generation, tracking, and image-to-4D cloth...
-
Beyond Static Gaussians: An Empirical Investigation of Architectural Paradigms for Dynamic 3D Scene Reconstruction
Structure-guided dynamic 3DGS methods deliver superior reconstruction fidelity and compactness on D-NeRF while gaussian-centric methods provide higher rendering speeds at the cost of quality variability and storage.
-
Real-Time Physics Simulation with Dynamic Mesh-Gaussian Reconstructions
Dual-representation framework pairs fixed-topology meshes for physics with Gaussian splatting for rendering, but two conversion strategies from varying-topology reconstructions cause 65-80% geometric degradation and u...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.