Pith. sign in

REVIEW 2 major objections 4 minor 1 cited by

HoliGS: Holistic Gaussian Splatting for Embodied View Synthesis

T0 review · 2 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Deformable Gaussian splatting reconstructs minute-long dynamic scenes from monocular video about ten times faster than prior neural radiance field pipelines while improving rendering quality.

desk verdict Solid deformable-Gaussian system with a useful hierarchy and honest ablations, but the monocular and embodied-view claims outrun the evaluation: initialization depends on stereo depth and ARKit poses, and the held-out views are the small-baseline right stereo camera. read the letter →

arxiv 2506.19291 v1 pith:VLCIZOMO submitted 2025-06-24 cs.CV

Xiaoyuan Wang , Yizhou Zhao , Botao Ye , Xiaojun Shan , Weijie Lyu , Lu Qi , Kelvin C.K. Chan , Yinxiao Li
show 1 more author
Ming-Hsuan Yang
This is my paper
classification cs.CV
keywords Gaussiansplattingembodiedviewsynthesisdynamicscenereconstructiondeformableneuralrenderingarticulatedobjectsmonocularvideoinvertibleflow4D
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

HoliGS is a deformable Gaussian splatting system for embodied view synthesis from long monocular RGB videos. The paper set out to show that dynamic scenes with multiple interacting humans and animals, captured over hundreds of frames with large viewpoint changes, can be reconstructed into temporally consistent geometry, appearance, and depth and rendered at interactive speeds. The central move is to decompose scene motion into a hierarchy: a rigid global transform, skeleton-driven articulated skinning, and an invertible flow that absorbs residual non-rigid deformation. A neural-SDF pre-training stage gives the Gaussians a good starting pose and scale. If the paper is right, minute-long phone captures become practical free-viewpoint 4D scenes, supporting egocentric and third-person-view replay, object removal, and dense depth estimation.

What carries the argument

The load-bearing object is the invertible deformation pipeline that combines three warping stages: a per-frame rigid SE(3) transform for the object root and background, dual-quaternion blend skinning over a learned bone hierarchy, and an invertible RealNVP flow for soft deformation. Because the flow is invertible, the model enforces a cycle-consistency loss between forward and inverse warps, which stabilizes training over long sequences. A second piece of machinery is the neural-SDF warm start: sampled points are centered on back-projected surface points from depth, and the warping network is pre-trained on photometric, depth, flow, cycle, and segmentation losses before dynamic Gaussian splatting takes over. Together these let the representation explain roughly ninety percent of visible motion with the skeleton, leaving the invertible flow to handle only low-amplitude residual detail, which is why the paper claims convergence in about two hours on a single GPU.

What would settle it

Run HoliGS on a monocular video with no paired depth and no external pose initialization. If the novel-view renderings reproduce the floating-foreground and scale-drift artifacts shown in the 'w/o L_depth' ablation (Table 4, row 2) or fail to converge as in the 'w/o root-body G_j' row, then the central claim of monocular embodied view synthesis is not sustained.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that HoliGS outperforms state-of-the-art monocular deformable NeRFs on both reconstruction quality and speed for embodied view synthesis. The claim is supported by experiments on 11 stereo-captured sequences of humans and animals: the method reports the best LPIPS, PSNR, SSIM, and depth accuracy across all sequences, cutting training time to about 1.2 hours versus roughly 12 hours for Total-Recon on the same GPU, and rendering novel views at about 20 fps on consumer hardware. The mechanism the paper credits for this is the hierarchical warping decomposition, in which a canonical point is warped through the inverse soft flow, the skeleton skinning, and the object-root camera transform as $X_t = {G_t^o}^{-1} {J_t}^{-1} S^{-1}(X^*, \omega_t^d)$, together with a two-stage optimization that first fits a neural SDF to initialize poses and scale, then switches to Gaussian splatting for final refinement.

Load-bearing premise

The method needs trustworthy depth and body/camera pose estimates during initialization; if a video lacks those, as a truly monocular phone capture would, the scale consistency and stable convergence the paper reports are not established.

Editorial extensions

If this is right

  • Minute-long monocular videos of dynamic scenes become tractable: full scenes with multiple deformable objects converge in about two hours on an H20 or A6000 GPU, roughly ten times faster than the strongest NeRF baseline.
  • Novel views from embodied trajectories, including egocentric, third-person follow, and overhead perspectives, can be rendered at interactive rates of about 20 fps, making AR/VR and virtual cinematography practical.
  • The same representation yields metric depth and deformable mesh recovery, enabling actor-specific view synthesis, object removal, and spatiotemporal trajectory analysis.
  • The hierarchical motion decomposition removes the need for expensive point-tracking pipelines and helps the optimization stay stable under occlusion and extreme viewpoint changes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's experiments use stereo-captured depth and device motion sensors for initialization, so its monocular claim is not yet demonstrated on a video without any depth signal; a natural next test is to run it on a standard phone clip and check whether the scale drift seen in the no-depth ablation reappears.
  • Because the skeleton is learned without parametric body models, the method trades anatomical precision for generality; combining the learned bones with a parametric human or animal prior might benefit the human sequences while keeping the approach applicable to arbitrary articulated objects.
  • The invertible flow formulation is general enough that the same three-way decomposition could in principle be applied to other articulated entities such as furniture or robots, though the paper only evaluates humans and animals.
  • The reported speed-up depends on the warm-start initialization; without it, performance drops sharply (Appendix Table 5), so the practical benefit may shrink for sequences where reliable depth and pose are unavailable.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper proposes HoliGS, a deformable Gaussian splatting framework for embodied view synthesis of long dynamic scenes. The scene is decomposed into a static background and time-varying foreground objects; motion is modeled hierarchically as global SE(3) transformations, skeleton-driven dual-quaternion blend skinning, and a residual invertible RealNVP soft-deformation field. A neural-SDF warm-start initializes the deformation network and Gaussian positions, after which a two-stage optimization (component pre-training and joint refinement) fits the full 4D representation. Experiments are conducted on a newly captured stereo dataset of 11 sequences of humans interacting with animals. The paper reports higher LPIPS/PSNR/SSIM and depth-accuracy numbers than dynamic NeRF and Gaussian baselines, as well as substantially lower training and rendering times.

Significance. If the stated claims are accepted, the paper would be a practical engineering contribution: it combines articulated motion priors with an invertible deformation model, avoids costly point-tracking pipelines, scales to minute-long captures, and reports 20 fps rendering on consumer hardware. The ablation study in Table 4 supports the value of the individual components, and the appendix provides useful implementation details. The main reservations concern the alignment between the claims in the abstract and the evaluation protocol: the method is demonstrated on stereo data with ground-truth depth and ARKit/PoseNet pose initialization, and the quantitative novel-view metrics are computed on right-stereo frames rather than on the egocentric or third-person trajectories that the paper identifies as the embodied-view setup. These gaps mean the headline claims of monocular input and embodied view synthesis are not currently established by the evidence presented.

major comments (2)
  1. [§3.2, Appendix A.1, Tables 4 and 5] The central claim that HoliGS works from long monocular RGB videos is not supported by the experimental setup. The deformation-network initialization samples points centered on the surface point derived from back-projecting ground-truth depth (Appendix A.1), and the background root pose is initialized from ARKit motion-sensor extrinsics while foreground poses come from PoseNet (§3.2). Table 4 shows that removing the depth loss (row 2) substantially degrades accuracy and that removing root-body poses entirely (row 7) prevents convergence (N/A); Table 5 shows that removing the initialization stage drops PSNR from 21.31 to 17.30. These results demonstrate that the method's success depends on stereo-derived depth and accurate pose priors, which are not available in a genuine monocular RGB capture. The authors should either evaluate on true monocular sequences with estimated depth and pose, or substantially reframe the claims to describe a stereo/pose-assisted system.
  2. [Appendix A.1, Table 2, Figure 4] The embodied-view-synthesis claim is not quantitatively validated. The held-out validation views in the quantitative NVS evaluation are the right-camera frames of a stereo pair, which have only a small baseline relative to the training views. The egocentric and third-person-follow trajectories that define EVS are only shown qualitatively in Figure 4, without metrics. This does not demonstrate robust free-viewpoint rendering under the extreme viewpoint changes claimed in the abstract and Table 1. The paper should add quantitative evaluation on embodied trajectories (or a proxy such as large-baseline re-rendering with ground-truth depth) and report numbers for those viewpoints, or restrict the EVS claims to what the current protocol supports.
minor comments (4)
  1. [Abstract and §1] The abstract repeatedly states the input is monocular RGB video, but the dataset and initialization rely on stereo depth and device motion sensors. The wording should be adjusted to reflect the actual input modalities or the experiments should be extended to include monocular captures.
  2. [Table 4 caption and §4.3] The row references in the text do not match the table. The caption says removing root-body poses entirely is row (5), but the table shows that row is the soft-deformation ablation and row (7) is the root-body-pose removal. Several other row citations in §4.3 are likewise inconsistent with the table. These references should be corrected.
  3. [Eq. (1) and Eq. (2)] The notation in Eq. (2) appears unbalanced: the left-hand side is a canonical point X*, while the right-hand side applies S, J_t, and G_t^o to a world-space point X^t. If this is intended as the forward warp, the inverse flow should be used or the notation should be clarified.
  4. [§4.1] The description of the loss weights is inconsistent: the text states default weights for λ_depth = 5 or 1.5 for HUMAN 1, but it is not stated whether these per-sequence weights are tuned on the validation set or fixed a priori. A brief statement about weight selection would improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: HoliGS is an empirical fitting/systems paper whose metrics are evaluated on held-out views, with no derivation step that reduces to its inputs.

full rationale

Walking the claimed derivation chain—hierarchical warping (Eqs. 1-2), neural-SDF pre-training (Sec. 3.2), Gaussian splatting optimization (Sec. 3.3), and embodied viewpoint transformation (Sec. 3.4)—no step defines a predicted quantity in terms of the same quantity or fits a parameter and then reports it as an independent prediction. The deformation field is initialized using external supervision (PoseNet, ARKit poses, RAFT flow, SAM masks, depth) and then optimized with photometric, depth, normal, and mask losses; this is parameter fitting, not circular derivation. Novel-view metrics (Table 2) are computed on the withheld right-camera frames while training uses left-camera frames, so the NVS comparison is an external evaluation. Depth metrics (Table 3) use stereo-derived ground truth, and depth supervision is applied on training views; even if the supervision source is described inconsistently (Sec. 3.2 says off-the-shelf monocular depth estimator [69], while Appendix A.1 describes paired depth images and the dataset details say stereo-derived ground truth), this is a data-supervision/correctness concern, not a self-referential loop. Ablations (Table 4) show expected component dependencies (removing depth or root-body poses hurts), but these are empirical sensitivity results, not circularity. No load-bearing self-citations appear: the cited deformation/skinning/pose tools (RealNVP [66], DQB [65], PoseNet [68], RAFT [70], SAM [71]) are external, independently published methods. Per-sequence loss-weight tuning (lambda_depth = 1.5 for HUMAN 1) is standard hyperparameter selection and does not constitute a fitted-input-called-prediction. The Limitations section's admission that generic pose estimation may mis-match SMPL-level anatomical accuracy is a stated weakness of robustness, not a circular step. Overall, the central claims stand on external benchmarks and held-out evaluation, so the circularity score is 0.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claims rest on tuned loss weights, external supervision from depth/flow/segmentation/pose priors, a warm start from a neural SDF fit to stereo depth, and an evaluation protocol that uses near-duplicate stereo views as novel views. There are no newly invented physical entities, but several domain assumptions about the reliability of priors and the representativeness of the evaluation.

free parameters (4)
  • lambda_depth (depth loss weight) = 5.0 default, 1.5 for HUMAN 1
    Chosen per sequence; the ablation shows removing depth loss drops LPIPS from .263 to .385 and Acc@0.1m from .896 to .847, so the central result depends on this tuned weight.
  • global scale factor for depth and ARKit extrinsics = 0.2
    Applied before training and reversed after; hand-chosen normalization that metric-depth results depend on.
  • lambda_flow, lambda_cycle, lambda_SDF = 1, 1, 0.001
    Default weights in the initialization loss (Eq. 3); these balance supervisory signals and are set by hand.
  • number of bones B and RealNVP coupling layers = not stated in the text
    Bone hierarchy size and flow capacity are architectural choices that determine model capacity; exact values are not given.
assumptions (5)
  • domain assumption The neural SDF pre-fit, supervised by ground-truth stereo depth, provides a warm start that avoids local minima in Gaussian optimization.
    Section 3.2 and Appendix A.1; Table 5 shows removing this initialization causes a large drop in PSNR and depth accuracy, so the final result is contingent on this stage working.
  • domain assumption Skeleton-driven DQB warping explains roughly 90% of motion energy, leaving only small residual deformations for the flow.
    Section 3.1; this is asserted empirically and is load-bearing for the hierarchy argument.
  • domain assumption Off-the-shelf PoseNet, RAFT optical flow, UniDepth, and SAM masks provide sufficiently accurate supervision for optimization.
    Section 3.2 loss terms (3) and (4); the paper itself notes PoseNet-based poses can be less anatomically accurate than parametric models in the Limitations section.
  • ad hoc to paper The right-camera frames of a stereo pair are representative of embodied view synthesis.
    Appendix A.1 dataset details; the central extreme-viewpoint claim is supported quantitatively only by near-baseline stereo views.
  • standard math RealNVP flow is invertible and can represent residual non-rigid deformations.
    Standard normalizing flow architecture used in Section 3.1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of HoliGS: Holistic Gaussian Splatting for Embodied View Synthesis." pith.science (2026). https://pith.science/paper/VLCIZOMO

@misc{pith2026250619291,
  author       = {Pith},
  title        = {Pith review of: HoliGS: Holistic Gaussian Splatting for Embodied View Synthesis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VLCIZOMO}},
  note         = {Machine review of arXiv:2506.19291}
}
read the original abstract

We propose HoliGS, a novel deformable Gaussian splatting framework that addresses embodied view synthesis from long monocular RGB videos. Unlike prior 4D Gaussian splatting and dynamic NeRF pipelines, which struggle with training overhead in minute-long captures, our method leverages invertible Gaussian Splatting deformation networks to reconstruct large-scale, dynamic environments accurately. Specifically, we decompose each scene into a static background plus time-varying objects, each represented by learned Gaussian primitives undergoing global rigid transformations, skeleton-driven articulation, and subtle non-rigid deformations via an invertible neural flow. This hierarchical warping strategy enables robust free-viewpoint novel-view rendering from various embodied camera trajectories by attaching Gaussians to a complete canonical foreground shape (\eg, egocentric or third-person follow), which may involve substantial viewpoint changes and interactions between multiple actors. Our experiments demonstrate that \ourmethod~ achieves superior reconstruction quality on challenging datasets while significantly reducing both training and rendering time compared to state-of-the-art monocular deformable NeRFs. These results highlight a practical and scalable solution for EVS in real-world scenarios. The source code will be released.

Figures

Figures reproduced from arXiv: 2506.19291 by the authors.

Figure 1
Figure 1. Overview. From a phone capture of humans and animals in motion, HoliGS reconstructs temporally consistent geometry, appearance, and depth, enabling novel-view synthesis, deformable mesh recovery, and dense depth estimation. These reconstructions support a range of embodied applications, including actor-specific view synthesis (e.g., third-person and egocentric perspectives), object-specific removal, and actor-centri… view at source ↗
Figure 2
Figure 2. Performance of SOTA methods. Furthermore, several existing methods [6–8] rely heavily on off-the-shelf point-tracking models [5], introducing significant computational overhead and exhibiting fragility under severe occlusions. These methods also fail to generalize effectively to arbitrary viewpoint trajectories essential for comprehensive EVS scenarios, severely limiting their utility in real-world conditions marked… view at source ↗
Figure 3
Figure 3. HoliGS Pipeline. Left—Warping network initialization: We jointly optimize poses, articulation, soft deformation, and in a neural SDF proxy to obtain a fast converging deformation field that provides a strong starting point for Gaussian splitting. Right—after initialization, the objective is switched to dynamic Gaussian splatting, and the deformed foreground is composited with the static background to yield the final… view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Foreground Embodied Trajectory. For two challenging sequences, HumanCat and HumanDog, we show: (i) the joint bird’s-eye-view (BEV) trajectory of a foreground actor, (ii) the articulated animal trajectory, (iii) the articulated human trajectory, and (iv) both objects’ e…
Figure 5
Figure 5. Figure 5: Baseline Comparison. We qualitatively compare HoliGS against four SOTA baselines and a direct NVS ground-truth reference across Dog 1, Cat 1, Human 1, and the challenging multi-actor Human 2 & Cat sequences. Each column shows photometric renderings (top) and correspond…
Figure 6
Figure 6. Figure 6: Ablation Studies Visualization. Qualitative impact of removing soft deformation, normal/depth supervision, root–body initialization, and skeleton deformation. Each omission introduces increasing blur, drift, and silhouette break-up, whereas the full model remains sharp…
Figure 7
Figure 7. Figure 7: NVS comparisons with Total-Recon. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: NVS comparisons with Total-Recon. 17 [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: NVS comparisons with Total-Recon. 18 [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: NVS comparisons with Total-Recon. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]
Figure 11
Figure 11. Figure 11: NVS comparisons with Total-Recon. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MemoBench: Benchmarking World Modeling in Dynamically Changing Environments

    cs.CV 2026-06 unverdicted novelty 7.0 of 10

    None of ten tested video-generation models reliably remembers objects after occlusion in dynamic scenes; static-camera videos inflate consistency scores.

Reference graph

Works this paper leans on

73 extracted references · 57 canonical work pages · cited by 1 Pith paper

  1. [1]

    Nerf: Representing scenes as neural radiance fields for view synthesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoor- thi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 2021. 1

    work page 2021

  2. [2]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM TOG, 2023. 1, 6

    work page 2023

  3. [3]

    Total- recon: Deformable scene reconstruction for embodied view synthesis

    Chonghyuk Song, Gengshan Yang, Kangle Deng, Jun-Yan Zhu, and Deva Ramanan. Total- recon: Deformable scene reconstruction for embodied view synthesis. In ICCV, 2023. 1, 2, 3, 7

    work page 2023

  4. [4]

    Modgs: Dynamic gaussian splatting from casually-captured monocular videos

    Qingming Liu, Yuan Liu, Jiepeng Wang, Xianqiang Lyv, Peng Wang, Wenping Wang, and Junhui Hou. Modgs: Dynamic gaussian splatting from casually-captured monocular videos. In ICLR, 2025. 1

    work page 2025

  5. [5]

    Cotracker3: Simpler and better point tracking by pseudo-labelling real videos

    Nikita Karaev, Iurii Makarov, Jianyuan Wang, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Cotracker3: Simpler and better point tracking by pseudo-labelling real videos. arXiv preprint arXiv:2410.11831, 2024. 1, 2

    arXiv 2024

  6. [6]

    Shape of motion: 4d reconstruction from a single video

    Qianqian Wang, Vickie Ye, Hang Gao, Jake Austin, Zhengqi Li, and Angjoo Kanazawa. Shape of motion: 4d reconstruction from a single video. arXiv preprint arXiv:2407.13764, 2024. 2

    arXiv 2024

  7. [7]

    Mosca: Dynamic gaussian fusion from casual videos via 4d motion scaffolds

    Jiahui Lei, Yijia Weng, Adam Harley, Leonidas Guibas, and Kostas Daniilidis. Mosca: Dynamic gaussian fusion from casual videos via 4d motion scaffolds. arXiv preprint arXiv:2405.17421, 2024

    arXiv 2024

  8. [8]

    Dynamic gaussian marbles for novel view synthesis of casual monocular videos

    Colton Stearns, Adam Harley, Mikaela Uy, Florian Dubost, Federico Tombari, Gordon Wet- zstein, and Leonidas Guibas. Dynamic gaussian marbles for novel view synthesis of casual monocular videos. In SIGGRAPH Asia, 2024. 2

    work page 2024

Show all 73 references
  1. [9]

    Ppr: Physically plausible reconstruction from monocular videos

    Gengshan Yang, Shuo Yang, John Z Zhang, Zachary Manchester, and Deva Ramanan. Ppr: Physically plausible reconstruction from monocular videos. In ICCV, 2023. 2

    2023

  2. [10]

    Lawrence Zitnick, Sing Bing Kang, Matthew Uyttendaele, Simon Winder, and Richard Szeliski

    C. Lawrence Zitnick, Sing Bing Kang, Matthew Uyttendaele, Simon Winder, and Richard Szeliski. High-quality video view interpolation using a layered representation. ACM TOG,

  3. [11]

    View and time interpo- lation in image space

    Timo Stich, Christian Linz, Georgia Albuquerque, and Marcus Magnor. View and time interpo- lation in image space. Computer Graphics Forum, 2008

    2008

  4. [12]

    X-fields: Implicit neural view-, light- and time-image interpolation

    Mojtaba Bemana, Karol Myszkowski, Hans-Peter Seidel, and Tobias Ritschel. X-fields: Implicit neural view-, light- and time-image interpolation. In SIGGRAPH Asia, 2020

    2020

  5. [13]

    4d visualization of dynamic events from unconstrained multi-view videos

    Aayush Bansal, Minh V o, Yaser Sheikh, Deva Ramanan, and Srinivasa Narasimhan. 4d visualization of dynamic events from unconstrained multi-view videos. In CVPR, 2020

    2020

  6. [14]

    Neural 3d video synthesis from multi-view video

    Tianye Li, Mira Slavcheva, Michael Zollhoefer, Simon Green, Christoph Lassner, Changil Kim, Tanner Schmidt, Steven Lovegrove, Michael Goesele, Richard Newcombe, et al. Neural 3d video synthesis from multi-view video. In CVPR, 2022

    2022

  7. [15]

    D-nerf: Neural radiance fields for dynamic scenes

    Albert Pumarola, Enric Corona, Gerard Pons-Moll, and Francesc Moreno-Noguer. D-nerf: Neural radiance fields for dynamic scenes. In CVPR, 2021

    2021

  8. [16]

    K-planes: Explicit radiance fields in space, time, and appearance

    Sara Fridovich-Keil, Giacomo Meanti, Frederik Warburg, Benjamin Recht, and Angjoo Kanazawa. K-planes: Explicit radiance fields in space, time, and appearance. arXiv preprint arXiv:2301.10241, 2023

    2023 arXiv

  9. [17]

    Hexplane: A fast representation for dynamic scenes

    Ang Cao and Justin Johnson. Hexplane: A fast representation for dynamic scenes. arXiv preprint arXiv:2301.09632, 2023. 3

    2023 arXiv

  10. [18]

    Hyperreel: High-fidelity 6-dof video with ray-conditioned sampling

    Benjamin Attal, Jia-Bin Huang, Christian Richardt, Michael Zollhoefer, Johannes Kopf, Matthew O’Toole, and Changil Kim. Hyperreel: High-fidelity 6-dof video with ray-conditioned sampling. In CVPR, 2023. 10

    2023

  11. [19]

    Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis

    Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis. arXiv preprint arXiv:2308.09713, 2023

    2023 arXiv

  12. [20]

    High- fidelity and real-time novel view synthesis for dynamic scenes

    Haotong Lin, Sida Peng, Zhen Xu, Tao Xie, Xingyi He, Hujun Bao, and Xiaowei Zhou. High- fidelity and real-time novel view synthesis for dynamic scenes. In SIGGRAPH Asia, 2023. 3

    2023

  13. [21]

    Novel view synthesis of dynamic scenes with globally coherent depths from a monocular camera

    Jae Shin Yoon, Kihwan Kim, Orazio Gallo, Hyun Soo Park, and Jan Kautz. Novel view synthesis of dynamic scenes with globally coherent depths from a monocular camera. In CVPR, 2020. 3

    2020

  14. [22]

    Neural scene flow fields for space-time view synthesis of dynamic scenes

    Zhengqi Li, Simon Niklaus, Noah Snavely, and Oliver Wang. Neural scene flow fields for space-time view synthesis of dynamic scenes. In CVPR, 2021. 3

    2021

  15. [23]

    Space-time neural irradiance fields for free-viewpoint video

    Wenqi Xian, Jia-Bin Huang, Johannes Kopf, and Changil Kim. Space-time neural irradiance fields for free-viewpoint video. In CVPR, 2021

    2021

  16. [24]

    Neural trajectory fields for dynamic novel view synthesis

    Chaoyang Wang, Ben Eckart, Simon Lucey, and Orazio Gallo. Neural trajectory fields for dynamic novel view synthesis. arXiv preprint arXiv:2105.05994, 2021

    2021 arXiv

  17. [25]

    Dynamic view synthesis from dynamic monocular video

    Chen Gao, Ayush Saraf, Johannes Kopf, and Jia-Bin Huang. Dynamic view synthesis from dynamic monocular video. In ICCV, 2021

    2021

  18. [26]

    Non-rigid neural radiance fields: Reconstruction and novel view synthesis of a dynamic scene from monocular video

    Edgar Tretschk, Ayush Tewari, Vladislav Golyanik, Michael Zollhöfer, Christoph Lassner, and Christian Theobalt. Non-rigid neural radiance fields: Reconstruction and novel view synthesis of a dynamic scene from monocular video. In ICCV, 2021

    2021

  19. [27]

    D2 nerf: Self-supervised decoupling of dynamic and static objects from a monocular video

    Tianhao Wu, Fangcheng Zhong, Andrea Tagliasacchi, Forrester Cole, and Cengiz Oztireli. D2 nerf: Self-supervised decoupling of dynamic and static objects from a monocular video. arXiv preprint arXiv:2205.15838, 2022

    2022 arXiv

  20. [28]

    Nerfplayer: A streamable dynamic scene representation with decomposed neural radiance fields

    Liangchen Song, Anpei Chen, Zhong Li, Zhang Chen, Lele Chen, Junsong Yuan, Yi Xu, and Andreas Geiger. Nerfplayer: A streamable dynamic scene representation with decomposed neural radiance fields. IEEE TVCG, 2023

    2023

  21. [29]

    Mononerf: Learning a generalizable dynamic radiance field from monocular videos

    Fengrui Tian, Shaoyi Du, and Yueqi Duan. Mononerf: Learning a generalizable dynamic radiance field from monocular videos. In ICCV, 2023

    2023

  22. [30]

    Decoupling dynamic monocular videos for dynamic view synthesis

    Meng You and Junhui Hou. Decoupling dynamic monocular videos for dynamic view synthesis. arXiv preprint arXiv:2304.01716, 2023

    2023 arXiv

  23. [31]

    De- formable 3d gaussians for high-fidelity monocular dynamic scene reconstruction

    Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. De- formable 3d gaussians for high-fidelity monocular dynamic scene reconstruction. arXiv preprint arXiv:2309.13101, 2023

    2023 arXiv

  24. [32]

    Gaufre: Gaussian deformation fields for real-time dynamic novel view synthesis

    Yiqing Liang, Numair Khan, Zhengqin Li, Thu Nguyen-Phuoc, Douglas Lanman, James Tompkin, and Lei Xiao. Gaufre: Gaussian deformation fields for real-time dynamic novel view synthesis. arXiv preprint arXiv:2312.11458, 2023

    2023 arXiv

  25. [33]

    Robust dynamic radiance fields

    Yu-Lun Liu, Chen Gao, Andreas Meuleman, Hung-Yu Tseng, Ayush Saraf, Changil Kim, Yung-Yu Chuang, Johannes Kopf, and Jia-Bin Huang. Robust dynamic radiance fields. In CVPR, 2023

    2023

  26. [34]

    Dyblurf: Dynamic deblurring neural radiance fields for blurry monocular video

    Minh-Quan Viet Bui, Jongmin Park, Jihyong Oh, and Munchurl Kim. Dyblurf: Dynamic deblurring neural radiance fields for blurry monocular video. arXiv preprint arXiv:2312.13528, 2023

    2023 arXiv

  27. [35]

    Ctnerf: Cross-time transformer for dynamic neural radiance field from monocular video

    Xingyu Miao, Yang Bai, Haoran Duan, Yawen Huang, Fan Wan, Yang Long, and Yefeng Zheng. Ctnerf: Cross-time transformer for dynamic neural radiance field from monocular video. arXiv preprint arXiv:2401.04861, 2024. 3

    2024 arXiv

  28. [36]

    Barron, Sofien Bouaziz, Dan B Goldman, Ricardo Martin-Brualla, and Steven M

    Keunhong Park, Utkarsh Sinha, Peter Hedman, Jonathan T. Barron, Sofien Bouaziz, Dan B Goldman, Ricardo Martin-Brualla, and Steven M. Seitz. Hypernerf: A higher-dimensional representation for topologically varying neural radiance fields. ACM TOG, 2021. 3 11

    2021

  29. [37]

    Nerfies: Deformable neural radiance fields

    Keunhong Park, Utkarsh Sinha, Jonathan T Barron, Sofien Bouaziz, Dan B Goldman, Steven M Seitz, and Ricardo Martin-Brualla. Nerfies: Deformable neural radiance fields. In ICCV, 2021. 3

    2021

  30. [38]

    Plenoxels: Radiance fields without neural networks

    Sara Fridovich-Keil and Alex Yu, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa. Plenoxels: Radiance fields without neural networks. In CVPR, 2022. 3

    2022

  31. [39]

    Tensorf: Tensorial radiance fields

    Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, and Hao Su. Tensorf: Tensorial radiance fields. In ECCV, 2022. 3

    2022

  32. [40]

    Fast dynamic radiance fields with time-aware neural voxels

    Jiemin Fang, Taoran Yi, Xinggang Wang, Lingxi Xie, Xiaopeng Zhang, Wenyu Liu, Matthias Nießner, and Qi Tian. Fast dynamic radiance fields with time-aware neural voxels. In SIG- GRAPH Asia, 2022. 3

    2022

  33. [41]

    Tensor4d: Efficient neural 4d decomposition for high-fidelity dynamic reconstruction and rendering

    Ruizhi Shao, Zerong Zheng, Hanzhang Tu, Boning Liu, Hongwen Zhang, and Yebin Liu. Tensor4d: Efficient neural 4d decomposition for high-fidelity dynamic reconstruction and rendering. In CVPR, 2023. 3

    2023

  34. [42]

    Monocular dynamic view synthesis: A reality check

    Hang Gao, Ruilong Li, Shubham Tulsiani, Bryan Russell, and Angjoo Kanazawa. Monocular dynamic view synthesis: A reality check. In NeurIPS, 2022. 3

    2022

  35. [43]

    Dynibar: Neural dynamic image-based rendering

    Zhengqi Li, Qianqian Wang, Forrester Cole, Richard Tucker, and Noah Snavely. Dynibar: Neural dynamic image-based rendering. In CVPR, 2023. 3

    2023

  36. [44]

    Smpl: A skinned multi-person linear model

    Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J Black. Smpl: A skinned multi-person linear model. ACM TOG, 2023. 3

    2023

  37. [45]

    Silvia Zuffi, Angjoo Kanazawa, David Jacobs, and Michael J. Black. 3D menagerie: Modeling the 3D shape and pose of animals. In CVPR, 2017

    2017

  38. [46]

    3d bird reconstruction: a dataset, model, and shape recovery from a single view

    Marc Badger, Yufu Wang, Adarsh Modh, Ammon Perkes, Nikos Kolotouros, Bernd G Pfrommer, Marc F Schmidt, and Kostas Daniilidis. 3d bird reconstruction: a dataset, model, and shape recovery from a single view. In ECCV, 2020

    2020

  39. [47]

    Bite: Beyond priors for improved three-d dog pose estimation

    Nadine Rüegg, Shashank Tripathi, Konrad Schindler, Michael J Black, and Silvia Zuffi. Bite: Beyond priors for improved three-d dog pose estimation. In CVPR, 2023. 3

    2023

  40. [48]

    End-to-end recovery of human shape and pose

    Angjoo Kanazawa, Michael J Black, David W Jacobs, and Jitendra Malik. End-to-end recovery of human shape and pose. In CVPR, 2018. 3

    2018

  41. [49]

    Learning to reconstruct 3d human pose and shape via model-fitting in the loop

    Nikos Kolotouros, Georgios Pavlakos, Michael J Black, and Kostas Daniilidis. Learning to reconstruct 3d human pose and shape via model-fitting in the loop. In ICCV, 2019

    2019

  42. [50]

    Humans in 4d: Reconstructing and tracking humans with transformers

    Shubham Goel, Georgios Pavlakos, Jathushan Rajasegaran, Angjoo Kanazawa, and Jitendra Malik. Humans in 4d: Reconstructing and tracking humans with transformers. arXiv preprint arXiv:2305.20091, 2023

    2023 arXiv

  43. [51]

    Refit: Recurrent fitting network for 3d human recovery

    Yufu Wang and Kostas Daniilidis. Refit: Recurrent fitting network for 3d human recovery. In ICCV, 2023. 3

    2023

  44. [52]

    Neural body: Implicit neural representations with structured latent codes for novel view synthesis of dynamic humans

    Sida Peng, Yuanqing Zhang, Yinghao Xu, Qianqian Wang, Qing Shuai, Hujun Bao, and Xiaowei Zhou. Neural body: Implicit neural representations with structured latent codes for novel view synthesis of dynamic humans. In CVPR, 2021. 3

    2021

  45. [53]

    Neural actor: Neural free-view synthesis of human actors with pose control

    Lingjie Liu, Marc Habermann, Viktor Rudnev, Kripasindhu Sarkar, Jiatao Gu, and Christian Theobalt. Neural actor: Neural free-view synthesis of human actors with pose control. ACM TOG, 2021

    2021

  46. [54]

    Humannerf: Free-viewpoint rendering of moving people from monocular video

    Chung-Yi Weng, Brian Curless, Pratul P Srinivasan, Jonathan T Barron, and Ira Kemelmacher- Shlizerman. Humannerf: Free-viewpoint rendering of moving people from monocular video. In CVPR, 2022

    2022

  47. [55]

    Snarf: Differ- entiable forward skinning for animating non-rigid neural implicit shapes

    Xu Chen, Yufeng Zheng, Michael J Black, Otmar Hilliges, and Andreas Geiger. Snarf: Differ- entiable forward skinning for animating non-rigid neural implicit shapes. In ICCV, 2021. 12

    2021

  48. [56]

    Fast-snarf: A fast deformer for articulated neural fields

    Xu Chen, Tianjian Jiang, Jie Song, Max Rietmann, Andreas Geiger, Michael J Black, and Otmar Hilliges. Fast-snarf: A fast deformer for articulated neural fields. IEEE TPAMI, 2023

    2023

  49. [57]

    Banmo: Building animatable 3d neural models from many casual videos

    Gengshan Yang, Minh V o, Natalia Neverova, Deva Ramanan, Andrea Vedaldi, and Hanbyul Joo. Banmo: Building animatable 3d neural models from many casual videos. In CVPR, 2022. 5

    2022

  50. [58]

    Reconstructing animatable categories from videos

    Gengshan Yang, Chaoyang Wang, N Dinesh Reddy, and Deva Ramanan. Reconstructing animatable categories from videos. In CVPR, 2023

    2023

  51. [59]

    Vidu4d: Single generated video to high-fidelity 4d reconstruction with dynamic gaussian surfels

    Yikai Wang, Xinzhou Wang, Zilong Chen, Zhengyi Wang, Fuchun Sun, and Jun Zhu. Vidu4d: Single generated video to high-fidelity 4d reconstruction with dynamic gaussian surfels. In NeurIPS, 2024. 3

    2024

  52. [60]

    Structure-from-motion revisited

    Johannes Lutz Schönberger and Jan-Michael Frahm. Structure-from-motion revisited. In CVPR,

  53. [61]

    ORB-SLAM: a versatile and accurate monocular SLAM system

    Raul Mur-Artal, Jose Maria Martinez Montiel, and Juan D Tardos. ORB-SLAM: a versatile and accurate monocular SLAM system. IEEE Transactions on Robotics, 2015

    2015

  54. [62]

    DROID-SLAM: Deep visual SLAM for monocular, stereo, and RGB-D cameras

    Zachary Teed and Jia Deng. DROID-SLAM: Deep visual SLAM for monocular, stereo, and RGB-D cameras. In NeurIPS, 2021. 3

    2021

  55. [63]

    Robust consistent video depth estimation

    Johannes Kopf, Xuejian Rong, and Jia-Bin Huang. Robust consistent video depth estimation. In CVPR, 2021. 3

    2021

  56. [64]

    Zhoutong Zhang, Forrester Cole, Zhengqi Li, Michael Rubinstein, Noah Snavely, and William T. Freeman. Structure and motion from casual videos. In ECCV, 2022. 3

    2022

  57. [65]

    Skinning with dual quaternions

    Ladislav Kavan, Steven Collins, Jiˇrí Žára, and Carol O’Sullivan. Skinning with dual quaternions. SI3D, 2007. 4

    2007

  58. [66]

    Density estimation using real nvp

    Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using real nvp. arXiv preprint arXiv:1605.08803, 2016. 4

    2016 arXiv

  59. [67]

    Tracking everything everywhere all at once

    Qianqian Wang, Yen-Yu Chang, Ruojin Cai, Zhengqi Li, Bharath Hariharan, Aleksander Holynski, and Noah Snavely. Tracking everything everywhere all at once. In ICCV, 2023. 4

    2023

  60. [68]

    Posenet: A convolutional network for real-time 6-dof camera relocalization

    Alex Kendall, Matthew Grimes, and Roberto Cipolla. Posenet: A convolutional network for real-time 6-dof camera relocalization. In ICCV, 2015. 5

    2015

  61. [69]

    Unidepth: Universal monocular metric depth estimation

    Luigi Piccinelli, Yung-Hsu Yang, Christos Sakaridis, Mattia Segu, Siyuan Li, Luc Van Gool, and Fisher Yu. Unidepth: Universal monocular metric depth estimation. In CVPR, 2024. 5

    2024

  62. [70]

    Raft: Recurrent all-pairs field transforms for optical flow

    Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In ECCV,

  63. [71]

    Sam 2: Segment anything in images and videos

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.00714, 2024. 5

    2024 arXiv

  64. [72]

    Deformable 3d gaussians for high-fidelity monocular dynamic scene reconstruction

    Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. Deformable 3d gaussians for high-fidelity monocular dynamic scene reconstruction. In CVPR, 2024. 7

    2024

  65. [73]

    4d gaussian splatting for real-time dynamic scene rendering

    Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4d gaussian splatting for real-time dynamic scene rendering. In CVPR, 2024. 7 13 A Appendix / supplemental material A.1 Implementation Details Data Preprocessing All...

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.