{"id":"fcedca69-4565-4a3d-9510-7a4a23d8bd8a","arxiv_id":"2506.19291","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A deformable Gaussian splatting framework with hierarchical rigid, skeleton-driven, and flow-based warping reconstructs dynamic scenes from long video captures with fast training and rendering.","lead":"HoliGS turns long videos of moving people and animals into interactive 3D scenes that can be viewed from new angles. The method splits a scene into static background and moving objects, then uses three layers of motion (rigid, bone-like, and soft flow) to keep the reconstruction stable and fast.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The monocular claim is contradicted by the use of stereo ground-truth depth and ARKit poses in initialization; ablations show these are essential, so the core claim is not established for genuine monocular video.","rationale":"After reading the paper, I find the reader's weakest assumption to be accurate and, if anything, the most critical issue. The paper's abstract and contributions consistently describe the method as operating on 'long monocular RGB videos,' yet the training pipeline relies on stereo ground-truth depth for surface-point sampling and ARKit poses for background initialization (Appendix A.1). The ablations in Table 4 demonstrate that these signals are load-bearing: removing depth supervision lowers Acc@0.1m from .896 to .847 and raises LPIPS from .263 to .385; removing root-body poses entirely yields N/A. This is not a minor robustness issue but a fundamental dependence on signals that are absent in true monocular video. Additionally, the only quantitative novel-view evaluation uses the right stereo camera, a small baseline that does not test the embodied viewpoints (egocentric/third-person follow) that motivate the work. Without a monocular-only experiment and wide-baseline evaluation, the central claim is unverified. The reader's CONDITIONAL verdict with high correctness risk is appropriate; I would not change it. My concrete test—rerunning with COLMAP/DROID-SLAM poses and UniDepth-only depth—would directly determine whether the monocular claim holds.","tokens_in":15270,"tokens_out":8837,"duration_ms":91866,"concrete_test":"Retrain HoliGS on the same 11 sequences but remove ARKit camera poses (use COLMAP or DROID-SLAM poses instead) and remove stereo ground-truth depth for surface sampling (use only UniDepth monocular depth). If PSNR on the right-camera validation drops by more than ~2 dB or training fails to converge on any sequence, the method's monocular claim is refuted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of embodied view synthesis from long monocular RGB videos (Abstract) is not supported by the experimental setup. In §3.2 and Appendix A.1, deformation-network initialization samples points centered at the surface point derived from back-projecting ground-truth depth from a stereo dataset, and root/background poses are initialized from PoseNet and ARKit motion sensors. Table 4 (rows 2, 6, 7) and Table 5 show that removing depth supervision, root-body initialization, or root-body poses causes significant degradation, and removing root-body poses entirely prevents convergence (N/A). Thus the method's reported success depends on reliable depth and pose signals that a genuine monocular capture may not provide. Furthermore, the quantitative NVS evaluation (Table 2) uses the right-camera stereo pair as held-out novel views—a small-baseline view that does not exercise the extreme egocentric or third-person-follow trajectories central to EVS; those are only shown qualitatively (Figure 4) without metrics. Consequently, the method's superiority for monocular input and embodied viewpoints is not quantitatively established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes HoliGS, a deformable Gaussian splatting framework for embodied view synthesis of long dynamic scenes. The scene is decomposed into a static background and time-varying foreground objects; motion is modeled hierarchically as global SE(3) transformations, skeleton-driven dual-quaternion blend skinning, and a residual invertible RealNVP soft-deformation field. A neural-SDF warm-start initializes the deformation network and Gaussian positions, after which a two-stage optimization (component pre-training and joint refinement) fits the full 4D representation. Experiments are conducted on a newly captured stereo dataset of 11 sequences of humans interacting with animals. The paper reports higher LPIPS/PSNR/SSIM and depth-accuracy numbers than dynamic NeRF and Gaussian baselines, as well as substantially lower training and rendering times.","tokens_in":15479,"tokens_out":3720,"duration_ms":43425,"significance":"If the stated claims are accepted, the paper would be a practical engineering contribution: it combines articulated motion priors with an invertible deformation model, avoids costly point-tracking pipelines, scales to minute-long captures, and reports 20 fps rendering on consumer hardware. The ablation study in Table 4 supports the value of the individual components, and the appendix provides useful implementation details. The main reservations concern the alignment between the claims in the abstract and the evaluation protocol: the method is demonstrated on stereo data with ground-truth depth and ARKit/PoseNet pose initialization, and the quantitative novel-view metrics are computed on right-stereo frames rather than on the egocentric or third-person trajectories that the paper identifies as the embodied-view setup. These gaps mean the headline claims of monocular input and embodied view synthesis are not currently established by the evidence presented.","major_comments":[{"comment":"The central claim that HoliGS works from long monocular RGB videos is not supported by the experimental setup. The deformation-network initialization samples points centered on the surface point derived from back-projecting ground-truth depth (Appendix A.1), and the background root pose is initialized from ARKit motion-sensor extrinsics while foreground poses come from PoseNet (§3.2). Table 4 shows that removing the depth loss (row 2) substantially degrades accuracy and that removing root-body poses entirely (row 7) prevents convergence (N/A); Table 5 shows that removing the initialization stage drops PSNR from 21.31 to 17.30. These results demonstrate that the method's success depends on stereo-derived depth and accurate pose priors, which are not available in a genuine monocular RGB capture. The authors should either evaluate on true monocular sequences with estimated depth and pose, or substantially reframe the claims to describe a stereo/pose-assisted system.","section":"§3.2, Appendix A.1, Tables 4 and 5"},{"comment":"The embodied-view-synthesis claim is not quantitatively validated. The held-out validation views in the quantitative NVS evaluation are the right-camera frames of a stereo pair, which have only a small baseline relative to the training views. The egocentric and third-person-follow trajectories that define EVS are only shown qualitatively in Figure 4, without metrics. This does not demonstrate robust free-viewpoint rendering under the extreme viewpoint changes claimed in the abstract and Table 1. The paper should add quantitative evaluation on embodied trajectories (or a proxy such as large-baseline re-rendering with ground-truth depth) and report numbers for those viewpoints, or restrict the EVS claims to what the current protocol supports.","section":"Appendix A.1, Table 2, Figure 4"}],"minor_comments":[{"comment":"The abstract repeatedly states the input is monocular RGB video, but the dataset and initialization rely on stereo depth and device motion sensors. The wording should be adjusted to reflect the actual input modalities or the experiments should be extended to include monocular captures.","section":"Abstract and §1"},{"comment":"The row references in the text do not match the table. The caption says removing root-body poses entirely is row (5), but the table shows that row is the soft-deformation ablation and row (7) is the root-body-pose removal. Several other row citations in §4.3 are likewise inconsistent with the table. These references should be corrected.","section":"Table 4 caption and §4.3"},{"comment":"The notation in Eq. (2) appears unbalanced: the left-hand side is a canonical point X*, while the right-hand side applies S, J_t, and G_t^o to a world-space point X^t. If this is intended as the forward warp, the inverse flow should be used or the notation should be clarified.","section":"Eq. (1) and Eq. (2)"},{"comment":"The description of the loss weights is inconsistent: the text states default weights for λ_depth = 5 or 1.5 for HUMAN 1, but it is not stated whether these per-sequence weights are tuned on the validation set or fixed a priori. A brief statement about weight selection would improve reproducibility.","section":"§4.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid systems contribution, but the evaluation does not match the headline claims. The stereo-depth and ARKit/PoseNet dependence is not a minor caveat; it is load-bearing for the monocular claim, and the right-stereo validation does not exercise the embodied-view trajectories that define EVS. I would be willing to consider a revised version that either adds quantitative evaluation on truly monocular inputs and embodied viewpoint trajectories, or clearly reframes the contribution and removes the unsupported claims. The Table 4 row-numbering inconsistency and the Eq. (2) notation issue should also be fixed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague—\n\nWhat you should know up front: this is a real systems/engineering contribution, but the 'monocular' framing is not supported by the experiments. The hierarchical decomposition (global SE(3), dual-quaternion blend skinning, RealNVP soft flow) is sensible, the ablations show each stage earns its place, and the reported speedup over Total-Recon is substantial. The neural-SDF warm start is a practical trick worth stealing.\n\nThe soft spots: (1) initialization requires stereo ground-truth depth and ARKit/device poses. Section 3.2 and Appendix A.1 say ray samples are centered at surface points back-projected from ground-truth depth, and the background root pose comes from ARKit. Table 4 shows removing depth supervision significantly hurts, removing root-body init hurts, and removing root-body poses entirely gives N/A. So the method works with strong pose+depth priors, not from a casual monocular phone capture. The authors should either reframe the claim or run a genuine monocular variant and report the drop. (2) The quantitative novel-view evaluation uses the right stereo camera as held-out views—a small baseline. The egocentric/third-person embodied trajectories, which are the paper's reason for existing, are only qualitative (Figure 4). No metrics on large viewpoint changes. That is a clear gap between claim and evidence. (3) Code and data aren't available yet; per-sequence tuning of lambda_depth is minor but worth noting.\n\nCredit where due: the approach is well-motivated, the math is coherent, and the ablations are honest about what hurts. This is not a sloppy paper. It is a solid depth/pose-assisted dynamic Gaussian splatting system with an oversold headline. I would send it to a serious referee, with the expectation of major revision: reframe the monocular claim, add large-baseline or held-out-trajectory metrics, and release code/data. If those revisions land, it becomes a genuinely useful reference for dynamic scene reconstruction.","headline":"Solid deformable-Gaussian system with a useful hierarchy and honest ablations, but the monocular and embodied-view claims outrun the evaluation: initialization depends on stereo depth and ARKit poses, and the held-out views are the small-baseline right stereo camera.","tokens_in":16060,"tokens_out":2841,"would_cite":false,"duration_ms":32511,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Deformable Gaussian splatting reconstructs minute-long dynamic scenes from monocular video about ten times faster than prior neural radiance field pipelines while improving rendering quality.","keywords":["Gaussian splatting","embodied view synthesis","dynamic scene reconstruction","deformable neural rendering","articulated objects","monocular video","invertible neural flow","4D reconstruction"],"falsifier":"Run HoliGS on a monocular video with no paired depth and no external pose initialization. If the novel-view renderings reproduce the floating-foreground and scale-drift artifacts shown in the 'w/o L_depth' ablation (Table 4, row 2) or fail to converge as in the 'w/o root-body G_j' row, then the central claim of monocular embodied view synthesis is not sustained.","tokens_in":15045,"feed_emoji":"🎥","tokens_out":10309,"duration_ms":90359,"temperature":0.7,"pith_summary":"HoliGS is a deformable Gaussian splatting system for embodied view synthesis from long monocular RGB videos. The paper set out to show that dynamic scenes with multiple interacting humans and animals, captured over hundreds of frames with large viewpoint changes, can be reconstructed into temporally consistent geometry, appearance, and depth and rendered at interactive speeds. The central move is to decompose scene motion into a hierarchy: a rigid global transform, skeleton-driven articulated skinning, and an invertible flow that absorbs residual non-rigid deformation. A neural-SDF pre-training stage gives the Gaussians a good starting pose and scale. If the paper is right, minute-long phone captures become practical free-viewpoint 4D scenes, supporting egocentric and third-person-view replay, object removal, and dense depth estimation.","feed_headline":"Gaussian splatting reconstructs minute-long dynamic scenes 10× faster","feed_subtitle":"New deformable Gaussian framework turns phone captures of humans and animals into interactive free-viewpoint 4D replay.","key_machinery":"The load-bearing object is the invertible deformation pipeline that combines three warping stages: a per-frame rigid SE(3) transform for the object root and background, dual-quaternion blend skinning over a learned bone hierarchy, and an invertible RealNVP flow for soft deformation. Because the flow is invertible, the model enforces a cycle-consistency loss between forward and inverse warps, which stabilizes training over long sequences. A second piece of machinery is the neural-SDF warm start: sampled points are centered on back-projected surface points from depth, and the warping network is pre-trained on photometric, depth, flow, cycle, and segmentation losses before dynamic Gaussian splatting takes over. Together these let the representation explain roughly ninety percent of visible motion with the skeleton, leaving the invertible flow to handle only low-amplitude residual detail, which is why the paper claims convergence in about two hours on a single GPU.","core_discovery":"On its own terms, the paper claims that HoliGS outperforms state-of-the-art monocular deformable NeRFs on both reconstruction quality and speed for embodied view synthesis. The claim is supported by experiments on 11 stereo-captured sequences of humans and animals: the method reports the best LPIPS, PSNR, SSIM, and depth accuracy across all sequences, cutting training time to about 1.2 hours versus roughly 12 hours for Total-Recon on the same GPU, and rendering novel views at about 20 fps on consumer hardware. The mechanism the paper credits for this is the hierarchical warping decomposition, in which a canonical point is warped through the inverse soft flow, the skeleton skinning, and the object-root camera transform as $X_t = {G_t^o}^{-1} {J_t}^{-1} S^{-1}(X^*, \\omega_t^d)$, together with a two-stage optimization that first fits a neural SDF to initialize poses and scale, then switches to Gaussian splatting for final refinement.","pith_inferences":["The paper's experiments use stereo-captured depth and device motion sensors for initialization, so its monocular claim is not yet demonstrated on a video without any depth signal; a natural next test is to run it on a standard phone clip and check whether the scale drift seen in the no-depth ablation reappears.","Because the skeleton is learned without parametric body models, the method trades anatomical precision for generality; combining the learned bones with a parametric human or animal prior might benefit the human sequences while keeping the approach applicable to arbitrary articulated objects.","The invertible flow formulation is general enough that the same three-way decomposition could in principle be applied to other articulated entities such as furniture or robots, though the paper only evaluates humans and animals.","The reported speed-up depends on the warm-start initialization; without it, performance drops sharply (Appendix Table 5), so the practical benefit may shrink for sequences where reliable depth and pose are unavailable."],"forward_implications":["Minute-long monocular videos of dynamic scenes become tractable: full scenes with multiple deformable objects converge in about two hours on an H20 or A6000 GPU, roughly ten times faster than the strongest NeRF baseline.","Novel views from embodied trajectories, including egocentric, third-person follow, and overhead perspectives, can be rendered at interactive rates of about 20 fps, making AR/VR and virtual cinematography practical.","The same representation yields metric depth and deformable mesh recovery, enabling actor-specific view synthesis, object removal, and spatiotemporal trajectory analysis.","The hierarchical motion decomposition removes the need for expensive point-tracking pipelines and helps the optimization stay stable under occlusion and extreme viewpoint changes."],"supporting_citations":[{"why":"Supplies the base differentiable Gaussian rendering primitive on which the entire framework is built.","marker":"[2]"},{"why":"Defines the embodied-view-synthesis benchmark and is the primary baseline whose quality and training time HoliGS must beat.","marker":"[3]"},{"why":"Provides the articulated canonical-space reconstruction and neural-SDF warm-start strategy that HoliGS adapts for initialization.","marker":"[57]"},{"why":"Dual quaternion skinning is the technique used to fuse bone transforms into a single SE(3) warp.","marker":"[65]"},{"why":"RealNVP gives the invertible flow architecture used for the soft deformation field and the cycle-consistency loss.","marker":"[66]"},{"why":"PoseNet supplies the initial rigid-body and object-root pose estimates used in the two-stage optimization.","marker":"[68]"},{"why":"UniDepth is the off-the-shelf monocular depth estimator used in the depth-supervision loss that the ablations show is critical.","marker":"[69]"},{"why":"RAFT optical flow is used to compute the flow loss that enforces motion consistency during initialization.","marker":"[70]"},{"why":"SAM 2 provides the segmentation masks used by the mask-supervision term in the loss.","marker":"[71]"}],"fun_headline_variants":["HoliGS: 10x faster dynamic scene reconstruction with Gaussian splatting","Deformable Gaussians cut training time 10x for embodied view synthesis","HoliGS turns minute-long monocular videos into 4D replay at 20fps","Hierarchical Gaussian warping beats deformable NeRFs in speed and quality"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method needs trustworthy depth and body/camera pose estimates during initialization; if a video lacks those, as a truly monocular phone capture would, the scale consistency and stable convergence the paper reports are not established.","fun_headline_variants_meta":{"raw":{"variants":["HoliGS: 10x faster dynamic scene reconstruction with Gaussian splatting","Deformable Gaussians cut training time 10x for embodied view synthesis","HoliGS turns minute-long monocular videos into 4D replay at 20fps","Hierarchical Gaussian warping beats deformable NeRFs in speed and quality"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000736,"raw_usage":{"total_tokens":3298,"prompt_tokens":959,"completion_tokens":2339,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":575,"completion_tokens_details":{"reasoning_tokens":2263}},"tokens_in":575,"tokens_out":2339,"duration_ms":16780,"temperature":1.0,"reasoning_tokens":2263,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T23:07:09.020605+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run HoliGS on a monocular video with no paired depth and no external pose initialization. If the novel-view renderings reproduce the floating-foreground and scale-drift artifacts shown in the 'w/o L_depth' ablation (Table 4, row 2) or fail to converge as in the 'w/o root-body G_j' row, then the central claim of monocular embodied view synthesis is not sustained.","supporting_citations":[{"cited_title":"3d gaussian splatting for real-time radiance field rendering","cited_arxiv_id":null,"evidence_quote":"Supplies the base differentiable Gaussian rendering primitive on which the entire framework is built."},{"cited_title":"Total- recon: Deformable scene reconstruction for embodied view synthesis","cited_arxiv_id":null,"evidence_quote":"Defines the embodied-view-synthesis benchmark and is the primary baseline whose quality and training time HoliGS must beat."},{"cited_title":"Banmo: Building animatable 3d neural models from many casual videos","cited_arxiv_id":null,"evidence_quote":"Provides the articulated canonical-space reconstruction and neural-SDF warm-start strategy that HoliGS adapts for initialization."},{"cited_title":"Skinning with dual quaternions","cited_arxiv_id":null,"evidence_quote":"Dual quaternion skinning is the technique used to fuse bone transforms into a single SE(3) warp."},{"cited_title":"Posenet: A convolutional network for real-time 6-dof camera relocalization","cited_arxiv_id":null,"evidence_quote":"PoseNet supplies the initial rigid-body and object-root pose estimates used in the two-stage optimization."},{"cited_title":"Unidepth: Universal monocular metric depth estimation","cited_arxiv_id":null,"evidence_quote":"UniDepth is the off-the-shelf monocular depth estimator used in the depth-supervision loss that the ablations show is critical."},{"cited_title":"Raft: Recurrent all-pairs field transforms for optical flow","cited_arxiv_id":null,"evidence_quote":"RAFT optical flow is used to compute the flow loss that enforces motion consistency during initialization."}],"review_version":1}