Pith. sign in

REVIEW 5 major objections 5 minor 65 references

Enhanced Velocity Field Modeling for Gaussian Video Reconstruction

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that deformation-based Gaussian splatting produces irregular trajectories in fast-moving video, and that rasterizing per-Gaussian velocities and supervising them with optical flow repairs that weakness.

desk verdict Velocity-field rendering is a plausible improvement over 4DGS, but broken SC-GS baselines and a circular trajectory metric make the empirical claims shaky. read the letter →

arxiv 2507.23704 v1 pith:Y4Z4I5KE submitted 2025-07-31 cs.CV cs.AI

Zhenyang Li , Xiaoyang Bai , Tongchen Zhang , Pengfei Shen , Weiwei Xu , Yifan Peng This is my paper
classification cs.CVcs.AI
keywords dynamicscenereconstruction3DGaussiansplattingvelocityfieldrenderingopticalflowsupervisionflow-assisteddensificationdeformationnovelviewsynthesisExtendedKalmanfilter
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FlowGaussian-VR claims that the main weakness of deformation-based Gaussian splatting for video—irregular, untrackable Gaussian trajectories in fast-moving scenes—can be repaired by treating motion itself as a renderable field. Each Gaussian carries a 2D velocity vector, and the rasterizer alpha-blends those vectors into a per-pixel velocity field, just as it blends color. That field is supervised with optical flow from RAFT through three losses, and flow gradients are used to add new Gaussians where dynamic content is underserved. On the Nvidia-long and Neu3D benchmarks, the pipeline lifts 4DGS novel-view PSNR by about 2.5 dB, with larger relative gains on dynamic-region pixels, and produces smoother trajectories. The paper's stated limitation is that rapid, abrupt appearance changes, as in parts of the 'Jumping' scene, remain hard.

What carries the argument

The central object is the velocity field rendering (VFR) operation: a Gaussian's projected 2D velocity is alpha-blended along each pixel ray with the same transmittance weights used for color, yielding a rendered optical-flow image that can be supervised directly. Around it sit three losses—the windowed velocity error ($L_{win}$), which compares rendered velocity to RAFT flow across a sliding window of frames; the flow warping error ($L_{warp}$), which warps the next rendered frame back and compares it to the current ground truth; and the dynamic rendering loss ($L_{dyn}$), which masks photometric error onto SAM-v2 dynamic regions. The flow-assisted adaptive densification (FAD) strategy selects pixels with high flow loss and flow-loss gradient, lifts them to 3D using rendered depth, thins them with farthest-point sampling, and interpolates attributes from nearby Gaussians before mapping them into canonical space.

What would settle it

Render a synthetic multi-view scene with exact ground-truth 3D motion and ground-truth optical flow, including fast occlusion and motion blur. If training with Lwin, Lwarp, and FAD does not reduce per-Gaussian trajectory error below the photometric-only 4DGS baseline—or if the RAFT-supervised velocities disagree with true motion while PSNR still rises—then the claim that the rendered velocity field faithfully represents scene motion is refuted.

Watch

Extended reading notes

Core claim

The central claim is that velocity can be rendered as a first-class output of Gaussian rasterization. Each Gaussian is assigned a velocity vector, and the same alpha-blending that composites color composites these vectors into a 2D velocity field, so the rendered field is directly differentiable with respect to the Gaussian attributes. Supervising this field with optical flow estimates, warping rendered frames along the field, and masking the loss onto dynamic regions gives the deformation network an explicit motion target instead of letting it fit photometric losses alone. The paper further claims that complementing the usual densification with flow-guided insertion of new Gaussians in dynamic regions closes coverage gaps that photometric gradients miss. If these claims are right, dynamic Gaussian models gain both rendering quality and physically coherent, trackable per-Gaussian motion.

Load-bearing premise

That a pixel's alpha-blended Gaussian velocity, trained against optical flow estimated by a pretrained network, is a faithful stand-in for true scene motion.

Editorial extensions

If this is right

  • Adding velocity field rendering and the three flow-based losses to 4DGS improves average novel-view PSNR by about 2.5 dB on both Nvidia-long and Neu3D, with dynamic-region PSNR gains of roughly 3.1 dB and 2.4 dB respectively.
  • The rendered velocity fields align more closely with ground-truth optical flow than the deformation network's raw displacements, which is evidence that per-Gaussian trajectories become physically consistent.
  • Larger sliding windows for the windowed velocity error give stronger temporal consistency and better reconstruction quality, as shown by the PSNR progression on 'Jumping' and 'Coffee-martini'.
  • The temporal velocity refinement (TVR) post-process using an extended Kalman filter smooths trajectories and improves motion accuracy at a small cost to rendering quality, because it currently updates only Gaussian centers.
  • Flow-based loss maps concentrate on foreground dynamic pixels, unlike the photometric loss map, so flow gradients are a more reliable signal than photometric gradients for deciding where to densify.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Not tested in the paper, but a direct extension: the same differentiable velocity-field rasterization could give a general route from 2D optical flow to 3D scene flow supervision, which would be useful for tracking and for robotics even where novel-view PSNR is secondary.
  • The FAD selection rule uses flow loss and gradient as its signal, but nothing in the mechanism is flow-specific; a similar lifted-densification step could be driven by depth maps, segmentation masks, or other auxiliary supervision in static scene reconstruction.
  • Because TVR's EKF only refines center positions and slightly hurts PSNR, one extension would be to carry shape, rotation, and opacity through the filter as well; the paper does not test whether that would recover the lost rendering quality while keeping the smoother trajectories.
  • If RAFT flow is biased in a target benchmark, the whole supervision stack inherits that bias; a prudent next step is to validate on synthetic scenes with known ground-truth motion, where trajectory error can be measured directly rather than inferred from flow maps.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes FlowGaussian-VR, a method for dynamic 3D Gaussian scene reconstruction that augments the 4DGS deformation-field backbone with a per-Gaussian 2D velocity attribute. The velocity is rendered via differentiable alpha-blended rasterization (velocity field rendering, VFR), supervised by RAFT optical flow through three losses (windowed velocity error Lwin, flow warping error Lwarp, and dynamic-region loss Ldyn), and complemented by a flow-assisted adaptive densification (FAD) strategy that adds Gaussians in dynamic regions. A post-processing step, temporal velocity refinement (TVR), applies an extended Kalman filter to smooth Gaussian trajectories. The paper reports roughly 2.5 dB PSNR improvement over 4DGS on Nvidia-long and Neu3D datasets, as well as qualitative and some quantitative evidence of more regular, flow-aligned Gaussian trajectories.

Significance. If the results are reliable, the core idea of directly rasterizing a velocity field from Gaussian attributes and using it to supervise deformation is a useful and potentially reusable technique for dynamic scene reconstruction and novel view synthesis. The use of flow cues for adaptive densification is also a sensible extension. The paper includes a reasonable amount of experimental material, including two datasets, several baselines, an ablation table, and an exploration of EKF-based refinement. However, the present evaluation contains serious validation gaps—implausible baseline numbers, a circular trajectory-evaluation setup, and a DPSNR metric that is trained on the same masks used for evaluation—so the strength of the central claims is not currently supported.

major comments (5)
  1. [Tables 1 and 2] The baseline numbers are implausible and must be corrected. In Table 1, SC-GS on Umbrella reports PSNR 6.76 dB and SSIM 0.000, and in Table 2 the SC-GS average across Neu3D is 11.42 dB PSNR, an order of magnitude below what is expected from that method. Additionally, the Neu3D average row for MotionGS lists SSIM 0.982, which is higher than any scene-level MotionGS SSIM (max 0.878) and inconsistent with its average PSNR of 23.65. These signs indicate an evaluation or hyperparameter problem for the baselines, not just a fair performance gap. Please provide corrected, properly tuned baseline results with multiple seeds and standard deviations, or restrict the comparison claims to the 4DGS backbone only.
  2. [Sec. 3.3, Eq. (3) and Sec. 5.3] The trajectory-accuracy claim is validated circularly. Lwin supervises the rendered velocity field with RAFT optical flow, and the quantitative evidence of improved motion is the agreement of the rendered velocity field with the same RAFT flows (Figs. 5 and 7, and the 'Evaluation of TVR' paragraph). Since the same estimator is both the training target and the evaluation yardstick, agreement with it cannot distinguish a correct physical motion model from one that simply memorizes the RAFT predictions. An independent trajectory metric—for example, manually annotated 3D tracks, or point correspondences from a different flow estimator or from structure-from-motion—is needed to support the claim of 'regularized and trackable per-Gaussian trajectories.'
  3. [Sec. 5.2 and Eq. (5)] The DPSNR evaluation is biased by the training objective. DPSNR is computed on dynamic regions identified by SAM-2 (Sec. 5.2), while Ldyn in Eq. (5) uses SAM-2 masks to compute the dynamic rendering loss, and FAD in Sec. 3.4 uses the same SAM-2 masks to exclude background pixels. As a result, the model is explicitly trained to improve rendering exactly on the pixels that DPSNR measures, which inflates the reported dynamic-region gains. Please evaluate DPSNR on independently annotated dynamic masks or, at minimum, report results from a setting where the evaluation masks are not used in any loss or densification step.
  4. [Table 3] The ablation does not support the claimed contribution of velocity field rendering. In all non-baseline rows, V.R. is enabled, so the table only toggles Lwarp and Lwin; the individual effect of V.R. itself is never measured. Moreover, Ldyn—introduced in Sec. 3.3 as one of the three flow-based losses—is not ablated at all. A proper ablation should include a row with V.R. disabled (using only photometric losses while keeping FAD as a control) and rows that toggle Ldyn, so that the contribution of each component can be isolated.
  5. [Sec. 4, Eq. (11)-(12)] The EKF derivation is not mathematically well-posed. The text states that the deformation network D is non-invertible, but then invokes the inverse function theorem to write Jf = JD(xt) * JD(xt−1)^{-1}. The inverse function theorem requires invertibility of the Jacobian at the relevant point, which is not guaranteed for a general neural-network deformation field. The finite-difference approximation in Eq. (12) also appears to contain a typo (f(xt−1 + δx) − xt instead of f(xt−1 + δx) − f(xt−1)). Since the TVR experiment is only qualitative (Fig. 7) and the reported PSNR actually decreases with TVR, the claim that TVR 'notably improved' motion accuracy is not quantitatively supported. Please either provide a correct derivation or describe TVR as a heuristic with a clearly separate validation.
minor comments (5)
  1. [Abstract and Sec. 5.2] The abstract claims 'over 2.5 dB gain in PSNR', but the average gain over 4DGS is 2.50 dB on Nvidia and 2.45 dB on Neu3D; 'over 2.5' is only true for the Nvidia dataset. Please state the gain more precisely.
  2. [Eq. (6)] Equation (6) is dimensionally inconsistent: K[R, T] is a 3×4 projection matrix, so multiplying it with a 3-vector (x2D, y2D, 1)^T · Z does not produce a valid 3D world coordinate. The intended inverse-projection procedure should be written with the inverse of K and the inverse rotation, or a correct homogenous-coordinate expression.
  3. [Sec. 5.1] The paper refers to RAFT outputs as 'ground truth optical flows'. Since RAFT is an estimator, these should be described as 'estimated' or 'pseudo ground truth' throughout.
  4. [Tables 1 and 2] The table entries are visually crowded and some rows appear to concatenate values without clear separators (e.g., '22.85 (21.24) / 0.527 / 0.416 24.01 ...'). Please reformat with consistent spacing or per-column entries, and fix the MotionGS average SSIM typo in Table 2.
  5. [Sec. 4, Eq. (12)] Please correct the finite-difference numerator: it should read f(xt−1 + δx) − f(xt−1), not f(xt−1 + δx) − xt.

Circularity Check

2 steps flagged · score 6.0 of 10

Motion-accuracy and dynamic-region gains are evaluated on the same RAFT flows and SAM-2 masks used as training supervision; core PSNR remains independent.

  1. fitted input called prediction [Sec. 3.3 Eq. (3); Sec. 4 'Accurate flow map localization'; Sec. 5.2 'Evaluation of TVR']
    "The windowed velocity error ( Lwin) is calculated between ˆVi(τ ) and ground truth optical flows eVi(τ ) at those timestamps. Formally: Lwin = ∥ ˆVi(τ ) − eVi(τ )∥1. ... For the kth time step, zk is calculated as: zk = F(p0 + Pk−1 i=0 zi), where F(p) denotes the optical flow value at coordinates p. ... we yet observe that the accuracy of Gaussians’ offsets (motion), and subsequently the modeling of scene dynamics, has notably improved, as shown in Fig. 7."

    The paper trains the rendered velocity field by directly minimizing its L1 distance to RAFT optical flow (Eq. 3), and the TVR post-process also assimilates RAFT optical flow as observations. The claimed 'trackable per-Gaussian trajectories' and 'alignment with ground truth optical flows' are then demonstrated only by comparing rendered velocity fields to those same RAFT flow maps (Figs. 5, 7–9) or by the TVR experiment. Because the evaluation signal is identical to the training target, the motion-accuracy claim is not independently established; it largely restates that the model was fitted to RAFT flow.

  2. fitted input called prediction [Sec. 3.3 Eq. (5); Sec. 5.2 'Results']
    "Ldyn = ∥ ˆI ⊙ Mdyn, ˜I ⊙ Mdyn∥1, where Mdyn represents the dynamic foreground segmentation mask. ... D-PSNR computes PSNR only in dynamic regions, identified by SAM-2."

    DPSNR is computed exactly on the SAM-2 dynamic masks that define the Ldyn training loss. Since Ldyn directly optimizes the rendered image against ground truth on those masked pixels, reporting DPSNR gains over baselines that were not trained with this mask partially measures the effect of optimizing on the evaluation region itself. The dynamic-region improvement is therefore partly forced by construction, although overall PSNR remains an external and independent check.

full rationale

The core visual-quality claim (about 2.5 dB PSNR gain) is not circular: PSNR, SSIM, and LPIPS are computed on held-out novel-view RGB frames without using the flow loss or the dynamic mask as the evaluation objective, so the improvement in rendering quality has genuine external grounding. No load-bearing self-citation or imported uniqueness theorem is present; the baseline 4DGS and external components (RAFT, SAM-2, EKF) are standard prior work cited normally. However, two auxiliary but central claims do reduce by construction. First, the claim of regularized, trackable per-Gaussian trajectories aligned with true scene motion is validated against RAFT optical flow, which is also the supervision signal in Lwin and the observation source in TVR; thus the motion-accuracy evidence is self-referential rather than an independent trajectory benchmark. Second, the DPSNR metric is defined on the same SAM-2 dynamic masks used to construct Ldyn, so the reported dynamic-region gains are partly a direct consequence of optimizing on those pixels. These are genuine partial circularities, bringing the score to 6 rather than a lower value; the score is not higher because the headline PSNR improvement remains independently measured.

Assumptions & free parameters 5 free parameters · 6 assumptions · 1 invented entities

The central claim depends on five tuned hyperparameters (FPS ratio, epsilon, window size, EKF noise, loss weights), six background assumptions about flow, depth, masks, and deformation invertibility, and one invented representation attribute. The flow and mask assumptions are load-bearing because they jointly drive optimization and evaluation.

free parameters (5)
  • FPS downsampling ratio r = 0.01 (Nvidia-long), 0.001 (Neu3D)
    Controls how many FAD-selected candidate points are kept; tuned per dataset in Sec 5.1.
  • FAD threshold epsilon = not reported
    Threshold on flow loss and gradient maps for pixel selection in Sec 3.4; the value is not given.
  • Sliding window size tau = up to 8
    Window length in Lwin; Table 4 shows performance increases with window size, with 8 being the largest tested.
  • EKF noise matrices Q and R = not reported
    State and observation noise covariances for the TVR post-processing in Sec 4; deferred to supplementary.
  • Loss weights for Lwin, Lwarp, Ldyn = not reported
    Relative weights of the flow and dynamic losses are not specified in the main text.
assumptions (6)
  • standard math Alpha-blending composition of 3D Gaussians reproduces image formation (Eq. 1).
    Inherited from 3DGS [9]; used as the rendering model for both color and velocity.
  • domain assumption RAFT optical flow is a reliable ground truth for Gaussian motion.
    Lwin and trajectory evaluation compare to RAFT outputs (Sec 3.3, 5.2), not to measured scene flow.
  • domain assumption The alpha-blended projection of per-Gaussian velocities equals optical flow.
    Stated in Sec 3.2 as 'a reasonable estimation'; no proof or error analysis is provided.
  • domain assumption SAM-2 masks correctly identify dynamic foreground.
    Used to define Ldyn and to compute DPSNR (Sec 3.3, 5.2), so the same masks influence both optimization and evaluation.
  • domain assumption Rendered depth is accurate enough to lift FAD-selected pixels to 3D.
    Eq. 6 in Sec 3.4 lifts pixels without error modeling; depth comes from the Gaussian rasterizer.
  • ad hoc to paper Deformation network is locally linear and invertible enough for the inverse function theorem.
    TVR in Sec 4 computes Jf via the inverse function theorem despite noting D is non-invertible; the approximation is acknowledged but unvalidated.
invented entities (1)
  • Per-Gaussian velocity vector attribute
    purpose: Enables velocity field rendering and flow-based supervision of Gaussian trajectories.
    A learned per-Gaussian parameter with no external falsifiable handle beyond the rendered velocities themselves.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enhanced Velocity Field Modeling for Gaussian Video Reconstruction." pith.science (2026). https://pith.science/paper/Y4Z4I5KE

@misc{pith2026250723704,
  author       = {Pith},
  title        = {Pith review of: Enhanced Velocity Field Modeling for Gaussian Video Reconstruction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Y4Z4I5KE}},
  note         = {Machine review of arXiv:2507.23704}
}
read the original abstract

High-fidelity 3D video reconstruction is essential for enabling real-time rendering of dynamic scenes with realistic motion in virtual and augmented reality (VR/AR). The deformation field paradigm of 3D Gaussian splatting has achieved near-photorealistic results in video reconstruction due to the great representation capability of deep deformation networks. However, in videos with complex motion and significant scale variations, deformation networks often overfit to irregular Gaussian trajectories, leading to suboptimal visual quality. Moreover, the gradient-based densification strategy designed for static scene reconstruction proves inadequate to address the absence of dynamic content. In light of these challenges, we propose a flow-empowered velocity field modeling scheme tailored for Gaussian video reconstruction, dubbed FlowGaussian-VR. It consists of two core components: a velocity field rendering (VFR) pipeline which enables optical flow-based optimization, and a flow-assisted adaptive densification (FAD) strategy that adjusts the number and size of Gaussians in dynamic regions. We validate our model's effectiveness on multi-view dynamic reconstruction and novel view synthesis with multiple real-world datasets containing challenging motion scenarios, demonstrating not only notable visual improvements (over 2.5 dB gain in PSNR) and less blurry artifacts in dynamic textures, but also regularized and trackable per-Gaussian trajectories.

Figures

Figures reproduced from arXiv: 2507.23704 by the authors.

Figure 1
Figure 1. Left: Deformation-based 4DGS encounters difficulties in reconstructing scenes and rendering novel views under challenging conditions, such as significant motion and other complex dynamics. Our FlowGaussian-VR exhibits commendable performance on given scenes (e.g., “cut-roasted-beef”). Middle: We compare the ground-truth optical flow, the deformation network of baseline (4DGS), and the velocity field rendered by our … view at source ↗
Figure 2
Figure 2. Overview of our FlowGaussian-VR pipeline. The velocity attribute is introduced into each Gaussian within the dynamic 3D GS pipeline to ensure that the motion of Gaussians closely mimics real-world physical motion. To achieve this, we impose several constraints on the rendered velocity: Lwarp ensures temporal color consistency by aligning the predicted velocity with sequential frame data; Lwin maximizes the supervisi… view at source ↗
Figure 3
Figure 3. Velocity Rendering in 3D Gaussian Rasterization. Gaussian velocities are alpha-blended along each projection ray. Through multi-view optimization, these Gaussian velocities progressively converge toward the actual motion of the object. as µt = µ0 + D(µ0, t). The deformation model D is trained end-to-end with the Gaussian parameters using image losses. During rendering, µt is projected onto the 2D plane as µ ′ t and … view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Flow-assisted adaptive densification (FAD) incorporates flow gradients and loss maps to identify pixel locations requiring densification. These pixels are lifted to the current frame’s 3D space using rendered depth and then transformed back into the canonical space via…
Figure 5
Figure 5. Figure 5: Novel view synthesis evaluation on the Nvidia “Truck” scene. We compare our work with the retrained baseline (4DGS) over a few timesteps. As each timestep, we show the rendered novel view image with PSNR (DPSNR) in dB, and the estimated velocity field (color gamma-tune…
Figure 6
Figure 6. Figure 6: Comparison of photometric and flow-based losses. We visualize the pixel-wise photometric loss and our proposed flow-based loss for a given frame. We observe that Lwarp and Lwin are primarily concentrated on foreground pixels. 9 [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Visualization of the effectiveness of Temporal Velocity Refinement (TVR) trajectory correction. The figure [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: Novel view synthesis evaluation on the Nvidia dataset. We present three scenes: “Balloon1”, “Umbrella”, and “Jumping” and compare our work with retrained 4DGS. The 3 primary columns show results of the ground truth, baseline, and our method. For each scene, ground trut…
Figure 9
Figure 9. Figure 9: Novel view synthesis evaluation on the Neu3d dataset. We visualize the “flame”, “martini”, and “beef” scenes and compare our approach with the retrained 4DGS. We rearranged the inset placement from figure 8 to suit the aspect ratio of this dataset, that the 3 main colu…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

65 extracted references · 44 canonical work pages

  1. [1]

    Navid Ashrafi, Francesco V ona, Carina Ringsdorf, Christian Hertel, Luca Toni, Sarina Kailer, Alice Bartels, Tanja Kojic, and Jan-Niklas V oigt-Antons. Enhancing job interview preparation through immersive experiences using 13 Enhanced Velocity Field Modeling for Gaussian Video Reconstruction photorealistic, ai-powered metahuman avatars. In 2024 IEEE Inte...

    work page 2024

  2. [2]

    Enhancing physics education: Designing customized virtual reality for teaching crystalline structures

    Shawheen Alipour, Audrey Rah, Bishnu Karki, Justin Burris, Leslie Coward, and Tony Liao. Enhancing physics education: Designing customized virtual reality for teaching crystalline structures. In 2024 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), pages 549–552. IEEE, 2024

    work page 2024

  3. [3]

    Vr-gs: A physical dynamics-aware interactive gaussian splatting system in virtual reality

    Ying Jiang, Chang Yu, Tianyi Xie, Xuan Li, Yutao Feng, Huamin Wang, Minchen Li, Henry Lau, Feng Gao, Yin Yang, et al. Vr-gs: A physical dynamics-aware interactive gaussian splatting system in virtual reality. In ACM SIGGRAPH 2024 Conference Papers, pages 1–1, 2024

    work page 2024

  4. [4]

    Dreamcinema: Cinematic transfer with free camera and 3d character

    Weiliang Chen, Fangfu Liu, Diankun Wu, Haowen Sun, Haixu Song, and Yueqi Duan. Dreamcinema: Cinematic transfer with free camera and 3d character. arXiv preprint arXiv:2408.12601, 2024

    arXiv 2024

  5. [5]

    Drivinggaussian: Composite gaussian splatting for surrounding dynamic autonomous driving scenes

    Xiaoyu Zhou, Zhiwei Lin, Xiaojun Shan, Yongtao Wang, Deqing Sun, and Ming-Hsuan Yang. Drivinggaussian: Composite gaussian splatting for surrounding dynamic autonomous driving scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21634–21643, 2024

    work page 2024

  6. [6]

    Robust collaborative visual-inertial slam for mobile augmented reality

    Xiaokun Pan, Gan Huang, Ziyang Zhang, Jinyu Li, Hujun Bao, and Guofeng Zhang. Robust collaborative visual-inertial slam for mobile augmented reality. IEEE Transactions on Visualization and Computer Graphics, 2024

    work page 2024

  7. [7]

    Hoimotion: Forecasting human motion during human-object interactions using egocentric 3d object bounding boxes

    Zhiming Hu, Zheming Yin, Daniel Haeufle, Syn Schmitt, and Andreas Bulling. Hoimotion: Forecasting human motion during human-object interactions using egocentric 3d object bounding boxes. IEEE Transactions on Visualization and Computer Graphics, 2024

    work page 2024

  8. [8]

    Co-designing dynamic mixed reality drill positioning widgets: A collaborative approach with dentists in a realistic setup

    Mine Dastan, Michele Fiorentino, Elias D Walter, Christian Diegritz, Antonio E Uva, Ulrich Eck, and Nassir Navab. Co-designing dynamic mixed reality drill positioning widgets: A collaborative approach with dentists in a realistic setup. IEEE Transactions on Visualization and Computer Graphics, 2024

    work page 2024

Show all 65 references
  1. [9]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139–1, 2023

    2023

  2. [10]

    Srinivasan, Matthew Tancik, Jonathan T

    Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. NeRF: Representing scenes as neural radiance fields for view synthesis. In ECCV, 2020

    2020

  3. [11]

    Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis

    Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis. arXiv preprint arXiv:2308.09713, 2023

    2023 arXiv

  4. [12]

    Spacetime gaussian feature splatting for real-time dynamic view synthesis

    Zhan Li, Zhang Chen, Zhong Li, and Yi Xu. Spacetime gaussian feature splatting for real-time dynamic view synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8508–8520, 2024

    2024

  5. [13]

    4d gaussian splatting for real-time dynamic scene rendering

    Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4d gaussian splatting for real-time dynamic scene rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20310–20320, 2024

    2024

  6. [14]

    4d gaussian splatting for real-time dynamic scene rendering

    Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4d gaussian splatting for real-time dynamic scene rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 20310...

    2024

  7. [15]

    Gaussianflow: Splatting gaussian dynamics for 4d content creation

    Quankai Gao, Qiangeng Xu, Zhe Cao, Ben Mildenhall, Wenchao Ma, Le Chen, Danhang Tang, and Ulrich Neumann. Gaussianflow: Splatting gaussian dynamics for 4d content creation. arXiv preprint arXiv:2403.12365, 2024

    2024 arXiv

  8. [16]

    Shape of motion: 4d reconstruction from a single video

    Qianqian Wang, Vickie Ye, Hang Gao, Jake Austin, Zhengqi Li, and Angjoo Kanazawa. Shape of motion: 4d reconstruction from a single video. arXiv preprint arXiv:2407.13764, 2024

    2024

  9. [17]

    A survey on trajectory-prediction methods for autonomous driving

    Yanjun Huang, Jiatong Du, Ziru Yang, Zewei Zhou, Lin Zhang, and Hong Chen. A survey on trajectory-prediction methods for autonomous driving. IEEE Transactions on Intelligent Vehicles, 7(3):652–674, 2022

    2022

  10. [18]

    Nicer-slam: Neural implicit scene encoding for rgb slam

    Zihan Zhu, Songyou Peng, Viktor Larsson, Zhaopeng Cui, Martin R Oswald, Andreas Geiger, and Marc Pollefeys. Nicer-slam: Neural implicit scene encoding for rgb slam. arXiv preprint arXiv:2302.03594, 2023

    2023 arXiv

  11. [19]

    Raft: Recurrent all-pairs field transforms for optical flow

    Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16, pages 402–419. Springer, 2020

    2020

  12. [20]

    Sam 2: Segment anything in images and videos

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao-Yuan Wu, Ross Girshick, Piotr Dollár, and Christoph Feichtenh...

    2024 arXiv

  13. [21]

    Extended kalman filter

    Keisuke Fujii. Extended kalman filter. Refernce Manual, 14:41, 2013

    2013

  14. [22]

    Novel view synthesis of dynamic scenes with globally coherent depths from a monocular camera

    Jae Shin Yoon, Kihwan Kim, Orazio Gallo, Hyun Soo Park, and Jan Kautz. Novel view synthesis of dynamic scenes with globally coherent depths from a monocular camera. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5336–5345, 2020

    2020

  15. [23]

    Neural 3d video synthesis from multi-view video

    Tianye Li, Mira Slavcheva, Michael Zollhoefer, Simon Green, Christoph Lassner, Changil Kim, Tanner Schmidt, Steven Lovegrove, Michael Goesele, Richard Newcombe, et al. Neural 3d video synthesis from multi-view video. In Proceedings of the IEEE/CVF Conference on Computer Vision...

    2022

  16. [24]

    Real-time photorealistic dynamic scene representation and rendering with 4d gaussian splatting

    Zeyu Yang, Hongye Yang, Zijie Pan, and Li Zhang. Real-time photorealistic dynamic scene representation and rendering with 4d gaussian splatting. In International Conference on Learning Representations (ICLR), 2024

    2024

  17. [25]

    Sc-gs: Sparse-controlled gaussian splatting for editable dynamic scenes

    Yi-Hua Huang, Yang-Tian Sun, Ziyi Yang, Xiaoyang Lyu, Yan-Pei Cao, and Xiaojuan Qi. Sc-gs: Sparse-controlled gaussian splatting for editable dynamic scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4220–4230, 2024

    2024

  18. [26]

    Neural prior for trajectory estimation

    Chaoyang Wang, Xueqian Li, Jhony Kaesemodel Pontes, and Simon Lucey. Neural prior for trajectory estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6532–6542, 2022

    2022

  19. [27]

    Deepdeform: Learning non-rigid rgb-d reconstruction with semi-supervised data

    Aljaz Bozic, Michael Zollhofer, Christian Theobalt, and Matthias Nießner. Deepdeform: Learning non-rigid rgb-d reconstruction with semi-supervised data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7002–7012, 2020

    2020

  20. [28]

    First order motion model for image animation

    Aliaksandr Siarohin, Stéphane Lathuilière, Sergey Tulyakov, Elisa Ricci, and Nicu Sebe. First order motion model for image animation. Advances in neural information processing systems, 32, 2019

    2019

  21. [29]

    Fusion4d: Real-time performance capture of challenging scenes

    Mingsong Dou, Sameh Khamis, Yury Degtyarev, Philip Davidson, Sean Ryan Fanello, Adarsh Kowdle, Sergio Orts Escolano, Christoph Rhemann, David Kim, Jonathan Taylor, et al. Fusion4d: Real-time performance capture of challenging scenes. ACM Transactions on Graphics (ToG), 35(4):1...

    2016

  22. [30]

    Robust consistent video depth estimation

    Johannes Kopf, Xuejian Rong, and Jia-Bin Huang. Robust consistent video depth estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1611–1621, 2021

    2021

  23. [31]

    Consistent depth of moving objects in video

    Zhoutong Zhang, Forrester Cole, Richard Tucker, William T Freeman, and Tali Dekel. Consistent depth of moving objects in video. ACM Transactions on Graphics (ToG), 40(4):1–12, 2021

    2021

  24. [32]

    Sync-nerf: Generalizing dynamic nerfs to unsynchronized videos

    Seoha Kim, Jeongmin Bae, Youngsik Yun, Hahyun Lee, Gun Bang, and Youngjung Uh. Sync-nerf: Generalizing dynamic nerfs to unsynchronized videos. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 38, pages 2777–2785, 2024

    2024

  25. [33]

    Sparsenerf: Distilling depth ranking for few-shot novel view synthesis

    Guangcong, Zhaoxi Chen, Chen Change Loy, and Ziwei Liu. Sparsenerf: Distilling depth ranking for few-shot novel view synthesis. IEEE/CVF International Conference on Computer Vision (ICCV), 2023

    2023

  26. [34]

    Nerf-supervised deep stereo

    Fabio Tosi, Alessio Tonioni, Daniele De Gregorio, and Matteo Poggi. Nerf-supervised deep stereo. In Conference on Computer Vision and Pattern Recognition (CVPR), pages 855–866, June 2023

    2023

  27. [35]

    Neural radiance flow for 4d view synthesis and video processing

    Yilun Du, Yinan Zhang, Hong-Xing Yu, Joshua B Tenenbaum, and Jiajun Wu. Neural radiance flow for 4d view synthesis and video processing. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 14304–14314. IEEE Computer Society, 2021

    2021

  28. [36]

    Non-rigid neural radiance fields: Reconstruction and novel view synthesis of a dynamic scene from monocular video

    Edgar Tretschk, Ayush Tewari, Vladislav Golyanik, Michael Zollhöfer, Christoph Lassner, and Christian Theobalt. Non-rigid neural radiance fields: Reconstruction and novel view synthesis of a dynamic scene from monocular video. In Proceedings of the IEEE/CVF International Confe...

    2021

  29. [37]

    Space-time neural irradiance fields for free- viewpoint video

    Wenqi Xian, Jia-Bin Huang, Johannes Kopf, and Changil Kim. Space-time neural irradiance fields for free- viewpoint video. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9421–9431, 2021

    2021

  30. [38]

    D-nerf: Neural radiance fields for dynamic scenes

    Albert Pumarola, Enric Corona, Gerard Pons-Moll, and Francesc Moreno-Noguer. D-nerf: Neural radiance fields for dynamic scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10318–10327, 2021

    2021

  31. [39]

    Nerfies: Deformable neural radiance fields

    Keunhong Park, Utkarsh Sinha, Jonathan T Barron, Sofien Bouaziz, Dan B Goldman, Steven M Seitz, and Ricardo Martin-Brualla. Nerfies: Deformable neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5865–5874, 2021

    2021

  32. [40]

    Hypernerf: A higher-dimensional representation for topologically varying neural radiance fields

    Keunhong Park, Utkarsh Sinha, Peter Hedman, Jonathan T Barron, Sofien Bouaziz, Dan B Goldman, Ricardo Martin-Brualla, and Steven M Seitz. Hypernerf: A higher-dimensional representation for topologically varying neural radiance fields. arXiv preprint arXiv:2106.13228, 2021. 15 ...

    2021 arXiv

  33. [41]

    Dynamic view synthesis from dynamic monocular video

    Chen Gao, Ayush Saraf, Johannes Kopf, and Jia-Bin Huang. Dynamic view synthesis from dynamic monocular video. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5712–5721, 2021

    2021

  34. [42]

    Neural scene flow fields for space-time view synthesis of dynamic scenes

    Zhengqi Li, Simon Niklaus, Noah Snavely, and Oliver Wang. Neural scene flow fields for space-time view synthesis of dynamic scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6498–6508, 2021

    2021

  35. [43]

    Dynibar: Neural dynamic image-based rendering

    Zhengqi Li, Qianqian Wang, Forrester Cole, Richard Tucker, and Noah Snavely. Dynibar: Neural dynamic image-based rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4273–4284, 2023

    2023

  36. [44]

    Octree-gs: Towards consistent real-time rendering with lod-structured 3d gaussians

    Kerui Ren, Lihan Jiang, Tao Lu, Mulin Yu, Linning Xu, Zhangkai Ni, and Bo Dai. Octree-gs: Towards consistent real-time rendering with lod-structured 3d gaussians. arXiv preprint arXiv:2403.17898, 2024

    2024 arXiv

  37. [45]

    Mip-splatting: Alias-free 3d gaussian splatting

    Zehao Yu, Anpei Chen, Binbin Huang, Torsten Sattler, and Andreas Geiger. Mip-splatting: Alias-free 3d gaussian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 19447–19456, 2024

    2024

  38. [46]

    Deformable 3d gaussians for high-fidelity monocular dynamic scene reconstruction

    Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. Deformable 3d gaussians for high-fidelity monocular dynamic scene reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20331–20341, 2024

    2024

  39. [47]

    Hexplane: A fast representation for dynamic scenes

    Ang Cao and Justin Johnson. Hexplane: A fast representation for dynamic scenes. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 130–141, 2023

    2023

  40. [48]

    3d as-rigid-as-possible deforma- tions using mls

    Alvaro Cuno, Claudio Esperança, Antonio Oliveira, and Paulo Roma Cavalcanti. 3d as-rigid-as-possible deforma- tions using mls. In Proceedings of the 27th computer graphics international conference, pages 115–122. Citeseer, 2007

    2007

  41. [49]

    Motion-aware 3d gaussian splatting for efficient dynamic scene reconstruction

    Zhiyang Guo, Wengang Zhou, Li Li, Min Wang, and Houqiang Li. Motion-aware 3d gaussian splatting for efficient dynamic scene reconstruction. arXiv preprint arXiv:2403.11447, 2024

    2024 arXiv

  42. [50]

    Motiongs: Exploring explicit motion guidance for deformable 3d gaussian splatting

    Ruijie Zhu, Yanzhe Liang, Hanzhi Chang, Jiacheng Deng, Jiahao Lu, Wenfei Yang, Tianzhu Zhang, and Yongdong Zhang. Motiongs: Exploring explicit motion guidance for deformable 3d gaussian splatting. Advances in Neural Information Processing Systems, 37:101790–101817, 2024

    2024

  43. [51]

    Representing long volumetric video with temporal gaussian hierarchy

    Zhen Xu, Yinghao Xu, Zhiyuan Yu, Sida Peng, Jiaming Sun, Hujun Bao, and Xiaowei Zhou. Representing long volumetric video with temporal gaussian hierarchy. ACM Transactions on Graphics (TOG), 43(6):1–18, 2024

    2024

  44. [52]

    Gflow: Recovering 4d world from monocular video

    Shizun Wang, Xingyi Yang, Qiuhong Shen, Zhenxiang Jiang, and Xinchao Wang. Gflow: Recovering 4d world from monocular video. arXiv preprint arXiv:2405.18426, 2024

    2024 arXiv

  45. [53]

    Modgs: Dynamic gaussian splatting from causually-captured monocular videos

    Qingming Liu, Yuan Liu, Jiepeng Wang, Xianqiang Lv, Peng Wang, Wenping Wang, and Junhui Hou. Modgs: Dynamic gaussian splatting from causually-captured monocular videos. arXiv preprint arXiv:2406.00434, 2024

    2024 arXiv

  46. [54]

    Geowizard: Unleashing the diffusion priors for 3d geometry estimation from a single image

    Xiao Fu, Wei Yin, Mu Hu, Kaixuan Wang, Yuexin Ma, Ping Tan, Shaojie Shen, Dahua Lin, and Xiaoxiao Long. Geowizard: Unleashing the diffusion priors for 3d geometry estimation from a single image. In European Conference on Computer Vision, pages 241–258. Springer, 2025

    2025

  47. [55]

    How nerfs and 3d gaussian splatting are reshaping slam: a survey

    Fabio Tosi, Youmin Zhang, Ziren Gong, Erik Sandström, Stefano Mattoccia, Martin R Oswald, and Matteo Poggi. How nerfs and 3d gaussian splatting are reshaping slam: a survey. arXiv preprint arXiv:2402.13255, 4, 2024

    2024 arXiv

  48. [56]

    Structure-from-motion revisited

    Johannes L Schonberger and Jan-Michael Frahm. Structure-from-motion revisited. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4104–4113, 2016

    2016

  49. [57]

    The farthest point strategy for progressive image sampling

    Yuval Eldar, Michael Lindenbaum, Moshe Porat, and Yehoshua Y Zeevi. The farthest point strategy for progressive image sampling. IEEE transactions on image processing, 6(9):1305–1315, 1997

    1997

  50. [58]

    Samplenet: Differentiable point cloud sampling

    Itai Lang, Asaf Manor, and Shai Avidan. Samplenet: Differentiable point cloud sampling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7578–7588, 2020

    2020

  51. [59]

    The unscented kalman filter for nonlinear estimation

    Eric A Wan and Rudolph Van Der Merwe. The unscented kalman filter for nonlinear estimation. In Proceedings of the IEEE 2000 adaptive systems for signal processing, communications, and control symposium (Cat. No. 00EX373), pages 153–158. Ieee, 2000

    2000

  52. [60]

    Kfd-nerf: Rethinking dynamic nerf with kalman filter

    Yifan Zhan, Zhuoxiao Li, Muyao Niu, Zhihang Zhong, Shohei Nobuhara, Ko Nishino, and Yinqiang Zheng. Kfd-nerf: Rethinking dynamic nerf with kalman filter. arXiv preprint arXiv:2407.13185, 2024

    2024 arXiv

  53. [61]

    Extended kalman filter tutorial

    Gabriel A Terejanu et al. Extended kalman filter tutorial. University at Buffalo, 27, 2008

    2008

  54. [62]

    An adaptive complementary kalman filter using fuzzy logic for a hybrid head tracker system

    Chang Ho Kang, Chan Gook Park, and Jin Woo Song. An adaptive complementary kalman filter using fuzzy logic for a hybrid head tracker system. IEEE Transactions on Instrumentation and Measurement, 65(9):2163–2173, 2016. 16 Enhanced Velocity Field Modeling for Gaussian Video Reco...

    2016

  55. [63]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4), July 2023

    2023

  56. [64]

    Pixelwise view selection for unstructured multi-view stereo

    Johannes L Schönberger, Enliang Zheng, Jan-Michael Frahm, and Marc Pollefeys. Pixelwise view selection for unstructured multi-view stereo. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part III 14, pages 5...

    2016

  57. [65]

    Panoptic studio: A massively multiview system for social interaction capture

    Hanbyul Joo, Tomas Simon, Xulong Li, Hao Liu, Lei Tan, Lin Gui, Sean Banerjee, Timothy Scott Godisart, Bart Nabbe, Iain Matthews, Takeo Kanade, Shohei Nobuhara, and Yaser Sheikh. Panoptic studio: A massively multiview system for social interaction capture. IEEE Transactions on...

    2017

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.