Pith. sign in

REVIEW 3 major objections 5 minor 67 references

Vid2Sim: Generalizable, Video-based Reconstruction of Appearance, Geometry and Physics for Mesh-free Simulation

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Vid2Sim claims that physical system identification from multi-view video becomes generalizable and fast when split into a feed-forward prediction stage and a short refinement stage, demonstrated with a mesh-free, reduced-order Linear…

desk verdict Solid systems paper with a genuinely new feed-forward physics estimation stage and fast refinement, but the 'generalizable' claim outruns the evidence—the reduced-order LBS backbone is never shown to cover the deformation space it claims. read the letter →

arxiv 2506.06440 v1 pith:6SZAN4WE submitted 2025-06-06 cs.GR cs.CV

Chuhao Chen , Zhiyang Dou , Chen Wang , Yiming Huang , Anjun Chen , Qiao Feng , Jiatao Gu , Lingjie Liu This is my paper
classification cs.GRcs.CV
keywords video-basedsystemidentification3DGaussianSplattingLinearBlendSkinningmesh-freesimulationphysicalpropertyestimationfeed-forwardreconstructionelasticdeformationdifferentiable
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Vid2Sim aims to make physical system identification from video generalizable and fast by replacing heavy per-scene optimization with a two-stage pipeline. In the first stage, a feed-forward network trained on simulated elastic-object videos reads a single-view video and outputs elastic parameters and skinning-based deformation weights, while a pretrained multi-view reconstruction model builds the object's geometry and appearance as 3D Gaussians. In the second stage, a lightweight differentiable mesh-free simulator refines appearance, geometry, and physical properties against multi-view videos, completing in about 15 minutes on one GPU. If the claim holds, recovering a simulatable digital twin from video stops being an hour-long per-scene optimization and becomes a quick, reusable step.

What carries the argument

The load-bearing object is a mesh-free, reduced-order elastic simulator built on Linear Blend Skinning: a set of 3D Gaussian points $X_i$ is deformed by $m$ affine handles through $x_i = X_i + \sum_{j=1}^m W_{\theta;j}(X_i) Z_j [X_i, 1]^\top$, with $m \ll n$ and an MLP predicting the per-point weights $W_{\theta;j}$. Time evolution uses an implicit Euler solver on the handle variables via an incremental potential with inertia and elastic energy terms, and only 500 cubature points are sampled for efficiency. To make refinement affordable, a Neural Jacobian module learns the deformation-gradient Jacobian $J_\theta(X) \approx \partial F(X,z)/\partial z$ from finite-difference targets, so the joint optimization over skinning weights, Jacobian, Young's modulus, and Poisson's ratio avoids expensive explicit Jacobian precomputation; in the paper's configuration 10 handles and 500 cubature points are used. Appearance and geometry are carried by 3D Gaussian Splatting primitives, deformed by the same skinning map, so a rendering loss can drive the refinement.

What would settle it

Render a full-fidelity FEM simulation of a slender elastic object undergoing localized bending or torsion into multi-view video, run Vid2Sim on the first 16 frames, and compare its future-state prediction against the FEM ground truth; a large error only on such out-of-subspace motions would show that the reduced-order LBS configuration, not the neural components, is the limiting factor.

Watch

Extended reading notes

Core claim

The paper's central claim is that appearance, geometry, and elastic material parameters can be recovered from multi-view video in a generalizable, feed-forward way, and then refined in minutes, without sacrificing reconstruction quality. The key design choice is to make the recovered object directly simulation-ready: 3D Gaussian points are deformed by a reduced-order Linear Blend Skinning model, so the same representation that is fitted to video can immediately be re-simulated under novel forces and boundary conditions. Vid2Sim reports that this two-stage scheme outperforms prior per-scene optimization methods on both dynamic reconstruction and future-state prediction in its synthetic and real experiments, while cutting per-iteration optimization cost by a substantial margin.

Load-bearing premise

The pipeline depends on a reduced-order simulation model in which 10 affine skinning handles and 500 cubature points must faithfully represent the object's real deformation; if an object's motion involves localized bending, torsion, or other modes outside that subspace, neither the feed-forward estimates nor the refinement can capture it, and the paper does not quantify this approximation error.

Editorial extensions

If this is right

  • Total per-scene cost drops to about 15 minutes with a 1.44 s per-iteration optimization, compared with the 54 to 120 minute totals reported for the per-scene optimization baselines.
  • The fitted 3D Gaussian representation is simulation-ready, so the same reconstruction can be re-simulated under novel forces, changed constitutive models, and different boundary conditions without another round of system identification.
  • Future-state prediction is substantially more accurate than the compared baselines on the synthetic benchmark (25.07 PSNR versus at most 20.11), indicating that the recovered physical parameters, not just appearance, carry forward in time.
  • Because the simulator operates in a reduced handle space with a learned Jacobian, it remains differentiable and cheap enough to embed directly in the fitting loop, avoiding the background-grid and time-integration constraints of grid-based continuum alternatives.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves the approximation error of its reduced-order model unquantified; a natural extension is to sweep handle and cubature-point counts and measure when future-state error stops improving.
  • Because Stage I predicts material parameters from a single view and Stage II only refines them, the same split could be retrained for plasticine or sand by adding yield parameters to the regression heads and swapping the energy function.
  • The paper states that its reduced-order simulator limits fluid-like materials; a hybrid that keeps skinning handles for global deformation and adds local particle degrees of freedom would be a natural next step, though that is an extension beyond the paper.
  • A unified single network predicting both 3D Gaussians and point-wise physics would remove the current dependence on a separate multi-view reconstruction model, and the paper's two-branch design suggests this is a feasible next step.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. Vid2Sim proposes a two-stage framework for reconstructing appearance, geometry, and elastic material parameters from multi-view video. Stage I uses a VideoMAE-based feed-forward network to predict Young's modulus E, Poisson's ratio ν, and LBS weights (via a hypernetwork), while a Large Multi-view Gaussian Model reconstructs 3D Gaussians. Stage II refines these estimates through a differentiable, reduced-order LBS simulator with an implicit Euler solver and a learned Neural Jacobian, supervised by a rendering loss. Experiments on 12 synthetic GSO objects (FEM reference videos) and 3 real-world objects report large PSNR/SSIM improvements over GIC, PAC-NeRF, and Spring-Gaus, plus faster runtime (about 15 minutes of refinement). The paper claims state-of-the-art accuracy and speed and positions Vid2Sim as the first generalizable, video-based reconstruction of appearance, geometry, and physics for mesh-free simulation.

Significance. If the reported results are robust, Vid2Sim is a meaningful step: it is the first feed-forward system that, combined with a short per-scene refinement, recovers simulation-ready 3D Gaussians and elastic parameters, and it outperforms the compared optimization-based baselines by a large margin on the authors' benchmarks. The paper's strengths include a carefully designed benchmark with FEM-generated reference videos, explicit future-state prediction evaluation, runtime comparisons, and ablations that probe the value of Stage I initialization and refinement. The experimental evidence is substantial and the methodological novelty (LBS-based differentiable simulation for vision tasks, Neural Jacobian, hypernetwork-predicted skinning weights) is clear. However, the central generalizability claim rests on assumptions that are not yet tested: the expressiveness of the 10-handle LBS subspace, the downstream effect of the Neural Jacobian approximation, and the independence of the physical prior from the simulator used for training.

major comments (3)
  1. [Section 5.2, Eq. (1)] The reduced-order LBS model uses 10 control handles and 500 cubature points, but the paper never quantifies the approximation error of this deformation subspace relative to the FEM reference deformations used in the synthetic test set. For fixed skinning weights, the deformed configuration space has at most 12*10=120 degrees of freedom, which cannot represent localized bending, torsion, or high-curvature contact in general. The paper mentions in Table 5 a 'more accurate' setting with 40 handles and 2000 cubature points, but reports no accuracy numbers for it. This is load-bearing for the 'generalizable' claim: if the reduced model cannot represent the observed deformations, neither the feed-forward prediction nor the refinement can succeed, regardless of the neural components. Please add an expressiveness analysis (e.g., per-object reconstruction error of the reduced model vs. handle count from 10 to 40 or more, and the corresponding error on held-out deformations), and show that the evaluated test scenarios are within the representable class.
  2. [Section 4.2, Eq. (5); Section 6.3] The Neural Jacobian is validated only for its own approximation accuracy (L2 error and timing in Supplementary Table 7), not for its effect on downstream simulation and parameter estimation. The text states 'We validate the effectiveness of the Neural Jacobian in Sec. 6.3', but Section 6.3 contains no ablation of the Neural Jacobian; the ablations there concern Stage I, geometry refinement, LBS fine-tuning, and physics initialization. Because the Neural Jacobian is used in the optimization loop, any approximation bias could shift the estimated E, ν, and LBS weights. Please add an experiment that compares the full pipeline using the Neural Jacobian against using the exact Jacobian (or a numerically computed one) in terms of final reconstruction PSNR, physical parameter MAE, and convergence speed, to rule out downstream bias.
  3. [Section 5.1] The feed-forward predictor is trained on animations generated by the paper's own reduced-order LBS simulator with randomly sampled E and ν. This means the learned 'physical world knowledge' is knowledge of that particular simulator, not of independent physics. The synthetic test set uses FEM-generated videos, but this does not break the circularity because Stage II refinement can correct a biased prior using the rendering loss; indeed, the ablation in Table 2 shows that removing the Stage I physics initialization (random init) yields PSNR 29.87 vs. 30.17 for the full model, a modest difference. Please either provide evidence that the Stage I predictions generalize beyond the training simulator's deformation distribution (for example, evaluate Stage I alone on FEM inputs, or train on a more diverse set of simulations) or soften the 'physical world knowledge' claim to 'trained on the same reduced-order simulator used for refinement'. This is important because the generalizability claim depends on the prior being useful in domains where the reduced model may be inaccurate.
minor comments (5)
  1. [Tables 1-4] The main tables report single-run metrics without error bars or multiple seeds. Given that some ablations differ by less than 0.3 PSNR (e.g., Table 2, full model vs. full w/o Stage I Phys.), please report mean and standard deviation over at least three runs.
  2. [Section 4.2] There is a typo: 'Neural Jacibian' should be 'Neural Jacobian'.
  3. [Equations (1)-(2)] The notation for the mass matrix M and the incremental potential energy in Eq. (2) is not defined, and the LBS weights W_θ are not explicitly constrained (e.g., normalization or partition of unity). Please clarify these definitions to make the method reproducible.
  4. [Table 5] The entry for 'Ours (w/o Jθ)' with 40 handles and 2000 cubature points reports timing but no corresponding accuracy. Please report PSNR/MAE for that setting, or state explicitly that it is a runtime-only comparison.
  5. [Section 6.5] The real-world evaluation uses only 3 objects (orange, bird, cup). While the results are compared with Spring-Gaus, the small number of objects makes it difficult to assess generalizability in real conditions; please add a discussion of this limitation or additional real-world cases.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central reconstruction is tested against independent FEM and real-video references, and the reduced-order simulator is a forward model rather than a re-labeling of the predictions.

full rationale

Vid2Sim's derivation chain is not circular. The feed-forward estimator is trained on synthetic videos generated by the paper's own reduced-order LBS simulator (Sec. 5.1), but it is not asked to regurgitate training labels at test time: the same method is evaluated against an independent FEM-generated synthetic test set (Sec. 5.2) and against real multi-view captures (Sec. 5.2), where it outperforms GIC, PAC-NeRF, and Spring-Gaus. The reduced-order LBS simulator (Eq. 1) and the implicit-Euler incremental-potential update (Eq. 2) are forward models taken from Simplicits [41], an external citation; no claimed uniqueness theorem or load-bearing self-citation is used to force the method's choices. The Neural Jacobian (Eq. 5) is a learned surrogate for a quantity that is well-defined by the LBS deformation map and is trained against finite-difference ground truth, so it is an approximation, not a renamed input. Stage II optimizes E, nu, LBS weights, and Jacobian against the observed rendering loss (Eq. 6), which is an external objective; the ablation 'Ours (full w/o Stage I Phys.)' shows the Stage I physics initialization slightly improves convergence but does not define the result. The reduced-order expressiveness concern (10 handles, 500 cubature points, Sec. 5.2) is a correctness or generalization risk about whether the model space can represent certain deformations, not a circularity: the paper does not define the target deformation space in terms of the LBS subspace. Hence no step reduces by construction to its own inputs.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on several unproven modeling choices: the reduced-order LBS capacity, the accuracy of the learned Jacobian in the downstream solve, the identifiability of material parameters from rendering loss, and the transferability of the pretrained video features. The feed-forward training data is produced by the same reduced simulator used at test time, so the 'physical world knowledge' is partly self-referential. The finite-element synthetic benchmark and real videos provide independent grounding, which keeps the circularity burden moderate rather than high.

free parameters (3)
  • Young's modulus E and Poisson's ratio nu = optimized per scene (Sec 4.2, Eq 6)
    These target parameters are obtained by minimizing a rendering loss against input videos; they are fitted values, not purely feed-forward predictions. Table 9 compares to FEM ground truth on 12 synthetic scenes, but no error bars are reported.
  • Control handle count m and cubature point count k = m=10, k=500 (Sec 5.2)
    Hand-chosen reduced-order capacity; no sensitivity analysis is provided, so the result depends on this untested choice.
  • Floor barrier coefficient E_f = 1e5 (supplementary Sec 9.3)
    Hand-chosen collision parameter that shapes all falling-object simulations.
assumptions (4)
  • domain assumption The LBS reduced-order model with a fixed small number of handles can faithfully represent the deformation space of the elastic bodies.
    Invoked by Eq (1) and used in all simulations; the paper does not measure the approximation error of the reduced model against full FEM on the test set.
  • ad hoc to paper The Neural Jacobian J_theta(X) is an accurate proxy for the exact Jacobian J(X) throughout the optimization.
    Trained with finite differences on random samples (Eq 5); no downstream comparison of final reconstruction using exact vs neural Jacobian is reported.
  • domain assumption E and nu are identifiable from the multi-view rendering loss over the observed frames.
    Stage II optimizes E,nu against Eq (6); identifiability is only indirectly supported by the synthetic MAE in Table 9, not by a conditioning or sensitivity analysis.
  • domain assumption Pre-trained VideoMAE features fine-tuned on synthetic falling-object videos transfer to unseen objects and real videos.
    The evidence is the marginal improvement in Table 2 (30.17 vs 29.87) and three real-world objects without physics ground truth; transferability is therefore weakly supported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Vid2Sim: Generalizable, Video-based Reconstruction of Appearance, Geometry and Physics for Mesh-free Simulation." pith.science (2026). https://pith.science/paper/6SZAN4WE

@misc{pith2026250606440,
  author       = {Pith},
  title        = {Pith review of: Vid2Sim: Generalizable, Video-based Reconstruction of Appearance, Geometry and Physics for Mesh-free Simulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6SZAN4WE}},
  note         = {Machine review of arXiv:2506.06440}
}
read the original abstract

Faithfully reconstructing textured shapes and physical properties from videos presents an intriguing yet challenging problem. Significant efforts have been dedicated to advancing such a system identification problem in this area. Previous methods often rely on heavy optimization pipelines with a differentiable simulator and renderer to estimate physical parameters. However, these approaches frequently necessitate extensive hyperparameter tuning for each scene and involve a costly optimization process, which limits both their practicality and generalizability. In this work, we propose a novel framework, Vid2Sim, a generalizable video-based approach for recovering geometry and physical properties through a mesh-free reduced simulation based on Linear Blend Skinning (LBS), offering high computational efficiency and versatile representation capability. Specifically, Vid2Sim first reconstructs the observed configuration of the physical system from video using a feed-forward neural network trained to capture physical world knowledge. A lightweight optimization pipeline then refines the estimated appearance, geometry, and physical properties to closely align with video observations within just a few minutes. Additionally, after the reconstruction, Vid2Sim enables high-quality, mesh-free simulation with high efficiency. Extensive experiments demonstrate that our method achieves superior accuracy and efficiency in reconstructing geometry and physical properties from video data.

Figures

Figures reproduced from arXiv: 2506.06440 by the authors.

Figure 1
Figure 1. Vid2Sim achieves high-quality reconstruction of appearance, geometry, and physics from multi-view videos effectively. The [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. An overview of Vid2Sim, comprising two stages. In Stage I, a generalizable feed-forward model reconstructs appearance, geometry, and physical properties, generating simulation-ready outputs. In Stage II, a lightweight optimization pipeline refines these estimated attributes to closely match the input video. We introduce a mesh-free reduced simulation based on Linear Blend Skinning (LBS), which provides high computat… view at source ↗
Figure 3
Figure 3. Comparison with the SOTA methods [5, 32, 64] on physics-aware dynamic reconstruction from multi-view videos (reference). Our method achieves the best quality in terms of textured shape and physical dynamics. method Vid2Sim achieves a much higher quality of recon￾struction for appearance and physics compared with all the SOTA methods across different objects. To be more spe￾cific, previous methods rely on optimizing … view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Visualization of dynamic reconstruction results of [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Detailed pipeline of the large video vision transformer. [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: Network structure of LBS network and Jacobian network. [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: More dynamic reconstruction results from the input videos. [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: More dynamic reconstruction results from the input videos. [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: More dynamic reconstruction results from real-world input videos. [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: Novel view synthesis of the dynamic reconstruction results. [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]
Figure 11
Figure 11. Figure 11: Simulation with different materials. We use [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]
Figure 12
Figure 12. Figure 12: Simulation results based on different boundary conditions. [PITH_FULL_IMAGE:figures/full_fig_p020_12.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

67 extracted references · 39 canonical work pages

  1. [1]

    Neural jaco- bian fields: Learning intrinsic mappings of arbitrary meshes

    Noam Aigerman, Kunal Gupta, Vladimir G Kim, Siddhartha Chaudhuri, Jun Saito, and Thibault Groueix. Neural jaco- bian fields: Learning intrinsic mappings of arbitrary meshes. arXiv preprint arXiv:2205.02904, 2022. 1

    arXiv 2022

  2. [2]

    Lumiere: A space- time diffusion model for video generation

    Omer Bar-Tal, Hila Chefer, Omer Tov, Charles Her- rmann, Roni Paiss, Shiran Zada, Ariel Ephrat, Junhwa Hur, Guanghui Liu, Amit Raj, et al. Lumiere: A space- time diffusion model for video generation. arXiv preprint arXiv:2401.12945, 2024. 3

    arXiv 2024

  3. [3]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023

    arXiv 2023

  4. [4]

    Video generation models as world simulators

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luh- man, Eric Luhman, et al. Video generation models as world simulators. 2024. URL https://openai. com/research/video- generation-models-as-world-simulators, 3, 2024. 3

    work page 2024

  5. [5]

    Gaussian- informed continuum for physical property identification and simulation

    Junhao Cai, Yuji Yang, Weihao Yuan, Yisheng He, Zilong Dong, Liefeng Bo, Hui Cheng, and Qifeng Chen. Gaussian- informed continuum for physical property identification and simulation. arXiv preprint arXiv:2406.14927, 2024. 2, 5, 6, 7, 3

    arXiv 2024

  6. [6]

    Can-pinn: A fast physics-informed neural network based on coupled-automatic–numerical dif- ferentiation method

    Pao-Hsiung Chiu, Jian Cheng Wong, Chinchun Ooi, My Ha Dao, and Yew-Soon Ong. Can-pinn: A fast physics-informed neural network based on coupled-automatic–numerical dif- ferentiation method. Computer Methods in Applied Mechan- ics and Engineering, 395:114909, 2022. 2

    work page 2022

  7. [7]

    Fast and accurate deep network learn- ing by exponential linear units (elus)

    Djork-Arn ´e Clevert. Fast and accurate deep network learn- ing by exponential linear units (elus). arXiv preprint arXiv:1511.07289, 2015. 1

    arXiv 2015

  8. [8]

    Scientific machine learning through physics– informed neural networks: Where we are and what’s next

    Salvatore Cuomo, Vincenzo Schiano Di Cola, Fabio Gi- ampaolo, Gianluigi Rozza, Maziar Raissi, and Francesco Piccialli. Scientific machine learning through physics– informed neural networks: Where we are and what’s next. Journal of Scientific Computing, 92(3):88, 2022. 2

    work page 2022

Show all 67 references
  1. [9]

    A procedural approach to authoring solid models

    Barbara Cutler, Julie Dorsey, Leonard McMillan, Matthias M¨uller, and Robert Jagnow. A procedural approach to authoring solid models. ACM Transactions on Graphics (TOG), 21(3):302–311, 2002. 2

    2002

  2. [10]

    Objaverse: A universe of annotated 3d objects

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Rec...

    2023

  3. [11]

    Smoothed particles: A new paradigm for ani- mating highly deformable bodies

    M Desbrun. Smoothed particles: A new paradigm for ani- mating highly deformable bodies. Computer Animation and Simulation/Springer Vienna, 1996. 2

    1996

  4. [12]

    Google scanned objects: A high- quality dataset of 3d scanned household items

    Laura Downs, Anthony Francis, Nate Koenig, Brandon Kin- man, Ryan Hickman, Krista Reymann, Thomas B McHugh, and Vincent Vanhoucke. Google scanned objects: A high- quality dataset of 3d scanned household items. In 2022 In- ternational Conference on Robotics and Automation (ICRA...

    2022

  5. [13]

    Diffpd: Differentiable projective dynamics

    Tao Du, Kui Wu, Pingchuan Ma, Sebastien Wah, Andrew Spielberg, Daniela Rus, and Wojciech Matusik. Diffpd: Differentiable projective dynamics. ACM Transactions on Graphics (TOG), 41(2):1–21, 2021. 2

    2021

  6. [14]

    Elastogen: 4d generative elastodynamics

    Yutao Feng, Yintong Shang, Xiang Feng, Lei Lan, Shandian Zhe, Tianjia Shao, Hongzhi Wu, Kun Zhou, Hao Su, Chen- fanfu Jiang, et al. Elastogen: 4d generative elastodynamics. arXiv preprint arXiv:2405.15056, 2024. 3

    2024

  7. [15]

    Pie-nerf: Physics-based in- teractive elastodynamics with nerf

    Yutao Feng, Yintong Shang, Xuan Li, Tianjia Shao, Chen- fanfu Jiang, and Yin Yang. Pie-nerf: Physics-based in- teractive elastodynamics with nerf. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4450–4461, 2024. 3

    2024

  8. [16]

    Dynamic view synthesis from dynamic monocular video

    Chen Gao, Ayush Saraf, Johannes Kopf, and Jia-Bin Huang. Dynamic view synthesis from dynamic monocular video. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5712–5721, 2021. 2

    2021

  9. [17]

    Add: Analytically differentiable dynamics for multi-body systems with frictional contact

    Moritz Geilinger, David Hahn, Jonas Zehnder, Moritz B¨acher, Bernhard Thomaszewski, and Stelian Coros. Add: Analytically differentiable dynamics for multi-body systems with frictional contact. ACM Transactions on Graphics (TOG), 39(6):1–15, 2020. 2

    2020

  10. [18]

    Hypernetworks

    David Ha, Andrew Dai, and Quoc V Le. Hypernetworks. arXiv preprint arXiv:1609.09106, 2016. 4

    2016 arXiv

  11. [19]

    Disect: A differentiable simulation engine for autonomous robotic cutting

    Eric Heiden, Miles Macklin, Yashraj Narang, Dieter Fox, Animesh Garg, and Fabio Ramos. Disect: A differentiable simulation engine for autonomous robotic cutting. arXiv preprint arXiv:2105.12244, 2021. 2

    2021 arXiv

  12. [20]

    Gaussian error linear units (gelus)

    Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415, 2016. 1

    2016 arXiv

  13. [21]

    Lrm: Large reconstruction model for single image to 3d

    Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. Lrm: Large reconstruction model for single image to 3d. arXiv preprint arXiv:2311.04400, 2023. 2

    2023 arXiv

  14. [22]

    A moving least squares material point method with displacement disconti- nuity and two-way rigid body coupling

    Yuanming Hu, Yu Fang, Ziheng Ge, Ziyin Qu, Yixin Zhu, Andre Pradhana, and Chenfanfu Jiang. A moving least squares material point method with displacement disconti- nuity and two-way rigid body coupling. ACM Transactions on Graphics (TOG), 37(4):1–14, 2018. 2

    2018

  15. [23]

    Difftaichi: Differentiable programming for physical simulation

    Yuanming Hu, Luke Anderson, Tzu-Mao Li, Qi Sun, Nathan Carr, Jonathan Ragan-Kelley, and Fr´edo Durand. Difftaichi: Differentiable programming for physical simulation. arXiv preprint arXiv:1910.00935, 2019. 2

    1910 arXiv

  16. [24]

    gradsim: Differentiable simulation for sys- tem identification and visuomotor control

    Krishna Murthy Jatavallabhula, Miles Macklin, Florian Golemo, Vikram V oleti, Linda Petrini, Martin Weiss, Brean- dan Considine, J ´erˆome Parent-L´evesque, Kevin Xie, Kenny Erleben, et al. gradsim: Differentiable simulation for sys- tem identification and visuomotor control. ...

    2021 arXiv

  17. [25]

    The material point method for simulating continuum materials

    Chenfanfu Jiang, Craig Schroeder, Joseph Teran, Alexey Stomakhin, and Andrew Selle. The material point method for simulating continuum materials. In Acm siggraph 2016 courses, pages 1–52. 2016. 2

    2016

  18. [26]

    Vr-gs: A physical dynamics-aware interactive gaussian splatting system in virtual reality

    Ying Jiang, Chang Yu, Tianyi Xie, Xuan Li, Yutao Feng, Huamin Wang, Minchen Li, Henry Lau, Feng Gao, Yin Yang, et al. Vr-gs: A physical dynamics-aware interactive gaussian splatting system in virtual reality. In ACM SIG- GRAPH 2024 Conference Papers, pages 1–1, 2024. 3

    2024

  19. [27]

    Improving physics-augmented continuum neural radiance field-based geometry-agnostic system iden- tification with lagrangian particle optimization

    Takuhiro Kaneko. Improving physics-augmented continuum neural radiance field-based geometry-agnostic system iden- tification with lagrangian particle optimization. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5470–5480, 2024. 2, 5

    2024

  20. [28]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139–1,

  21. [29]

    Adam: A method for stochastic optimization

    Diederick P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. 2015. 5

    2015

  22. [30]

    Fast corotated elastic sph solids with implicit zero-energy mode control

    Tassilo Kugelstadt, Jan Bender, Jos ´e Antonio Fern ´andez- Fern´andez, Stefan Rhys Jeske, Fabian L ¨oschner, and An- dreas Longva. Fast corotated elastic sph solids with implicit zero-energy mode control. Proceedings of the ACM on Com- puter Graphics and Interactive Technique...

    2021

  23. [31]

    Learning a model of facial shape and expression from 4d scans

    Tianye Li, Timo Bolkart, Michael J Black, Hao Li, and Javier Romero. Learning a model of facial shape and expression from 4d scans. ACM Trans. Graph., 36(6):194–1, 2017. 2

    2017

  24. [32]

    Pac-nerf: Physics augmented continuum neural ra- diance fields for geometry-agnostic system ident ification

    Xuan Li, Yi-Ling Qiao, Peter Yichen Chen, Krishna Murthy Jatavallabhula, Ming Lin, Chenfanfu Jiang, and Chuang Gan. Pac-nerf: Physics augmented continuum neural ra- diance fields for geometry-agnostic system ident ification. arXiv preprint arXiv:2303.05512, 2023. 2, 5, 6, 3

    2023 arXiv

  25. [33]

    Real-time high-resolution background matting

    Shanchuan Lin, Andrey Ryabtsev, Soumyadip Sen- gupta, Brian Curless, Steve Seitz, and Ira Kemelmacher- Shlizerman. Real-time high-resolution background matting. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021. 5

    2021

  26. [34]

    Physics3d: Learning physical properties of 3d gaussians via video diffusion.arXiv preprint arXiv:2406.04338, 2024

    Fangfu Liu, Hanyang Wang, Shunyu Yao, Shengjun Zhang, Jie Zhou, and Yueqi Duan. Physics3d: Learning physical properties of 3d gaussians via video diffusion.arXiv preprint arXiv:2406.04338, 2024. 3

    2024 arXiv

  27. [35]

    Physgen: Rigid-body physics-grounded image- to-video generation

    Shaowei Liu, Zhongzheng Ren, Saurabh Gupta, and Shen- long Wang. Physgen: Rigid-body physics-grounded image- to-video generation. In European Conference on Computer Vision, pages 360–378. Springer, 2025. 3

    2025

  28. [36]

    Smpl: A skinned multi- person linear model

    Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J Black. Smpl: A skinned multi- person linear model. In Seminal Graphics Papers: Pushing the Boundaries, Volume 2, pages 851–866. 2023. 2

    2023

  29. [37]

    Manigaussian: Dynamic gaus- sian splatting for multi-task robotic manipulation

    Guanxing Lu, Shiyi Zhang, Ziwei Wang, Changliu Liu, Ji- wen Lu, and Yansong Tang. Manigaussian: Dynamic gaus- sian splatting for multi-task robotic manipulation. In Eu- ropean Conference on Computer Vision , pages 349–366. Springer, 2025. 3

    2025

  30. [38]

    Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis

    Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis. arXiv preprint arXiv:2308.09713, 2023. 2

    2023 arXiv

  31. [39]

    Risp: Rendering-invariant state predictor with differentiable simulation and render- ing for cross-domain parameter estimation

    Pingchuan Ma, Tao Du, Joshua B Tenenbaum, Wojciech Matusik, and Chuang Gan. Risp: Rendering-invariant state predictor with differentiable simulation and render- ing for cross-domain parameter estimation. arXiv preprint arXiv:2205.05678, 2022. 2

    2022 arXiv

  32. [40]

    Nerf: Representing scenes as neural radiance fields for view syn- thesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. Communications of the ACM , 65(1):99–106, 2021. 2

    2021

  33. [41]

    Simplicits: Mesh-free, geometry-agnostic elastic simulation

    Vismay Modi, Nicholas Sharp, Or Perel, Shinjiro Sueda, and David IW Levin. Simplicits: Mesh-free, geometry-agnostic elastic simulation. ACM Transactions on Graphics (TOG) , 43(4):1–11, 2024. 2, 3, 4, 5, 1

    2024

  34. [42]

    Nerfies: Deformable neural radiance fields

    Keunhong Park, Utkarsh Sinha, Jonathan T Barron, Sofien Bouaziz, Dan B Goldman, Steven M Seitz, and Ricardo Martin-Brualla. Nerfies: Deformable neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5865–5874, 2021. 2

    2021

  35. [43]

    Hypernerf: A higher- dimensional representation for topologically varying neural radiance fields

    Keunhong Park, Utkarsh Sinha, Peter Hedman, Jonathan T Barron, Sofien Bouaziz, Dan B Goldman, Ricardo Martin- Brualla, and Steven M Seitz. Hypernerf: A higher- dimensional representation for topologically varying neural radiance fields. arXiv preprint arXiv:2106.13228, 2021. 2

    2021 arXiv

  36. [44]

    An implicit sph formulation for incompressible linearly elastic solids

    Andreas Peer, Christoph Gissler, Stefan Band, and Matthias Teschner. An implicit sph formulation for incompressible linearly elastic solids. In Computer Graphics Forum, pages 135–148. Wiley Online Library, 2018. 2

    2018

  37. [45]

    D-nerf: Neural radiance fields for dynamic scenes

    Albert Pumarola, Enric Corona, Gerard Pons-Moll, and Francesc Moreno-Noguer. D-nerf: Neural radiance fields for dynamic scenes. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 10318–10327, 2021. 2

    2021

  38. [46]

    Differentiable simulation of soft multi-body systems

    Yiling Qiao, Junbang Liang, Vladlen Koltun, and Ming Lin. Differentiable simulation of soft multi-body systems. Advances in Neural Information Processing Systems , 34: 17123–17135, 2021. 2

    2021

  39. [47]

    Neu- physics: Editable neural geometry and physics from monoc- ular videos

    Yi-Ling Qiao, Alexander Gao, and Ming Lin. Neu- physics: Editable neural geometry and physics from monoc- ular videos. Advances in Neural Information Processing Sys- tems, 35:12841–12854, 2022. 2

    2022

  40. [48]

    L4gm: Large 4d gaus- sian reconstruction model

    Jiawei Ren, Kevin Xie, Ashkan Mirzaei, Hanxue Liang, Xi- aohui Zeng, Karsten Kreis, Ziwei Liu, Antonio Torralba, Sanja Fidler, Seung Wook Kim, et al. L4gm: Large 4d gaus- sian reconstruction model. arXiv preprint arXiv:2406.10324,

    arXiv

  41. [49]

    Dif- ferentiable implicit soft-body physics

    Junior Rojas, Eftychios Sifakis, and Ladislav Kavan. Dif- ferentiable implicit soft-body physics. arXiv preprint arXiv:2102.05791, 2021. 2

    2021 arXiv

  42. [50]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 3

    2022

  43. [51]

    Em- bodied hands: Modeling and capturing hands and bodies to- gether

    Javier Romero, Dimitrios Tzionas, and Michael J Black. Em- bodied hands: Modeling and capturing hands and bodies to- gether. arXiv preprint arXiv:2201.02610, 2022. 2

    2022 arXiv

  44. [52]

    Decaf: Monocular deformation capture for face and hand interactions

    Soshi Shimada, Vladislav Golyanik, Patrick P ´erez, and Christian Theobalt. Decaf: Monocular deformation capture for face and hand interactions. ACM Transactions on Graph- ics (ToG), 42(6):1–16, 2023. 2

    2023

  45. [53]

    Lgm: Large multi-view gaussian model for high-resolution 3d content creation

    Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, and Ziwei Liu. Lgm: Large multi-view gaussian model for high-resolution 3d content creation. In European Conference on Computer Vision, pages 1–18. Springer, 2025. 2, 4, 7

    2025

  46. [54]

    Videomae: Masked autoencoders are data-efficient learners for self-supervised video pre-training

    Zhan Tong, Yibing Song, Jue Wang, and Limin Wang. Videomae: Masked autoencoders are data-efficient learners for self-supervised video pre-training. Advances in neural information processing systems, 35:10078–10093, 2022. 2, 4, 1

    2022

  47. [55]

    Non- rigid neural radiance fields: Reconstruction and novel view synthesis of a dynamic scene from monocular video

    Edgar Tretschk, Ayush Tewari, Vladislav Golyanik, Michael Zollh¨ofer, Christoph Lassner, and Christian Theobalt. Non- rigid neural radiance fields: Reconstruction and novel view synthesis of a dynamic scene from monocular video. In Proceedings of the IEEE/CVF International Con...

    2021

  48. [56]

    4d gaussian splatting for real-time dynamic scene rendering

    Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4d gaussian splatting for real-time dynamic scene rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20310–20320, 2024. 2

    2024

  49. [57]

    Dice: End-to-end deformation capture of hand-face interactions from a single image

    Qingxuan Wu, Zhiyang Dou#, Sirui Xu, Soshi Shimada, Chen Wang, Zhengming Yu, Yuan Liu, Cheng Lin, Zeyu Cao, Taku Komura, et al. Dice: End-to-end deformation capture of hand-face interactions from a single image. arXiv preprint arXiv:2406.17988, 2024. 2

    2024 arXiv

  50. [58]

    Space-time neural irradiance fields for free-viewpoint video

    Wenqi Xian, Jia-Bin Huang, Johannes Kopf, and Changil Kim. Space-time neural irradiance fields for free-viewpoint video. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9421–9431,

  51. [59]

    Physgaussian: Physics- integrated 3d gaussians for generative dynamics

    Tianyi Xie, Zeshun Zong, Yuxing Qiu, Xuan Li, Yutao Feng, Yin Yang, and Chenfanfu Jiang. Physgaussian: Physics- integrated 3d gaussians for generative dynamics. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4389–4398, 2024. 3

    2024

  52. [60]

    Real-time photorealistic dynamic scene representation and rendering with 4d gaussian splatting

    Zeyu Yang, Hongye Yang, Zijie Pan, and Li Zhang. Real-time photorealistic dynamic scene representation and rendering with 4d gaussian splatting. arXiv preprint arXiv:2310.10642, 2023. 2

    2023 arXiv

  53. [61]

    Deformable 3d gaussians for high- fidelity monocular dynamic scene reconstruction

    Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. Deformable 3d gaussians for high- fidelity monocular dynamic scene reconstruction. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20331–20341, 2024. 2

    2024

  54. [62]

    Gs-lrm: Large recon- struction model for 3d gaussian splatting

    Kai Zhang, Sai Bi, Hao Tan, Yuanbo Xiangli, Nanxuan Zhao, Kalyan Sunkavalli, and Zexiang Xu. Gs-lrm: Large recon- struction model for 3d gaussian splatting. In European Con- ference on Computer Vision, pages 1–19. Springer, 2025. 2

    2025

  55. [63]

    Physics-based interaction with 3d ob- jects via video generation

    Tianyuan Zhang, Hong-Xing Yu, Rundi Wu, Brandon Y Feng, Changxi Zheng, Noah Snavely, Jiajun Wu, and William T Freeman. Physics-based interaction with 3d ob- jects via video generation. 3

  56. [64]

    Reconstruction and simulation of elastic objects with spring- mass 3d gaussians

    Licheng Zhong, Hong-Xing Yu, Jiajun Wu, and Yunzhu Li. Reconstruction and simulation of elastic objects with spring- mass 3d gaussians. In European Conference on Computer Vision, pages 407–423. Springer, 2025. 2, 5, 6, 7, 3 Vid2Sim: Generalizable, Video-based Reconstruction of...

    2025

  57. [65]

    Large Video Vision Transformer The pipeline of our Large Video Vision Transformer is shown in Fig

    More Implementation Details 9.1. Large Video Vision Transformer The pipeline of our Large Video Vision Transformer is shown in Fig. 5. In our framework, we fine-tune the back- bone network, VideoMAE [54], which is pre-trained on 16- frame videos at a resolution of 224 × 224. T...

    2000

  58. [66]

    More Results on Dynamic Reconstruction In Sec. 10.1, we provide a comprehensive investigation by showcasing additional qualitative results of dynamic recon- struction and future state prediction across baselines, our Stage I model, and our full model. In Sec. 10.2, we eval- ua...

  59. [67]

    11.1 and provide additional simulation results on different boundary conditions in Sec

    Generalization Capability We provide more simulation results on changed materials in Sec. 11.1 and provide additional simulation results on different boundary conditions in Sec. 11.2. 11.1. Generalized to Different Materials Although our method mainly focuses on reconstructing...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.