REVIEW 9 cited by
gradSim: Differentiable simulation for system identification and visuomotor control
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
gradSim: Differentiable simulation for system identification and visuomotor control
read the original abstract
We consider the problem of estimating an object's physical properties such as mass, friction, and elasticity directly from video sequences. Such a system identification problem is fundamentally ill-posed due to the loss of information during image formation. Current solutions require precise 3D labels which are labor-intensive to gather, and infeasible to create for many systems such as deformable solids or cloth. We present gradSim, a framework that overcomes the dependence on 3D supervision by leveraging differentiable multiphysics simulation and differentiable rendering to jointly model the evolution of scene dynamics and image formation. This novel combination enables backpropagation from pixels in a video sequence through to the underlying physical attributes that generated them. Moreover, our unified computation graph -- spanning from the dynamics and through the rendering process -- enables learning in challenging visuomotor control tasks, without relying on state-based (3D) supervision, while obtaining performance competitive to or better than techniques that rely on precise 3D labels.
Forward citations
Cited by 9 Pith papers
-
Fast and Reliable Gradients for Deformables Across Frictional Contact Regimes
An analytical adjoint with smoothed Fischer–Burmeister contact and a Woodbury-preconditioned Krylov solver yields stable gradients for frictional deformable contact.
-
Learning Physics-Guided Residual Dynamics for Deformable Object Simulation
Physics-guided residual dynamics, a spring-mass simulator plus a network that predicts velocity corrections, yields the most accurate deformable-object simulation in the paper's real-world tests.
-
RigPI: Dynamic Parameter Identification of Rigid Body via VLM-Seeded Differentiable Simulation
RigPI combines VLM semantic priors with two-stage gradient optimization in differentiable simulation to identify inertial and frictional parameters of rigid bodies from robot-object interactions.
-
$\Delta$ynamics: Language-Based Representation for Inferring Rigid-Body Dynamics From Videos
A vision-language framework generates text-based rigid-body scene configurations from videos using motion reasoning and optical flow, reporting 0.30 IoU on CLEVRER (7x over baselines) and transfer to 235 real videos.
-
Zero-Shot Sim-to-Real Robot Learning: A Dexterous Manipulation Study on Reactive Catching
DRIS improves zero-shot sim-to-real transfer for reactive catching by maintaining and acting on sets of randomized dynamics instances instead of single instances per episode.
-
Robust and Efficient MuJoCo-based Model Predictive Control via Web of Affine Spaces Derivatives
WASP derivative reuse speeds up MuJoCo MPC by 1.26–2.08x versus finite differences on selected locomotion tasks, with comparable task costs.
-
RigPI: Dynamic Parameter Identification of Rigid Body via VLM-Seeded Differentiable Simulation
RigPI combines VLM initialization with two-stage gradient-based optimization in differentiable simulation to estimate dynamic parameters of rigid bodies from real robot interactions.
-
Physically Viable World Models: A Case for Query-Conditioned Embodied AI
Embodied AI requires query-conditioned world models that select the simplest physical abstraction sufficient to answer intervention queries.
-
Fast and Reliable Gradients for Deformables Across Frictional Contact Regimes
The authors develop a differentiable simulator enforcing Markovian dynamics on a position-velocity manifold and using a mass-aligned preconditioner with a soft Fischer-Burmeister operator to produce stable gradients f...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.