Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Learning velocity corrections on a spring-mass simulator outperforms pure physics and pure learning for deformable-object prediction.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 05:08 UTC pith:OVNXF74V

load-bearing objection Solid hybrid sim paper with a real-robot win, but its headline metrics rest on an unvalidated tracking pipeline; worth serious review, not desk rejection. the 4 major comments →

arxiv 2607.13451 v1 pith:OVNXF74V submitted 2026-07-15 cs.RO cs.AIcs.CV

Learning Physics-Guided Residual Dynamics for Deformable Object Simulation

Shivansh Patel , Kaifeng Zhang , Sanjay Pokkali , Svetlana Lazebnik , Yunzhu Li This is my paper
classification cs.RO cs.AIcs.CV
keywords deformable object simulationresidual dynamicsspring-mass modelphysics-guided learningvelocity-based residualmodel predictive control3D Gaussian splattingrobotic manipulation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes Physics-Guided Residual Dynamics (PGRD), a hybrid simulator that wraps a spring-mass physics engine in a neural network predicting per-particle velocity corrections. The claim is that this combination consistently beats both purely physics-based and purely learning-based baselines on tracking accuracy, and that the corrected simulator can serve as a forward model for robotic planning and photorealistic video prediction. The reason a sympathetic reader should care: deformable-object simulation is a bottleneck for robot manipulation of cables, cloth, and soft toys, and the paper shows a practical recipe that needs little data and remains stable over long horizons.

Core claim

PGRD's central claim is that residual corrections to a physics backbone, trained in a second stage after physics parameter fitting, capture dynamics that neither component alone can represent. The key is to predict residual velocities rather than positional offsets, which keeps the corrected state consistent and avoids simulator blow-ups. On six real objects—rope, paper, flag, sloth toy, duster, teddy—PGRD reports the lowest mean distance, chamfer distance, and earth mover's distance, and is the only method that correctly handles a heterogeneous duster with a rigid stem and soft feathers. The same forward model drives a model-predictive control planner that reroutes a cable through a narrow

What carries the argument

The central object is the residual velocity field: at each step, an optimized spring-mass simulator produces positions and velocities, and a Point Transformer encodes the state and history to predict a per-particle velocity correction that is time-integrated into the final position. A sliding-window transformer with a gated blend stabilizes the temporal refinement. Because the physics backbone provides the coarse dynamics, the network only learns the gap, which keeps data requirements low and rollouts stable.

Load-bearing premise

The 3D point tracks used as ground truth come from an image tracker lifted to 3D with depth, and every loss and metric is computed against those tracks; if the tracks drift or lose particles, the learned residuals and all reported errors inherit that error.

What would settle it

Use a synthetic scene with known ground-truth dynamics and run the same pipeline: if the residual network fails to reduce the error to near zero (or learns nonzero corrections when the physics backbone is already exact), the reported gains on real objects may reflect tracking noise rather than physics. Alternatively, re-evaluate on real objects with tracks replaced by a marker-based motion-capture ground truth.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If correct, PGRD offers a practical path to accurate deformable-object simulation from a few minutes of RGBD data.
  • The velocity-residual formulation may generalize to other physics backbones, not just spring-mass.
  • Accurate forward models enable model-predictive control for cable routing, cloth smoothing, and similar tasks.
  • Action-conditioned video prediction turns the simulator into a visual planner that can use generated goal images from language commands.
  • Heterogeneous objects with mixed rigid and soft parts become tractable without hand-tuning material models.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The two-stage pipeline suggests that physics parameter fitting and residual learning can be decoupled, making PGRD a drop-in accuracy layer for existing simulators.
  • Because the residual is trained on only a few minutes of data per object, the approach hints at sample-efficient sim-to-real transfer, though the paper does not test how accuracy degrades with less data.
  • A natural extension is to use the residual network with a differentiable physics backbone, which could permit joint end-to-end training rather than the current sequential fitting.
  • The k-nearest-neighbor velocity propagation in the tracking pipeline could drift over long rollouts; if that drift is correlated with the residual, the learned corrections might partly encode tracker artifacts rather than true dynamics.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Physics-Guided Residual Dynamics (PGRD), a hybrid deformable-object simulator that combines an optimizable spring-mass backbone with a learned per-particle residual velocity network. The physics parameters are first calibrated with CMA-ES; the residual network (PTv3 encoder, NeRF-style decoder, and a gated sliding-window transformer) is then trained to correct the discrepancy between the physics prediction and tracked 3D point trajectories, using multi-step rollouts and a velocity-based formulation to avoid instability. Experiments on six real objects report tracking metrics (MDE, CD, EMD) and action-conditioned video metrics (IoU, F-score, LPIPS) showing PGRD with the lowest errors across objects and metrics. The paper also presents MPPI manipulation planning, including a cable-rerouting task with 8/10 success versus 2/10 for the backbone, language-conditioned goal generation, and interactive 3D Gaussian Splatting rendering driven by the predicted particle states.

Significance. If the empirical claims hold, PGRD is a practically attractive hybrid paradigm: it retains the interpretability and sample efficiency of a physics simulator while using a learned residual to capture effects that spring-mass models miss, such as heterogeneous stiffness and contact-rich deformation. The main strengths are the real-world multi-object evaluation, the velocity-residual formulation that avoids training instability, the multi-step rollout training that directly addresses error accumulation, and the application-level validation through cable-rerouting and language-conditioned planning. The cable-rerouting success rate (Section V-A) is an especially valuable external check because it involves an interaction type not present in the training data. However, the central claim rests on the quality of automatically tracked 3D point trajectories, and the paper does not independently validate that ground truth; several reported improvements are also within one standard deviation of the best baseline. The paper is a solid contribution if these issues are resolved, but the current evidence is not fully sufficient for the strength of the claims.

major comments (4)
  1. [IV-A, App. D, Eq. (1), Table I] All training losses and all three tracking metrics are computed against 3D point trajectories obtained from CoTracker 2D tracks lifted by multi-view depth and refined by the iterative kNN velocity-propagation procedure described in App. D. This pipeline is never independently validated. CoTracker can drift or jump on occluded, fast-moving, self-contacting deformable objects; depth sensors add noise; and the kNN rollout averages velocities from neighboring points, which can smooth real motion or propagate correspondence errors. Since the same tracks define both the fitted residual and the evaluation, systematic tracking error could be absorbed into the model and also corrupt the baselines. Some margins are small (e.g., Flag EMD 2.2 vs 2.3; Duster CD 3.8 vs 4.0), so even sub-centimeter track error could change rankings. Please provide independent validation of the tracking pipeline (e.g.,
  2. [III-A, App. D, Eq. (1)] For volumetric objects (Sloth Toy, Teddy Toy), Section III-A introduces internal particles sampled inside a watertight mesh to prevent collapse, and the ablation in Table III shows that removing them degrades MDE from 2.7 to 3.8 on the Sloth Toy. However, the paper never specifies the ground-truth targets for these internal particles. App. D describes tracking surface point clouds only; it does not explain how X*_t in Eq. (1) is defined for interior particles, which are not directly observable. Without this specification, the training objective and the reported volumetric-object results are ambiguous. Please state whether internal particles are supervised by some reconstructed volume field, are unobserved and only indirectly constrained, or are handled differently.
  3. [IV-A] The data-collection procedure states that consecutive episodes overlap by 2 seconds. The paper does not specify how the 100 training episodes and 20 validation episodes are split. If the split is a contiguous partition of the 120 episodes, then validation episode 101 starts at the same physical time as the final 2 seconds of training episode 100, so the validation set is not temporally disjoint from the training set. This would compromise the claim that Table I reports held-out generalization. Please state the split rule explicitly, or better, use a temporally disjoint validation split and re-report the affected results.
  4. [IV-D, Table I] The abstract and Section IV-D claim PGRD 'consistently achieves the lowest error across all objects and metrics.' Numerically this is true in Table I, but the margins are often within one standard deviation of the best baseline (e.g., Pag EMD 1.4 vs 1.5, Duster CD 3.8 vs 4.0, Flag CD exactly tied at 4.3). Since all methods are evaluated on the same episodes, the appropriate comparison is a paired significance test or confidence interval on per-episode differences, not just mean±std. Without this, the 'consistently' claim is stronger than the evidence supports. Please add paired significance testing or otherwise quantify the reliability of the ranking.
minor comments (5)
  1. [IV-C] The text says 'five baselines: two analytical physics-based simulations and two learning-based approaches,' but then lists five named baselines where Diff. Spring-Mass is a third physics-based method. Please correct the count or the description.
  2. [App. B] Typo: 'Our network network consists of three primary components' should read 'Our network consists...'.
  3. [Table III] The ablation table reports only means without standard deviations or significance. Given the small differences (e.g., 2.7 vs 2.8 for removing the temporal aggregator), add error bars or note that the differences are not statistically tested.
  4. [Fig. 3] The caption 'see text' is uninformative; a short object description in the caption would improve readability.
  5. [V-A] The sentence 'The parameters for planning experiments is provided in App. E' has a subject-verb disagreement; change 'is' to 'are'.

Circularity Check

0 steps flagged

No circularity: PGRD's residual model is a supervised fit, but all central claims are tested on held-out episodes and an untrained slot-rerouting scenario.

full rationale

The paper's core method is explicitly a two-stage supervised pipeline: the spring–mass backbone is optimized with CMA-ES on training trajectories, and the residual network is trained with L_res in Eq. (1) on the same training data. This is a fit, but not circularity: the paper's empirical claims, including Table I, are evaluated on held-out validation episodes ('For training, we use 100 episodes, and for validation, we use 20 episodes') and over a longer horizon than training ('We train PGRD with 5-step rollout, but evaluate it over the full evaluation episode length of 37 steps'). The cable-rerouting success is additionally tested on a scenario that was explicitly excluded from residual training: 'the residual dynamics model is not trained on examples containing rope-slot interaction data.' The loss and Table I both depend on the CoTracker/depth-based 3D tracks, but that is a data-quality concern, not an equivalence-by-construction; all methods are compared on the same tracks. The velocity-based residual formulation and gated transformer are design choices motivated by external references, not by a self-citation chain. The paper does cite its own prior work (e.g., PhysTwin [34], PGND [93], Patel et al. [60]), but these are used as baselines or methodological details, not as load-bearing justifications of the central claim. No uniqueness theorem is imported from the authors, and no 'prediction' is defined in terms of the quantity it is said to predict. Therefore no circular step is present, and the correct score is 0.

Axiom & Free-Parameter Ledger

4 free parameters · 7 axioms · 0 invented entities

The central claim rests on fitted parameters (spring-mass theta, residual network weights, gating scale) and several domain assumptions about tracking accuracy, simulator expressiveness, and generalization. The ledger shows that the contribution is empirical fitting and residual correction rather than a parameter-free derivation; no new physical entities are introduced.

free parameters (4)
  • Spring-mass backbone parameters theta (stiffness, damping, threshold, max springs per node, ground friction) = Per object via CMA-ES; values not reported
    Section III-A: fitted to minimize trajectory MSE on training data; these parameters define the physics backbone and directly affect residual targets.
  • Residual network weights (PTv3 encoder, NeRF-style decoder, temporal transformer, gating projections) = Not enumerated
    Trained via supervised loss Eq. (1) on position MSE; the central prediction (residual velocities) is the output of these fitted weights.
  • Gating scale 0.1 = 0.1
    Section III-C: chosen by hand to limit correction magnitude and avoid instability; included in the final residual update equation.
  • MPPI hyperparameters (trajectory count, horizon, iterations, action noise covariance) = 8/15, 10/15, 3/6, 0.1
    Appendix E: affect planning success rates; set separately for simple tasks vs cable rerouting.
axioms (7)
  • domain assumption The tracked 3D point trajectories produced by Grounded SAM 2 + CoTracker + multi-view depth fusion are accurate ground truth for training and evaluation.
    Section IV-A and App D: The supervised loss and all metrics are computed against these tracks; any tracking drift or correspondence error propagates directly into fitted residuals and reported errors. No independent validation of tracking accuracy is given.
  • domain assumption Spring-mass discretization with semi-implicit Euler and local springs is an adequate backbone; residual velocities can correct its systematic errors.
    Section III-A: The backbone's functional form is assumed to capture broad deformation features; if it is too poor (e.g., cannot represent topology changes), residuals may not compensate.
  • domain assumption Multi-step training with 5-step rollouts transfers to full 37-step evaluation rollouts.
    Section III-B/IV-A: Multi-step rollouts reduce distribution shift, but there is no formal guarantee for longer horizons; evaluation only covers 37 steps.
  • domain assumption CMA-ES optimization on training trajectories finds a good enough physics parameter set.
    Section III-A: Relies on black-box optimization convergence; no analysis of identifiability, multiple optima, or sensitivity to initialization.
  • domain assumption 3D Gaussian Splatting with fixed appearance and updated positions yields valid visual predictions.
    Section IV-B: Visual metrics assume appearance/color of Gaussians remains constant under deformation; this may fail under occlusions, lighting changes, and large topological changes.
  • domain assumption The language-conditioned goal image generation (Nano Banana Pro) and monocular depth alignment produce a reasonable target point cloud.
    Section V-A: Goal quality is not quantitatively evaluated; depth alignment via affine fit to background assumes the generated image is metric and the background is static.
  • domain assumption The object state is fully observable from RGBD surface points, with internal particles sufficient for volumetric objects.
    Section III-A and App D: Simulation starts from reconstructed point clouds; holes, reconstruction errors, and missing internal structure affect the initial state and all subsequent predictions.

pith-pipeline@v1.3.0-alltime-deepseek · 21402 in / 14320 out tokens · 138700 ms · 2026-08-02T05:08:45.575170+00:00 · methodology

0 comments
read the original abstract

Simulating deformable objects is essential for a wide range of robotic manipulation applications, yet accurately predicting their dynamics remains challenging. We propose Physics-Guided Residual Dynamics (PGRD), a hybrid simulation framework that combines the advantages of physics-based and learning-based approaches. Specifically, PGRD combines an optimizable spring-mass simulator as a backbone with a learned neural network that predicts residual corrections to the physics-based predictions. We adopt a velocity-based formulation to ensure stable simulation and a sliding-window transformer architecture to capture temporal dependencies. We show that PGRD produces more accurate results than both purely physics-based and learning-based methods on a set of diverse real-world deformable objects. We further demonstrate the utility of PGRD in two applications: manipulation planning via Model Predictive Control, including a language-conditioned setting with a generated goal image; and interactive simulation via action-conditioned video prediction by 3D Gaussian Splatting.

Figures

Figures reproduced from arXiv: 2607.13451 by Kaifeng Zhang, Sanjay Pokkali, Shivansh Patel, Svetlana Lazebnik, Yunzhu Li.

Figure 1
Figure 1. Figure 1: Physics-Guided Residual Dynamics Overview. Given RGBD observations, we extract surface points to instantiate the simulation. Red dots indicate the point where gripper held the object. The framework rolls out an optimized spring-mass physics backbone from the current object state Xt, Vt and robot actions at to produce simulation prediction X sim t+1, V sim t+1. Subsequently, our residual dynamics network pr… view at source ↗
Figure 2
Figure 2. Figure 2: Network Architecture. The points encoder extracts spa￾tiotemporal features, which the decoder projects to estimate initial residual velocity. Finally, the temporal aggregator refines these esti￾mates using a recurrent history to output the final residuals ∆V residual t+1 . To parameterize the residual velocity, we design a network consisting of three primary components: encoder, decoder, and temporal aggre… view at source ↗
Figure 3
Figure 3. Figure 3: Experimental Objects (see text). a) Rope. A mostly one-dimensional object grasped at one end and manipulated by dragging across the table or lifting and lowering motions. b) Paper. A thin two-dimensional sheet grasped at the top and manipulated by suspending it in the air while per￾forming waving motions, exhibiting bending and twisting. c) Flag. A cloth flag, demonstrating large area deformation dynamics … view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative comparison of PGRD on dynamics prediction. Columns shaded in green show the input state with gripper positions (red spheres) and the ground truth object configuration after taking the action. The remaining columns show predictions from each method. PGRD accurately captures deformations across all objects. For the duster, only PGRD maintains stem rigidity while allowing deformation for the feath… view at source ↗
Figure 5
Figure 5. Figure 5: Example pair used to compute the visual metrics. We evaluate only the object of interest, ignoring the background. The mask error visualization shows true-positive object pixels in black, false negatives in red, and false positives in cyan. The RGB error heat map shows the per-pixel appearance difference within the evaluated object region, with stronger colors indicating larger RGB error. Part of the upper… view at source ↗
Figure 6
Figure 6. Figure 6: Planning results on rope manipulation tasks. Top row shows start and end configurations for four tasks. The bottom row shows the Chamfer Distance to the target during MPPI planning. PGRD (green) converges faster and achieves lower final error than the optimized spring-mass baseline (red) across all tasks. CD = 0.04 CD = 0.12 Spring -Mass Ours t Start and Goal Configuration [PITH_FULL_IMAGE:figures/full_fi… view at source ↗
Figure 8
Figure 8. Figure 8: Cable rerouting through narrow slot. Left: distribution of start and end configurations. Right: Chamfer Distance during MPPI planning steps. PGRD achieves lower distance than the spring-mass baseline, successfully threading the rope through the slot in 8 out of 10 trials compared to 2 out of 10 for the baseline. mass model converges more slowly and sometimes fails to reach a comparable final error. Additio… view at source ↗
Figure 9
Figure 9. Figure 9: Execution of PGRD with generated goals. The first column shows the initial image, and the second column shows the language￾conditioned generated goal image. Subsequent columns show robot execution from a higher-quality demo camera. The target point cloud from the goal image is overlaid on the last image. PGRD successfully executes a range of tasks, including passing a rope through a specified slot and movi… view at source ↗
Figure 10
Figure 10. Figure 10: Interactive rope manipulation with PGRD rendered using 3DGS. The visualization overlays three timesteps, with later configurations in lighter opacity. As PGRD predicts the dynamics, the 3D Gaussians attached to the particles are updated to render images that maintain visual consistency. VI. CONCLUSIONS This paper presented Physics-Guided Residual Dynamics (PGRD), a hybrid simulation framework that bridges… view at source ↗
Figure 11
Figure 11. Figure 11: Planning results on sloth toy manipulation tasks. Top row displays the initial and target configurations for four manipulation objectives. Bottom row plots the Chamfer Distance evolution throughout MPPI planning. PGRD (green) consistently achieves superior convergence compared to the tuned spring-mass baseline (red). each transformer layer. Finally, the gating scale controls the magnitude of temporal corr… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. PhysCoRe: Physics-Corrected Residual World Models for Material-Aware Deformable Dynamics

    cs.RO 2026-07 conditional novelty 5.0

    PhysCoRe uses a differentiable MPM simulator with neural material inference and residual velocity correction, and reports more accurate future prediction on real deformable-object manipulation than optimization baselines.

Reference graph

Works this paper leans on

99 extracted references · 29 linked inside Pith · cited by 1 Pith paper

  1. [1]

    A review of learning-based dynam- ics models for robotic manipulation.Science Robotics, 10(106):eadt1497, 2025

    Bo Ai, Stephen Tian, Haochen Shi, Yixuan Wang, Tobias Pfaff, Cheston Tan, Henrik I Christensen, Hao Su, Jiajun Wu, and Yunzhu Li. A review of learning-based dynam- ics models for robotic manipulation.Science Robotics, 10(106):eadt1497, 2025

    2025

  2. [2]

    Augmenting physical simulators with stochastic neural networks: Case study of planar pushing and bouncing

    Anurag Ajay, Jiajun Wu, Nima Fazeli, Maria Bauza, Leslie P Kaelbling, Joshua B Tenenbaum, and Al- berto Rodriguez. Augmenting physical simulators with stochastic neural networks: Case study of planar pushing and bouncing. In2018 IEEE/RSJ International Confer- ence on Intelligent Robots and Systems (IROS), pages 3066–3073. IEEE, 2018

    2018

  3. [3]

    Rewiring techniques to mitigate oversquashing and oversmoothing in gnns: A survey.arXiv preprint arXiv:2411.17429, 2024

    Hugo Attali, Davide Buscaldi, and Nathalie Pernelle. Rewiring techniques to mitigate oversquashing and oversmoothing in gnns: A survey.arXiv preprint arXiv:2411.17429, 2024

    Pith/arXiv arXiv 2024

  4. [4]

    Large steps in cloth simulation

    David Baraff and Andrew Witkin. Large steps in cloth simulation. InProceedings of 25th Annual Conference on Computer Graphics and Interactive Techniques (SIG- GRAPH ’98), pages 43 – 54. SIGGRAPH, July 1998

    1998

  5. [5]

    Interaction networks for learning about objects, relations and physics.Advances in neural information processing systems, 29, 2016

    Peter Battaglia, Razvan Pascanu, Matthew Lai, Danilo Jimenez Rezende, et al. Interaction networks for learning about objects, relations and physics.Advances in neural information processing systems, 29, 2016

    2016

  6. [6]

    Dough- net: A visual predictive model for topological manipula- tion of deformable objects

    Dominik Bauer, Zhenjia Xu, and Shuran Song. Dough- net: A visual predictive model for topological manipula- tion of deformable objects. InEuropean Conference on Computer Vision, pages 92–108. Springer, 2024

    2024

  7. [7]

    Neurobem: Hy- brid aerodynamic quadrotor model.arXiv preprint arXiv:2106.08015, 2021

    Leonard Bauersfeld, Elia Kaufmann, Philipp Foehn, Si- hao Sun, and Davide Scaramuzza. Neurobem: Hy- brid aerodynamic quadrotor model.arXiv preprint arXiv:2106.08015, 2021

    Pith/arXiv arXiv 2021

  8. [8]

    A smooth analytical formulation of collision detection and rigid body dynamics with contact.arXiv preprint arXiv:2503.11736, 2025

    Onur Beker, Nico G ¨urtler, Ji Shi, A Ren ´e Geist, Amir- reza Razmjoo, Georg Martius, and Sylvain Calinon. A smooth analytical formulation of collision detection and rigid body dynamics with contact.arXiv preprint arXiv:2503.11736, 2025

    Pith/arXiv arXiv 2025

  9. [9]

    Combining differentiable pde solvers and graph neural networks for fluid flow prediction

    Filipe De Avila Belbute-Peres, Thomas Economon, and Zico Kolter. Combining differentiable pde solvers and graph neural networks for fluid flow prediction. Ininter- national conference on machine learning, pages 2402–

  10. [10]

    Comphy: Compositional physical reasoning of objects and events from videos.arXiv preprint arXiv:2205.01089, 2022

    Zhenfang Chen, Kexin Yi, Yunzhu Li, Mingyu Ding, Antonio Torralba, Joshua B Tenenbaum, and Chuang Gan. Comphy: Compositional physical reasoning of objects and events from videos.arXiv preprint arXiv:2205.01089, 2022

    Pith/arXiv arXiv 2022

  11. [11]

    Learning phrase representations using rnn encoder–decoder for statistical machine translation

    Kyunghyun Cho, Bart Van Merri ¨enboer, C ¸ a˘glar Gulc ¸ehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder–decoder for statistical machine translation. InProceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1724–1734, 2014

    2014

  12. [12]

    Deep reinforcement learning in a handful of trials using probabilistic dynamics models

    Kurtland Chua, Roberto Calandra, Rowan McAllister, and Sergey Levine. Deep reinforcement learning in a handful of trials using probabilistic dynamics models. Advances in neural information processing systems, 31, 2018

    2018

  13. [13]

    Physically consistent neural networks for building thermal modeling: theory and analysis.Applied Energy, 325:119806, 2022

    Loris Di Natale, Bratislav Svetozarevic, Philipp Heer, and Colin N Jones. Physically consistent neural networks for building thermal modeling: theory and analysis.Applied Energy, 325:119806, 2022

    2022

  14. [14]

    Learning humanoid end-effector control for open- vocabulary visual loco-manipulation.arXiv preprint arXiv:2602.16705, 2026

    Runpei Dong, Ziyan Li, Xialin He, and Saurabh Gupta. Learning humanoid end-effector control for open- vocabulary visual loco-manipulation.arXiv preprint arXiv:2602.16705, 2026

    Pith/arXiv arXiv 2026

  15. [15]

    Diffpd: Differentiable projective dynamics.ACM Transactions on Graphics (ToG), 41(2):1–21, 2021

    Tao Du, Kui Wu, Pingchuan Ma, Sebastien Wah, Andrew Spielberg, Daniela Rus, and Wojciech Matusik. Diffpd: Differentiable projective dynamics.ACM Transactions on Graphics (ToG), 41(2):1–21, 2021

    2021

  16. [16]

    Duisterhof, Jan Oberst, Bowen Wen, Stan Birchfield, Deva Ramanan, and Jeffrey Ichnowski

    Bardienus P. Duisterhof, Jan Oberst, Bowen Wen, Stan Birchfield, Deva Ramanan, and Jeffrey Ichnowski. Rayst3r: Predicting novel depth maps for zero-shot object completion, 2025. URL https://arxiv.org/abs/2506.05285

    Pith/arXiv arXiv 2025

  17. [17]

    Context is everything: Implicit identification for dynamics adap- tation

    Ben Evans, Abitha Thankaraj, and Lerrel Pinto. Context is everything: Implicit identification for dynamics adap- tation. In2022 International Conference on Robotics and Automation (ICRA), pages 2642–2648. IEEE, 2022

    2022

  18. [18]

    Anygrasp: Robust and efficient grasp perception in spatial and temporal domains.IEEE Transactions on Robotics, 39(5):3929–3945, 2023

    Hao-Shu Fang, Chenxi Wang, Hongjie Fang, Minghao Gou, Jirong Liu, Hengxu Yan, Wenhai Liu, Yichen Xie, and Cewu Lu. Anygrasp: Robust and efficient grasp perception in spatial and temporal domains.IEEE Transactions on Robotics, 39(5):3929–3945, 2023

    2023

  19. [19]

    Sim-to-real of soft robots with learned residual physics.IEEE Robotics and Automation Letters, 2024

    Junpeng Gao, Mike Y Michelis, Andrew Spielberg, and Robert K Katzschmann. Sim-to-real of soft robots with learned residual physics.IEEE Robotics and Automation Letters, 2024

    2024

  20. [20]

    Model predictive control: Theory and practice—a survey

    Carlos E Garcia, David M Prett, and Manfred Morari. Model predictive control: Theory and practice—a survey. Automatica, 25(3):335–348, 1989

    1989

  21. [21]

    Add: Analytically differentiable dynamics for multi-body systems with frictional contact.ACM Transactions on Graphics (TOG), 39(6):1–15, 2020

    Moritz Geilinger, David Hahn, Jonas Zehnder, Moritz B¨acher, Bernhard Thomaszewski, and Stelian Coros. Add: Analytically differentiable dynamics for multi-body systems with frictional contact.ACM Transactions on Graphics (TOG), 39(6):1–15, 2020

    2020

  22. [22]

    Sim-to-real transfer with neural- augmented robot simulation

    Florian Golemo, Adrien Ali Taiga, Aaron Courville, and Pierre-Yves Oudeyer. Sim-to-real transfer with neural- augmented robot simulation. InConference on Robot Learning, pages 817–828. PMLR, 2018

    2018

  23. [23]

    Residual model learning for microrobot con- trol

    Joshua Gruenstein, Tao Chen, Neel Doshi, and Pulkit Agrawal. Residual model learning for microrobot con- trol. In2021 IEEE International Conference on Robotics and Automation (ICRA), pages 7219–7226. IEEE, 2021

    2021

  24. [24]

    Set-valued rigid body dynamics for simultaneous frictional impact.arXiv preprint arXiv:2103.15714, 2021

    Mathew Halm and Michael Posa. Set-valued rigid body dynamics for simultaneous frictional impact.arXiv preprint arXiv:2103.15714, 2021

    Pith/arXiv arXiv 2021

  25. [25]

    Completely derandomized self-adaptation in evolution strategies

    Nikolaus Hansen and Andreas Ostermeier. Completely derandomized self-adaptation in evolution strategies. Evolutionary computation, 9(2):159–195, 2001

    2001

  26. [26]

    Disect: A differen- tiable simulation engine for autonomous robotic cutting

    Eric Heiden, Miles Macklin, Yashraj Narang, Dieter Fox, Animesh Garg, and Fabio Ramos. Disect: A differen- tiable simulation engine for autonomous robotic cutting. arXiv preprint arXiv:2105.12244, 2021

    Pith/arXiv arXiv 2021

  27. [27]

    Long short- term memory.Neural computation, 9(8):1735–1780, 1997

    Sepp Hochreiter and J ¨urgen Schmidhuber. Long short- term memory.Neural computation, 9(8):1735–1780, 1997

    1997

  28. [28]

    Learning to control pdes with differentiable physics.arXiv preprint arXiv:2001.07457, 2020

    Philipp Holl, Vladlen Koltun, and Nils Thuerey. Learning to control pdes with differentiable physics.arXiv preprint arXiv:2001.07457, 2020

    Pith/arXiv arXiv 2001

  29. [29]

    Particleformer: A 3d point cloud world model for multi-object, multi-material robotic manipulation.arXiv preprint arXiv:2506.23126, 2025

    Suning Huang, Qianzhong Chen, Xiaohan Zhang, Jiankai Sun, and Mac Schwager. Particleformer: A 3d point cloud world model for multi-object, multi-material robotic manipulation.arXiv preprint arXiv:2506.23126, 2025

    Pith/arXiv arXiv 2025

  30. [30]

    Pointworld: Scaling 3d world models for in-the-wild robotic manipulation.arXiv preprint arXiv:2601.03782, 2026

    Wenlong Huang, Yu-Wei Chao, Arsalan Mousavian, Ming-Yu Liu, Dieter Fox, Kaichun Mo, and Li Fei- Fei. Pointworld: Scaling 3d world models for in-the-wild robotic manipulation.arXiv preprint arXiv:2601.03782, 2026

    arXiv 2026

  31. [31]

    Plas- ticinelab: A soft-body manipulation benchmark with dif- ferentiable physics.arXiv preprint arXiv:2104.03311, 2021

    Zhiao Huang, Yuanming Hu, Tao Du, Siyuan Zhou, Hao Su, Joshua B Tenenbaum, and Chuang Gan. Plas- ticinelab: A soft-body manipulation benchmark with dif- ferentiable physics.arXiv preprint arXiv:2104.03311, 2021

    Pith/arXiv arXiv 2021

  32. [32]

    gradsim: Differentiable sim- ulation for system identification and visuomotor control

    Krishna Murthy Jatavallabhula, Miles Macklin, Florian Golemo, Vikram V oleti, Linda Petrini, Martin Weiss, Breandan Considine, J ´erˆome Parent-L ´evesque, Kevin Xie, Kenny Erleben, et al. gradsim: Differentiable sim- ulation for system identification and visuomotor control. arXiv preprint arXiv:2104.02646, 2021

    Pith/arXiv arXiv 2021

  33. [33]

    The affine particle-in-cell method.ACM Transactions on Graphics (TOG), 34(4): 1–10, 2015

    Chenfanfu Jiang, Craig Schroeder, Andrew Selle, Joseph Teran, and Alexey Stomakhin. The affine particle-in-cell method.ACM Transactions on Graphics (TOG), 34(4): 1–10, 2015

    2015

  34. [34]

    Phystwin: Physics- informed reconstruction and simulation of deformable objects from videos.arXiv preprint arXiv:2503.17973, 2025

    Hanxiao Jiang, Hao-Yu Hsu, Kaifeng Zhang, Hsin-Ni Yu, Shenlong Wang, and Yunzhu Li. Phystwin: Physics- informed reconstruction and simulation of deformable objects from videos.arXiv preprint arXiv:2503.17973, 2025

    Pith/arXiv arXiv 2025

  35. [35]

    Cotracker3: Simpler and better point tracking by pseudo- labelling real videos

    Nikita Karaev, Yuri Makarov, Jianyuan Wang, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Cotracker3: Simpler and better point tracking by pseudo- labelling real videos. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 6013–6022, 2025

    2025

  36. [36]

    Online friction coefficient identification for legged robots on slippery terrain using smoothed contact gradients.IEEE Robotics and Automa- tion Letters, 2025

    Hajun Kim, Dongyun Kang, Min-Gyu Kim, Gijeong Kim, and Hae-Won Park. Online friction coefficient identification for legged robots on slippery terrain using smoothed contact gradients.IEEE Robotics and Automa- tion Letters, 2025

    2025

  37. [37]

    Neural relational inference for interacting systems

    Thomas Kipf, Ethan Fetaya, Kuan-Chieh Wang, Max Welling, and Richard Zemel. Neural relational inference for interacting systems. InInternational conference on machine learning, pages 2688–2697. Pmlr, 2018

    2018

  38. [38]

    Contrastive learning of structured world models.arXiv preprint arXiv:1911.12247, 2019

    Thomas Kipf, Elise Van der Pol, and Max Welling. Contrastive learning of structured world models.arXiv preprint arXiv:1911.12247, 2019

    Pith/arXiv arXiv 1911

  39. [39]

    Learning particle dynamics for manipulating rigid bodies, deformable objects, and fluids.arXiv preprint arXiv:1810.01566, 2018

    Yunzhu Li, Jiajun Wu, Russ Tedrake, Joshua B Tenen- baum, and Antonio Torralba. Learning particle dynamics for manipulating rigid bodies, deformable objects, and fluids.arXiv preprint arXiv:1810.01566, 2018

    Pith/arXiv arXiv 2018

  40. [40]

    Propagation networks for model-based control under partial observa- tion

    Yunzhu Li, Jiajun Wu, Jun-Yan Zhu, Joshua B Tenen- baum, Antonio Torralba, and Russ Tedrake. Propagation networks for model-based control under partial observa- tion. In2019 International Conference on Robotics and Automation (ICRA), pages 1205–1211. IEEE, 2019

    2019

  41. [41]

    Differ- entiable cloth simulation for inverse problems.Advances in neural information processing systems, 32, 2019

    Junbang Liang, Ming Lin, and Vladlen Koltun. Differ- entiable cloth simulation for inverse problems.Advances in neural information processing systems, 32, 2019

    2019

  42. [42]

    End-to-end and highly-efficient differentiable simulation for robotics

    Quentin Le Lidec, Louis Montaut, Yann de Mont-Marin, Fabian Schramm, and Justin Carpentier. End-to-end and highly-efficient differentiable simulation for robotics. arXiv preprint arXiv:2409.07107, 2024

    Pith/arXiv arXiv 2024

  43. [43]

    Learning visible connectivity dynamics for cloth smoothing

    Xingyu Lin, Yufei Wang, Zixuan Huang, and David Held. Learning visible connectivity dynamics for cloth smoothing. InConference on Robot Learning, pages 256–266. PMLR, 2022

    2022

  44. [44]

    Soft- mac: Differentiable soft body simulation with forecast- based contact model and two-way coupling with articu- lated rigid bodies and clothes

    Min Liu, Gang Yang, Siyuan Luo, and Lin Shao. Soft- mac: Differentiable soft body simulation with forecast- based contact model and two-way coupling with articu- lated rigid bodies and clothes. In2024 IEEE/RSJ Inter- national Conference on Intelligent Robots and Systems (IROS), pages 12008–12015. IEEE, 2024

    2024

  45. [45]

    Fast simulation of mass-spring systems

    Tiantian Liu, Adam W Bargteil, James F O’Brien, and Ladislav Kavan. Fast simulation of mass-spring systems. ACM Transactions on Graphics (TOG), 32(6):1–7, 2013

    2013

  46. [46]

    Iden- tification of spring parameters for deformable object sim- ulation.IEEE transactions on visualization and computer graphics, 13(5):1081–1094, 2007

    Bryn Lloyd, G ´abor Sz´ekely, and Matthias Harders. Iden- tification of spring parameters for deformable object sim- ulation.IEEE transactions on visualization and computer graphics, 13(5):1081–1094, 2007

    2007

  47. [47]

    Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis

    Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis. In2024 International Conference on 3D Vision (3DV), pages 800–809. IEEE, 2024

    2024

  48. [48]

    Risp: Rendering-invariant state predictor with differentiable simulation and ren- dering for cross-domain parameter estimation.arXiv preprint arXiv:2205.05678, 2022

    Pingchuan Ma, Tao Du, Joshua B Tenenbaum, Wojciech Matusik, and Chuang Gan. Risp: Rendering-invariant state predictor with differentiable simulation and ren- dering for cross-domain parameter estimation.arXiv preprint arXiv:2205.05678, 2022

    Pith/arXiv arXiv 2022

  49. [49]

    Learning neural constitutive laws from motion observations for generalizable pde dynamics

    Pingchuan Ma, Peter Yichen Chen, Bolei Deng, Joshua B Tenenbaum, Tao Du, Chuang Gan, and Wojciech Ma- tusik. Learning neural constitutive laws from motion observations for generalizable pde dynamics. InInterna- tional Conference on Machine Learning, pages 23279– 23300. PMLR, 2023

    2023

  50. [50]

    Learning latent graph dynamics for visual manipulation of deformable objects

    Xiao Ma, David Hsu, and Wee Sun Lee. Learning latent graph dynamics for visual manipulation of deformable objects. In2022 International Conference on Robotics and Automation (ICRA), pages 8266–8273. IEEE, 2022

    2022

  51. [51]

    Unified particle physics for real-time applications.ACM Transactions on Graphics (TOG), 33 (4):1–12, 2014

    Miles Macklin, Matthias M ¨uller, Nuttapong Chentanez, and Tae-Yong Kim. Unified particle physics for real-time applications.ACM Transactions on Graphics (TOG), 33 (4):1–12, 2014

    2014

  52. [52]

    An introduction to the fundamentals of mesh generation in finite element analysis

    Dominique Madier. An introduction to the fundamentals of mesh generation in finite element analysis. Technical report, FEA Academy, Montreal, Canada, December

  53. [53]

    L Angela Mihai and Alain Goriely. How to character- ize a nonlinear elastic material? a review on nonlinear constitutive parameters in isotropic finite elasticity.Pro- ceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 473(2207):20170607, 2017

    2017

  54. [54]

    Impulse-based simulation of rigid bodies

    Brian Mirtich and John Canny. Impulse-based simulation of rigid bodies. InProceedings of the 1995 symposium on Interactive 3D graphics, pages 181–ff, 1995

    1995

  55. [55]

    Flexible neural representation for physics pre- diction.Advances in neural information processing systems, 31, 2018

    Damian Mrowca, Chengxu Zhuang, Elias Wang, Nick Haber, Li F Fei-Fei, Josh Tenenbaum, and Daniel L Yamins. Flexible neural representation for physics pre- diction.Advances in neural information processing systems, 31, 2018

    2018

  56. [56]

    Interactive virtual materials

    Matthias M ¨uller and Markus H Gross. Interactive virtual materials. InGraphics interface, volume 2004, pages 239–246, 2004

    2004

  57. [57]

    Position based dynamics.Journal of Visual Communication and Image Representation, 18(2): 109–118, 2007

    Matthias M ¨uller, Bruno Heidelberger, Marcus Hennix, and John Ratcliff. Position based dynamics.Journal of Visual Communication and Image Representation, 18(2): 109–118, 2007

    2007

  58. [58]

    gradsim: Differentiable simulation for system identification and visuomotor control

    J Krishna Murthy, Miles Macklin, Florian Golemo, Vikram V oleti, Linda Petrini, Martin Weiss, Breandan Considine, J ´erˆome Parent-L ´evesque, Kevin Xie, Kenny Erleben, et al. gradsim: Differentiable simulation for system identification and visuomotor control. InInter- national conference on learning representations, 2020

    2020

  59. [59]

    Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193, 2023

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fer- nandez, Daniel Haziza, Francisco Massa, Alaaeldin El- Nouby, et al. Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193, 2023

    Pith/arXiv arXiv 2023

  60. [60]

    Robotic manip- ulation by imitating generated videos without physical demonstrations.arXiv preprint arXiv:2507.00990, 2025

    Shivansh Patel, Shraddhaa Mohan, Hanlin Mai, Unnat Jain, Svetlana Lazebnik, and Yunzhu Li. Robotic manip- ulation by imitating generated videos without physical demonstrations.arXiv preprint arXiv:2507.00990, 2025

    Pith/arXiv arXiv 2025

  61. [61]

    Learning mesh-based simulation with graph networks

    Tobias Pfaff, Meire Fortunato, Alvaro Sanchez-Gonzalez, and Peter Battaglia. Learning mesh-based simulation with graph networks. InInternational conference on learning representations, 2020

    2020

  62. [62]

    Pointnet++: Deep hierarchical feature learning on point sets in a metric space.Advances in neural information processing systems, 30, 2017

    Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space.Advances in neural information processing systems, 30, 2017

    2017

  63. [63]

    Differentiable simulation of soft multi-body sys- tems.Advances in Neural Information Processing Sys- tems, 34:17123–17135, 2021

    Yiling Qiao, Junbang Liang, Vladlen Koltun, and Ming Lin. Differentiable simulation of soft multi-body sys- tems.Advances in Neural Information Processing Sys- tems, 34:17123–17135, 2021

    2021

  64. [64]

    Introducing nano banana pro

    Naina Raisinghani. Introducing nano banana pro. https://blog.google/innovation-and-ai/products/ nano-banana-pro/, November 2025. Google DeepMind Blog. Accessed: 2026-03-29

    2025

  65. [65]

    Grounded sam: Assembling open-world models for di- verse visual tasks, 2024

    Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kun- chang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, Zhaoyang Zeng, Hao Zhang, Feng Li, Jie Yang, Hongyang Li, Qing Jiang, and Lei Zhang. Grounded sam: Assembling open-world models for di- verse visual tasks, 2024

    2024

  66. [66]

    Differentiable implicit soft-body physics.arXiv preprint arXiv:2102.05791, 2021

    Junior Rojas, Eftychios Sifakis, and Ladislav Kavan. Differentiable implicit soft-body physics.arXiv preprint arXiv:2102.05791, 2021

    Pith/arXiv arXiv 2021

  67. [67]

    Learning to simulate complex physics with graph net- works

    Alvaro Sanchez-Gonzalez, Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, and Peter Battaglia. Learning to simulate complex physics with graph net- works. InInternational conference on machine learning, pages 8459–8468. PMLR, 2020

    2020

  68. [68]

    Transformer with implicit edges for particle-based physics simulation

    Yidi Shao, Chen Change Loy, and Bo Dai. Transformer with implicit edges for particle-based physics simulation. InEuropean conference on computer vision, pages 549–

  69. [69]

    Robocook: Long-horizon elasto-plastic object manipulation with diverse tools.arXiv preprint arXiv:2306.14447, 2023

    Haochen Shi, Huazhe Xu, Samuel Clarke, Yunzhu Li, and Jiajun Wu. Robocook: Long-horizon elasto-plastic object manipulation with diverse tools.arXiv preprint arXiv:2306.14447, 2023

    Pith/arXiv arXiv 2023

  70. [70]

    Robocraft: Learning to see, simulate, and shape elasto-plastic objects in 3d with graph networks

    Haochen Shi, Huazhe Xu, Zhiao Huang, Yunzhu Li, and Jiajun Wu. Robocraft: Learning to see, simulate, and shape elasto-plastic objects in 3d with graph networks. The International Journal of Robotics Research, 43(4): 533–549, 2024

    2024

  71. [71]

    Fem simulation of 3d deformable solids: a practitioner’s guide to theory, discretization and model reduction

    Eftychios Sifakis and Jernej Barbic. Fem simulation of 3d deformable solids: a practitioner’s guide to theory, discretization and model reduction. InAcm siggraph 2012 courses, pages 1–50. ACM, 2012

    2012

  72. [72]

    An implicit time-stepping scheme for rigid body dynamics with coulomb friction

    David Stewart and Jeffrey C Trinkle. An implicit time-stepping scheme for rigid body dynamics with coulomb friction. InProceedings 2000 ICRA. Mil- lennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No. 00CH37065), volume 1, pages 162–169. IEEE, 2000

    2000

  73. [73]

    A material point method for snow simulation.ACM Transactions on Graphics (TOG), 32(4):1–10, 2013

    Alexey Stomakhin, Craig Schroeder, Lawrence Chai, Joseph Teran, and Andrew Selle. A material point method for snow simulation.ACM Transactions on Graphics (TOG), 32(4):1–10, 2013

    2013

  74. [74]

    A hybrid model method for accurate surface deformation and incision based on fem and pbd.Scientific Programming, 2021(1):8343312, 2021

    Shijie Tan, Hongjun Zhou, and Jinjin Zheng. A hybrid model method for accurate surface deformation and incision based on fem and pbd.Scientific Programming, 2021(1):8343312, 2021

    2021

  75. [75]

    Solver-in-the-loop: Learning from differentiable physics to interact with iterative pde- solvers.Advances in neural information processing systems, 33:6111–6122, 2020

    Kiwon Um, Robert Brand, Yun Raymond Fei, Philipp Holl, and Nils Thuerey. Solver-in-the-loop: Learning from differentiable physics to interact with iterative pde- solvers.Advances in neural information processing systems, 33:6111–6122, 2020

    2020

  76. [76]

    Gener- alization of neural network models for complex network dynamics.Communications Physics, 7(1):348, 2024

    Vaiva Vasiliauskaite and Nino Antulov-Fantulin. Gener- alization of neural network models for complex network dynamics.Communications Physics, 7(1):348, 2024

    2024

  77. [77]

    Offline-online learning of deformation model for cable manipulation with graph neural networks.IEEE Robotics and Automation Letters, 7(2):5544–5551, 2022

    Changhao Wang, Yuyou Zhang, Xiang Zhang, Zheng Wu, Xinghao Zhu, Shiyu Jin, Te Tang, and Masayoshi Tomizuka. Offline-online learning of deformation model for cable manipulation with graph neural networks.IEEE Robotics and Automation Letters, 7(2):5544–5551, 2022

    2022

  78. [78]

    Graph neural network sensitivity under probabilistic er- ror model

    Xinjue Wang, Esa Ollila, and Sergiy A V orobyov. Graph neural network sensitivity under probabilistic er- ror model. In2022 30th European Signal Processing Conference (EUSIPCO), pages 2146–2150. IEEE, 2022

    2022

  79. [79]

    Causal dynamics learning for task-independent state abstraction.arXiv preprint arXiv:2206.13452, 2022

    Zizhao Wang, Xuesu Xiao, Zifan Xu, Yuke Zhu, and Pe- ter Stone. Causal dynamics learning for task-independent state abstraction.arXiv preprint arXiv:2206.13452, 2022

    Pith/arXiv arXiv 2022

  80. [80]

    Design and kinematic modeling of constant curvature continuum robots: A review.The International Journal of Robotics Research, 29(13):1661–1683, 2010

    Robert J Webster III and Bryan A Jones. Design and kinematic modeling of constant curvature continuum robots: A review.The International Journal of Robotics Research, 29(13):1661–1683, 2010

    2010

Showing first 80 references.