Pith. sign in

REVIEW 3 major objections 7 minor 70 references

Cloth-Splatting: 3D Cloth State Estimation from RGB Supervision

T0 review · 3 major / 7 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Cloth-Splatting shows that the 3D state of a cloth can be tracked accurately from a few RGB views by combining a learned dynamics model with a mesh-constrained 3D Gaussian Splatting measurement model, cutting convergence time by about 85…

desk verdict Promising RGB-only cloth tracker, but the headline accuracy claim doesn't match its own table and the main baseline is evaluated outside its design regime. read the letter →

arxiv 2501.01715 v1 pith:4V445JGA submitted 2025-01-03 cs.CV cs.RO

Alberta Longhini , Marcel Büsching , Bardienus P. Duisterhof , Jens Lundell , Jeffrey Ichnowski , M{aa}rten Björkman , Danica Kragic This is my paper · ORCID
classification cs.CVcs.RO
keywords 3DstateestimationGaussianSplattingclothtrackingdeformableobjectmanipulationRGBsupervisiongraphneuralnetworkdynamicsclosed-loop
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to establish that the 3D state of a cloth can be estimated from RGB images alone, without depth sensors, by unrolling a learned cloth dynamics model and refining its predictions with a differentiable renderer. It casts the problem as Bayesian filtering: an action-conditioned graph neural network proposes the next mesh state, and a mesh-constrained 3D Gaussian Splatting model renders that state into the observed camera views, so gradient descent on the photometric error can correct the proposal. The reported experiments on 75 synthetic scenes show a median trajectory error of 3.284 mm versus 3.635 mm for the best baseline, about 85 percent faster convergence, and real-robot demonstrations of closed-loop cloth folding. The reason to care is that accurate RGB-only state estimation is a bottleneck for robotic manipulation of deformable objects, where depth is often sparse, noisy, or unavailable.

What carries the argument

The load-bearing object is the mesh-constrained 3D Gaussian Splatting measurement model. Each Gaussian lives on a face of the cloth mesh: its mean is expressed in barycentric coordinates of the face's vertices and its rotation is expressed relative to the face, so when the mesh deforms the Gaussians follow the cloth while their colors and barycentric coordinates stay fixed. This makes the image render a differentiable function of the cloth state, providing the measurement likelihood for the update step. A small MLP outputs a per-timestep residual correction, and the photometric loss is combined with SSIM, isometric, and motion-magnitude regularizers to stabilize the refinement.

What would settle it

Run Cloth-Splatting on a cloth under a moving light source or cast shadow while recording ground-truth mesh state; if median trajectory error grows substantially or the refined mesh tracks the shadow edge rather than the cloth, the static-appearance assumption is violated. A simulator version with time-varying texture or lighting would settle it directly.

Watch

Extended reading notes

Core claim

The central claim is that the cloth mesh can act as the bridge between a learned dynamics prior and image observations. Each Gaussian is anchored to a mesh face with fixed barycentric coordinates and a face-relative rotation, so the rendered image is a differentiable function of the vertex positions; optimizing the photometric loss between the render and the observed RGB image updates the vertex positions through a learned residual. This gives a measurement model for a prediction-update loop that refines GNN predictions without retraining the dynamics network. In the paper's evaluation the refined estimates beat the prior mesh predictions and the dynamic-Gaussian baselines, and the refinement is what makes closed-loop model-predictive folding approach the accuracy of an oracle.

Load-bearing premise

The tracking assumes the cloth's visual appearance is static: Gaussian colors and the Gaussians' attachment to mesh faces are fixed during refinement, so if lighting changes or shadows move, the photometric loss can drag the mesh toward visual artifacts instead of the cloth's true motion.

Editorial extensions

If this is right

  • With only three or four calibrated RGB cameras, and no depth at run time, cloth state can be refined well enough to outperform dense-view dynamic Gaussian baselines in the paper's experiments.
  • Refining GNN rollouts with the photometric loss corrects error accumulation, so the tracker can run in a fast rollout mode rather than requiring iterative re-prediction at every step.
  • The same prediction-update architecture transfers to closed-loop manipulation: model-predictive control with Cloth-Splatting state estimates folds cloths to final states close to those achieved by an oracle with ground-truth states.
  • Faster convergence makes per-scene online adaptation practical, addressing a limitation of prior per-scene optimization approaches that need many camera views and long optimization times.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The static-appearance assumption suggests a concrete extension: periodically re-estimating Gaussian colors, or adding an explicit lighting model, would let the tracker survive moving shadows without abandoning the mesh constraint.
  • Because the residual state update is a per-scene MLP optimized from scratch, an amortized network that predicts residuals directly from images could trade a little accuracy for much faster deployment-time adaptation.
  • The monocular ablation shows a larger accuracy drop than going from four views to three, which points to view diversity rather than sheer camera count as the main driver; a carefully placed stereo pair may capture most of the benefit.
  • The paper's own note about occlusion-free initialization could be combined with template-based reconstruction of crumpled cloth to remove the initial-mesh restriction and broaden the method to heavily wrinkled cloths.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes Cloth-Splatting, a prediction-update framework for 3D cloth state estimation from RGB images. The method couples a GNN-based action-conditioned dynamics model (trained on simulated cloth data) with a mesh-constrained 3D Gaussian Splatting measurement model. At test time, a per-scene residual MLP is optimized by minimizing photometric and regularization losses between rendered and observed RGB images, thereby refining mesh-vertex positions. Experiments on 75 synthetic scenes and a real-world folding setup compare Cloth-Splatting against several baselines, reporting improved median trajectory error (MTE) and faster convergence, and the paper demonstrates a closed-loop manipulation use case.

Significance. If the quantitative claims hold, Cloth-Splatting is a useful step toward RGB-only 3D cloth tracking with sparse camera views. The mesh-constrained Gaussian Splatting formulation is a clean and potentially reusable contribution, and the combination of a learned dynamics prior with differentiable rendering is a sensible design. The paper promises code and videos, which would aid reproducibility. However, the support for the headline accuracy claim is weakened by an internal inconsistency in the reported improvement and by concerns about the fairness of the baseline evaluation protocol.

major comments (3)
  1. [Abstract and Section 1 vs. Table 1] The abstract and Section 1 state that Cloth-Splatting is "57 % more accurate" than the best-performing baseline, but Table 1 reports mean MTE of 3.284 mm for Cloth-Splatting versus 3.635 mm for MD-Splatting, which is a 9.66% improvement. Moreover, on the TSHIRT category Cloth-Splatting is worse than MD-Splatting (3.159 vs. 2.109 mm). The headline number is therefore not supported by the tabulated results, and the paper must either reconcile the 57% figure with Table 1 or remove it.
  2. [Section 2 and Section 5.1 / Table 1] Section 2 states that MD-Splatting and DeformGS "require dense observations (at least 50 cameras)", yet Table 1 evaluates MD-Splatting with only 4 camera views. This places the main baseline outside its design regime and makes the reported accuracy advantage of Cloth-Splatting difficult to interpret. Please either re-run the comparison with the baselines' required camera count, or explicitly frame the comparison as sparse-view tracking and analyze the effect of view count on all methods. In addition, Section 5.1 says that depth is used only for the baseline methods; please clarify which baselines receive depth and whether the comparison gives Cloth-Splatting an information disadvantage.
  3. [Table 1] The central accuracy claim rests on a small average MTE difference (3.284 vs. 3.635 mm) with large reported standard deviations (3.722 vs. 6.235 mm). The paper does not report per-scene paired comparisons or significance tests. Please provide such an analysis, or temper the "outperforms all baselines" claim accordingly.
minor comments (7)
  1. [Figure 4 caption] The caption contains the unexplained expression "t = 16.42 min (627.83%)" and the axis mixes minutes and seconds; please clarify what is being plotted and correct the units.
  2. [Table 1] The header "Survial rate" is a typo and should read "Survival rate".
  3. [Section 4.1 and Appendix A.1] The GNN input is described as the last m states of the mesh, but m is never defined; the appendix mentions the past k = 3 velocities, yet the relationship between m and k is unclear.
  4. [Algorithm 1] Line 6, "δ ˆMt+1 ∝ ∇Lobs", is not a valid update rule; please replace it with an explicit gradient descent step such as δ ← δ − λ∇δ Lobs.
  5. [Equation (4)] The covariance decomposition should be written as Σ = R S S^T R^T (or with explicit matrix products) to avoid ambiguity in the current notation "RSSTRT".
  6. [Section 5.4 and Appendix F] The oracle baseline is referred to as both OL-ORACLE and MPC-ORACLE; please use a single consistent name.
  7. [Table 3 and Table 6] The manipulation results appear twice as Table 3 in the main text and Table 6 in the appendix; please assign unique numbers and cite each consistently.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the GNN prior is trained independently of the test observations, the per-scene residual update is the intended Bayesian refinement, and accuracy is scored against external ground-truth meshes.

full rationale

Cloth-Splatting is an empirical method paper whose derivation chain is: (i) an action-conditioned GNN f_theta predicts a next-state prior from previous states and actions (Section 4.1); (ii) a mesh-constrained 3D Gaussian Splatting model h_GS provides a differentiable map from mesh state to image space (Section 4.2); (iii) a per-scene residual MLP u_psi is optimized against the observation loss L_obs = ||Y - h_GS(M_hat + delta)||^2 plus regularization (Section 4.3, Eq. 3, Eq. 8); (iv) accuracy is evaluated by comparing the refined mesh states to external ground-truth meshes from the simulator (Section 5.2, Table 1). The residual MLP is admittedly fitted per scene to the observations, but this is the intended Bayesian update step, not a disguised prediction: the quantity being evaluated is the resulting 3D state, which is compared to ground-truth meshes that are never seen by the optimizer. The GNN prior is trained on simulator data (Section 4.1 and Appendix A.1) on a subset of TOWEL trajectories and is not fitted to the test observations; the ablation in Table 2 shows that removing the GNN (A2) degrades performance substantially, so the prior is not redundant. The Gaussian appearance is optimized at t=0 and then frozen; Appendix A.2 explicitly anneals and freezes Gaussian attributes so that the residual dynamics model, rather than the appearance, absorbs temporal deformation, which is the opposite of a fitted parameter being renamed as a prediction. No uniqueness theorem, ansatz-by-citation, or load-bearing self-citation is used to force the method. The related-work claim that MD-Splatting and DeformGS require at least 50 cameras (Section 2) is a baseline-fairness concern, not a circular step, and the discrepancy between the abstract's '57% more accurate' and the Table 1 mean improvement of 9.66% is an internal-consistency issue, not circularity. Therefore, no circular step can be quoted and the paper merits a score of 0.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

No new physical entities are introduced. The 3D Gaussians on mesh faces are a representational device, not a novel particle or force. The central claim rests on learned parameters (GNN and residual MLP), a fixed-topology mesh assumption, a static appearance assumption, and an isometric deformation prior.

free parameters (6)
  • GNN dynamics parameters theta = Trained on simulator (NVIDIA Flex) towel data
    The transition model f_theta is learned from synthetic data; the central claim of accurate state prediction relies on this learned prior. It is a fitted model, not derived.
  • Residual MLP parameters psi = Per-scene optimized (7,000 to 8,000 iterations)
    The state update delta-M is produced by an MLP optimized on the target scene's RGB observations and regularization. This is the core refinement mechanism and is fitted to the test data, which is intended but must be recognized as a fitted component.
  • Gaussian appearance attributes (color, scale, rotation, opacity) = Optimized per scene
    Appearance parameters of the 3D Gaussians are optimized against the first frame and then frozen; these affect the rendering loss and thus the state update.
  • Regularization weights for Liso and Lmagn = Not specified numerically
    The relative weighting of the isometric and motion losses is not reported; the ablation shows these regularizers are crucial (MTE rises from 8.923 to 15.772 without them), so the central result depends on these unstated hyperparameters.
  • Number of Gaussians per face (2) and barycentric sampling distribution N(1/3, 0.05) = Fixed by design
    The density of Gaussians on the mesh and their initial placement affect the quality of the rendering gradient; these are chosen by hand without a sensitivity study.
  • Optimization schedules and learning rates = 1.5k and 6k iterations, annealing described qualitatively
    The training schedule for Gaussian attributes and residual MLP is described but exact learning rates and annealing curves are omitted.
assumptions (5)
  • domain assumption Grasped particle is rigidly attached to the gripper.
    Section 4.1: the GNN conditioning assumes the robot gripper rigidly moves a set of vertices, so the dynamics model propagates actions through the graph. This is a standard simplification for cloth manipulation.
  • domain assumption Mesh topology (edges) is fixed over time.
    Section 3: the edge set E_t is assumed constant, so the cloth is modeled as a deforming triangulated surface without topology changes (no tearing or large self-reconnections).
  • domain assumption Cloth visual appearance is static and time-invariant.
    Section 4.2: Gaussian colors and the barycentric mapping are fixed during tracking; changing lighting or shadows violate this, and the limitations section acknowledges this can cause tracking artifacts.
  • domain assumption Cloth deformation is approximately isometric (As-Rigid-As-Possible).
    Section 4.3 and Appendix A.4: the Liso loss enforces constant edge lengths between neighboring vertices; the method's stability and accuracy depend on this prior, and the ablation (A3) shows removing it degrades results.
  • domain assumption Initial mesh at t=0 is available from a point cloud and is occlusion-free.
    Section 5.1 and Limitations: the method requires a reconstructed initial mesh, typically from depth or multi-view stereo, with the cloth fully visible; the limitations acknowledge this.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Cloth-Splatting: 3D Cloth State Estimation from RGB Supervision." pith.science (2026). https://pith.science/paper/4V445JGA

@misc{pith2026250101715,
  author       = {Pith},
  title        = {Pith review of: Cloth-Splatting: 3D Cloth State Estimation from RGB Supervision},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4V445JGA}},
  note         = {Machine review of arXiv:2501.01715}
}
read the original abstract

We introduce Cloth-Splatting, a method for estimating 3D states of cloth from RGB images through a prediction-update framework. Cloth-Splatting leverages an action-conditioned dynamics model for predicting future states and uses 3D Gaussian Splatting to update the predicted states. Our key insight is that coupling a 3D mesh-based representation with Gaussian Splatting allows us to define a differentiable map between the cloth state space and the image space. This enables the use of gradient-based optimization techniques to refine inaccurate state estimates using only RGB supervision. Our experiments demonstrate that Cloth-Splatting not only improves state estimation accuracy over current baselines but also reduces convergence time.

Figures

Figures reproduced from arXiv: 2501.01715 by the authors.

Figure 1
Figure 1. Cloth-Splatting state estimation of a real-world cloth. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of Cloth-Splatting. We estimate the 3D state of the cloth through a prediction-update framework. We use a GNN to approximate the transition function that estimates the next state Mˆ t+1 given an action at and the state Mt . We then employ our proposed mesh-constrained GS as a measurement model hGS to obtain an estimate of the observation Y˜ t+1 given the current refined state M˜ t+1 = Mˆ t+1 + δMˆ t+1. The … view at source ↗
Figure 4
Figure 4. Time ablation. Convergence times of Cloth-Splatting and MD-Splatting on a SHORTS scene. The offset to the y-axis arises from the GNN predictions (1.36 s) and the 3D Gaussian initialization (26.55 s). 5.4 Robotic Manipulation Use Case [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figures from the paper (5 more)
Figure 5
Figure 5. Figure 5: Example of synthetic images generated for the objects considered in our experiments (towel, shorts, t-shirt). For each object, we show the flat (top row) and the deformed (bottom row) states, rendered with Blender. C Real-world Set-up and Data Collection The real-world…
Figure 6
Figure 6. Figure 6: Overview of experimental set-up. the GNN input with the refined states. ROLLOUT predicts and updates the full 16 timesteps of the scene jointly. Training parameters are the same for both methods, with the difference that we up￾date the predictions ITERATIVE for 1000 it…
Figure 7
Figure 7. Figure 7: Qualitative results of the half folding for different manipulation strategies. START GOAL (a) FIXED (b) MPC-OL (c) MPC-CS (d) MPC-ORACLE [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]
Figure 8
Figure 8. Figure 8: Qualitative results of the half folding for different manipulation strategies. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]
Figure 9
Figure 9. Figure 9: Qualitative results of the manipulation outcomes for the methods (a) FIXED, (b) MPC-OL, and (c) MPC-CS are presented. Each method is illustrated at four distinct time points during execution, with the final fold shown at Time 4. Best viewed with zoom. 21 [PITH_FULL_IM…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

70 extracted references · 61 canonical work pages

  1. [1]

    Longhini, Y

    A. Longhini, Y . Wang, I. Garcia-Camacho, D. Blanco-Mulero, M. Moletta, M. Welle, G. Aleny`a, H. Yin, Z. Erickson, D. Held, et al. Unfolding the literature: A review of robotic cloth manipulation. arXiv preprint arXiv:2407.01361, 2024

    arXiv 2024

  2. [2]

    J. Zhu, A. Cherubini, C. Dune, D. Navarro-Alarcon, F. Alambeigi, D. Berenson, F. Ficuciello, K. Harada, J. Kober, X. Li, et al. Challenges and outlook in robotic manipulation of deformable objects. Robotics & Automation Magazine, 29(3):67–77, 2022

    work page 2022

  3. [3]

    Avigal, L

    Y . Avigal, L. Berscheid, T. Asfour, T. Kr ¨oger, and K. Goldberg. Speedfolding: Learning efficient bimanual folding of garments. In IEEE/RSJ IROS, pages 1–8, 2022

    work page 2022

  4. [4]

    Salhotra, I.-C

    G. Salhotra, I.-C. A. Liu, M. Dominguez-Kuhne, and G. S. Sukhatme. Learning deformable object manipulation from expert demonstrations. RA-L, 7(4):8775–8782, 2022

    work page 2022

  5. [5]

    Z. Xu, C. Chi, B. Burchfiel, E. Cousineau, S. Feng, and S. Song. Dextairity: Deformable manipulation can be a breeze. RSS, 2022

    work page 2022

  6. [6]

    AdaFold: Adapting Folding Trajectories of Cloths via Feedback-loop Manipulation

    A. Longhini, M. C. Welle, Z. Erickson, and D. Kragic. AdaFold: Adapting folding trajectories of cloths via feedback-loop manipulation. arXiv preprint arXiv:2403.06210, 2024

    work page Pith review arXiv 2024

  7. [7]

    X. Lin, Y . Wang, Z. Huang, and D. Held. Learning visible connectivity dynamics for cloth smoothing. In CoRL, pages 256–266. PMLR, 2022

    work page 2022

  8. [8]

    Huang, X

    Z. Huang, X. Lin, and D. Held. Mesh-based dynamics with occlusion reasoning for cloth manipulation. RSS, 2022

    work page 2022

Show all 70 references
  1. [9]

    Sanchez-Gonzalez, J

    A. Sanchez-Gonzalez, J. Godwin, T. Pfaff, R. Ying, J. Leskovec, and P. Battaglia. Learning to simulate complex physics with graph networks. In ICML, pages 8459–8468. PMLR, 2020

    2020

  2. [10]

    Longhini, M

    A. Longhini, M. Moletta, A. Reichlin, M. C. Welle, D. Held, Z. Erickson, and D. Kragic. Edo- net: Learning elastic properties of deformable objects from graph dynamics. In IEEE ICRA, pages 3875–3881, 2023

    2023

  3. [11]

    Sundaresan, R

    P. Sundaresan, R. Antonova, and J. Bohgl. Diffcloud: Real-to-sim from point clouds with differentiable simulation and rendering of deformable objects. In 2022 IEEE/RSJ IROS, pages 10828–10835. IEEE, 2022

    2022

  4. [12]

    X. Ma, D. Hsu, and W. S. Lee. Learning latent graph dynamics for visual manipulation of deformable objects. In IEEE ICRA, pages 8266–8273, 2022

    2022

  5. [13]

    Huang, X

    Z. Huang, X. Lin, and D. Held. Self-supervised cloth reconstruction via action-conditioned cloth tracking. In IEEE ICRA, pages 7111–7118, 2023

    2023

  6. [14]

    Kerbl, G

    B. Kerbl, G. Kopanas, T. Leimk ¨uhler, and G. Drettakis. 3D Gaussian Splatting for real-time radiance field rendering. ACM ToG, 42(4), July 2023

    2023

  7. [15]

    Seita, A

    D. Seita, A. Ganapathi, R. Hoque, M. Hwang, E. Cen, A. K. Tanwani, A. Balakrishna, B. Thananjeyan, J. Ichnowski, N. Jamali, et al. Deep imitation learning of sequential fabric smoothing from an algorithmic supervisor. In IEEE/RSJ IROS, pages 9651–9658, 2020. 9

    2020

  8. [16]

    Y . Wu, W. Yan, T. Kurutach, L. Pinto, and P. Abbeel. Learning to manipulate deformable objects without demonstrations. RSS, 2020

    2020

  9. [17]

    Y . Wang, Z. Sun, Z. Erickson, and D. Held. Learning to dress people with diverse poses and garments. RSS, 2023

    2023

  10. [18]

    Matas, S

    J. Matas, S. James, and A. J. Davison. Sim-to-real reinforcement learning for deformable object manipulation. In CoRL, pages 734–743. PMLR, 2018

    2018

  11. [19]

    K. Mo, Y . Deng, C. Xia, and X. Wang. Learning language-conditioned deformable object manipulation with graph dynamics. arXiv preprint arXiv:2303.01310, 2023

    2023 arXiv

  12. [20]

    Hoque, D

    R. Hoque, D. Seita, A. Balakrishna, A. Ganapathi, A. K. Tanwani, N. Jamali, K. Yamane, S. Iba, and K. Goldberg. Visuospatial foresight for physical sequential fabric manipulation. Autonomous Robots, pages 1–25, 2022

    2022

  13. [21]

    Y . Li, Y . Yue, D. Xu, E. Grinspun, and P. K. Allen. Folding deformable objects using predictive simulation and trajectory optimization. In IEEE/RSJ IROS, pages 6000–6006, 2015

    2015

  14. [22]

    Garcia-Camacho, A

    I. Garcia-Camacho, A. Longhini, M. Welle, G. Alenya, D. Kragic, and J. Borras. Standard- ization of cloth objects and its relevance in robotic manipulation. In IEEE ICRA, pages 8298–

  15. [23]

    Petr´ık and V

    V . Petr´ık and V . Kyrki. Feedback-based fabric strip folding. In IEEE/RSJ IROS, pages 773– 778, 2019

    2019

  16. [24]

    Hietala, D

    J. Hietala, D. Blanco-Mulero, G. Alcan, and V . Kyrki. Learning visual feedback control for dynamic cloth folding. In IEEE/RSJ IROS, pages 1455–1462, 2022

    2022

  17. [25]

    Saito, J

    S. Saito, J. Yang, Q. Ma, and M. J. Black. Scanimate: Weakly supervised learning of skinned clothed avatar networks. In IEEE/CVF CVPR, pages 2886–2897, 2021

    2021

  18. [26]

    Z. Su, T. Yu, Y . Wang, and Y . Liu. Deepcloth: Neural garment representation for shape and style editing. IEEE TPAMI, 45(2):1581–1593, 2022

    2022

  19. [27]

    Dan ˇeˇrek, E

    R. Dan ˇeˇrek, E. Dibra, C. ¨Oztireli, R. Ziegler, and M. Gross. Deepgarment: 3d garment shape estimation from a single image. In Computer Graphics F orum, volume 36, pages 269–280. Wiley Online Library, 2017

    2017

  20. [28]

    Jiang, J

    B. Jiang, J. Zhang, Y . Hong, J. Luo, L. Liu, and H. Bao. BCNet: Learning body and cloth shape from a single image. In ECCV, pages 18–35, 2020

    2020

  21. [29]

    Chi and S

    C. Chi and S. Song. Garmentnets: Category-level pose estimation for garments via canonical space shape completion. In IEEE/CVF ICCV, pages 3324–3333, 2021

    2021

  22. [30]

    Y . Li, Y . Wang, M. Case, S.-F. Chang, and P. K. Allen. Real-time pose estimation of deformable objects using a volumetric approach. In IEEE/RSJ IROS, pages 1046–1052. IEEE, 2014

    2014

  23. [31]

    Karaev, I

    N. Karaev, I. Rocco, B. Graham, N. Neverova, A. Vedaldi, and C. Rupprecht. CoTracker: It is better to track together. arXiv preprint arXiv:2307.07635, 2023

    2023 arXiv

  24. [32]

    X. Shi, Z. Huang, W. Bian, D. Li, M. Zhang, K. C. Cheung, S. See, H. Qin, J. Dai, and H. Li. VideoFlow: Exploiting temporal cues for multi-frame optical flow estimation. In IEEE/CVF ICCV, pages 12435–12446, 2023

    2023

  25. [33]

    Doersch, A

    C. Doersch, A. Gupta, L. Markeeva, A. Recasens, L. Smaira, Y . Aytar, J. Carreira, A. Zis- serman, and Y . Yang. Tap-vid: A benchmark for tracking any point in a video. Advances in Neural Information Processing Systems, 35:13610–13626, 2022

    2022

  26. [34]

    Wang, Y .-Y

    Q. Wang, Y .-Y . Chang, R. Cai, Z. Li, B. Hariharan, A. Holynski, and N. Snavely. Tracking everything everywhere all at once. In IEEE/CVF ICCV, pages 19738–19749, 2023. 10

    2023

  27. [35]

    Teed and J

    Z. Teed and J. Deng. Raft: Recurrent all-pairs field transforms for optical flow. In ECCV, pages 402–419. Springer, 2020

    2020

  28. [36]

    Teed and J

    Z. Teed and J. Deng. RAFT-3D: Scene flow using rigid-motion embeddings. In IEEE/CVF CVPR, pages 8375–8384, 2021

    2021

  29. [37]

    W. Yuan, Z. Lv, T. Schmidt, and S. Lovegrove. STaR: Self-supervised tracking and reconstruc- tion of rigid objects in motion with neural rendering. InIEEE/CVF CVPR, pages 13144–13152, 2021

    2021

  30. [38]

    In ECCV, pages 405– 421, 2020

    NeRF: Representing scenes as neural radiance fields for view synthesis. In ECCV, pages 405– 421, 2020

    2020

  31. [39]

    Pumarola, E

    A. Pumarola, E. Corona, G. Pons-Moll, and F. Moreno-Noguer. D-NeRF: Neural Radiance Fields for Dynamic Scenes. In IEEE/CVF CVPR, pages 10313–10322, 2021

    2021

  32. [40]

    K. Park, U. Sinha, J. T. Barron, S. Bouaziz, D. B. Goldman, S. M. Seitz, and R. Martin-Brualla. Nerfies: Deformable Neural Radiance Fields. In IEEE/CVF ICCV, pages 5845–5854, 2021

    2021

  33. [41]

    Z. Li, S. Niklaus, N. Snavely, and O. Wang. Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes. In IEEE/CVF CVPR, pages 6494–6504, 2021

    2021

  34. [42]

    Z. Li, Q. Wang, F. Cole, R. Tucker, and N. Snavely. DynIBaR: Neural Dynamic Image-Based Rendering. In IEEE/CVF CVPR, pages 4273–4284, 2023

    2023

  35. [43]

    H. Gao, R. Li, S. Tulsiani, B. Russell, and A. Kanazawa. Monocular Dynamic View Synthesis: A Reality Check. In NeurIPS, pages 33768–33780, 2022

    2022

  36. [44]

    Luiten, G

    J. Luiten, G. Kopanas, B. Leibe, and D. Ramanan. Dynamic 3D Gaussians: Tracking by persistent dynamic view synthesis. In International Conference on 3D Vision (3DV) , pages 800–809, 2024

    2024

  37. [45]

    G. Wu, T. Yi, J. Fang, L. Xie, X. Zhang, W. Wei, W. Liu, Q. Tian, and X. Wang. 4D Gaussian Splatting for real-time dynamic scene rendering. In IEEE/CVF CVPR, pages 20310–20320, 2024

    2024

  38. [46]

    B. P. Duisterhof, Z. Mandi, Y . Yao, J.-W. Liu, M. Z. Shou, S. Song, and J. Ichnowski. MD- Splatting: Learning metric deformation from 4D gaussians in highly deformable scenes. arXiv preprint arXiv:2312.00583, 2023

    2023 arXiv

  39. [47]

    B. P. Duisterhof, Z. Mandi, Y . Yao, J.-W. Liu, J. Seidenschwarz, M. Z. Shou, R. Deva, S. Song, S. Birchfield, B. Wen, and J. Ichnowski. DeformGS: Scene flow in highly deformable scenes for deformable object manipulation. WAFR, 2024

    2024

  40. [48]

    Waczy ´nska, P

    J. Waczy ´nska, P. Borycki, S. Tadeja, J. Tabor, and P. Spurek. GaMeS: Mesh-based adapting and modification of gaussian splatting. arXiv preprint arXiv:2402.01459, 2024

    2024 arXiv

  41. [49]

    L. Gao, J. Yang, B.-T. Zhang, J.-M. Sun, Y .-J. Yuan, H. Fu, and Y .-K. Lai. Mesh-based gaussian splatting for real-time large-scale deformation. arXiv preprint arXiv:2402.04796, 2024

    2024 arXiv

  42. [50]

    Jiang, C

    Y . Jiang, C. Yu, T. Xie, X. Li, Y . Feng, H. Wang, M. Li, H. Lau, F. Gao, Y . Yang, and C. Jiang. VR-GS: A physical dynamics-aware interactive gaussian splatting system in virtual reality. In ACM SIGGRAPH, number 23, 2024

    2024

  43. [51]

    T. Xie, Z. Zong, Y . Qiu, X. Li, Y . Feng, Y . Yang, and C. Jiang. PhysGaussian: Physics- integrated 3d gaussians for generative dynamics. InIEEE/CVF CVPR, pages 4389–4398, 2024

    2024

  44. [52]

    Y . Wang, D. Held, and Z. Erickson. Visual haptic reasoning: Estimating contact forces by observing deformable object interactions. RA-L, 7(4):11426–11433, 2022. 11

    2022

  45. [53]

    Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. Image quality assessment: from error visibility to structural similarity. TIP, 2004

    2004

  46. [54]

    Sorkine and M

    O. Sorkine and M. Alexa. As-rigid-as-possible surface modeling. In Symposium on Geometry processing, volume 4, pages 109–116. Citeseer, 2007

    2007

  47. [55]

    Lips, V .-L

    T. Lips, V .-L. De Gusseme, et al. Learning keypoints for robotic cloth manipulation using synthetic data. arXiv preprint arXiv:2401.01734, 2024

    2024 arXiv

  48. [56]

    Macklin, M

    M. Macklin, M. M ¨uller, N. Chentanez, and T.-Y . Kim. Unified particle physics for real-time applications. ACM Transactions on Graphics (TOG), 33(4):1–12, 2014

    2014

  49. [57]

    X. Lin, Y . Wang, J. Olkin, and D. Held. Softgym: Benchmarking deep reinforcement learning for deformable object manipulation. In CoRL, pages 432–448. PMLR, 2021

    2021

  50. [58]

    B. O. Community. Blender - a 3D modelling and rendering package . Blender Foundation, Stichting Blender Foundation, Amsterdam, 2018. URL http://www.blender.org

    2018

  51. [59]

    Delaunay et al

    B. Delaunay et al. Sur la sphere vide. Izv. Akad. Nauk SSSR, Otdelenie Matematicheskii i Estestvennyka Nauk, 7(793-800):1–2, 1934

    1934

  52. [60]

    S. Wang, V . Leroy, Y . Cabon, B. Chidlovskii, and J. Revaud. Dust3r: Geometric 3d vision made easy. arXiv preprint arXiv:2312.14132, 2023

    2023 arXiv

  53. [61]

    Zheng, A

    Y . Zheng, A. W. Harley, B. Shen, G. Wetzstein, and L. J. Guibas. PointOdyssey: A large-scale synthetic dataset for long-term point tracking. In CVPR, pages 19855–19865, 2023

    2023

  54. [62]

    Teed and J

    Z. Teed and J. Deng. RAFT: Recurrent All-Pairs Field Transforms for Optical Flow. In ECCV, pages 402–419, 2020

    2020

  55. [63]

    Garcia-Camacho, M

    I. Garcia-Camacho, M. Lippi, M. C. Welle, H. Yin, R. Antonova, A. Varava, J. Borras, C. Tor- ras, A. Marino, G. Alenya, et al. Benchmarking bimanual cloth manipulation. RA-L, 5(2): 1111–1118, 2020

    2020

  56. [64]

    W. Wang, G. Li, M. Zamora, and S. Coros. Trtm: Template-based reconstruction and target- oriented manipulation of crumpled cloths. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 12522–12528. IEEE, 2024

    2024

  57. [65]

    D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. In ICLR, 2015

    2015

  58. [66]

    Br ´egier

    R. Br ´egier. Deep regression on manifolds: A 3D rotation case study. In International Confer- ence on 3D Vision (3DV), pages 166–174, 2021

    2021

  59. [67]

    Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing , 13(4):600– 612, 2004

    2004

  60. [68]

    S. Liu, Z. Zeng, T. Ren, F. Li, H. Zhang, J. Yang, C. Li, J. Yang, H. Su, J. Zhu, et al. Ground- ing DINO: Marrying DINO with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499, 2023

    2023 arXiv

  61. [69]

    Kirillov, E

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo, et al. Segment anything. InIEEE/CVF CVPR, pages 3992–4003, 2023

    2023

  62. [70]

    cloth” and “robot gripper

    H. K. Cheng and A. G. Schwing. XMem: Long-term video object segmentation with an atkinson-shiffrin memory model. In ECCV, pages 640–658. Springer, 2022. 12 A Cloth-Splatting Implementation A.1 Action-Conditioned Dynamics Architecture and Training The action-conditioned dynamic...

    2022

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.