Pith. sign in

REVIEW 3 major objections 4 minor 84 references

SAFT: Shape and Appearance of Fabrics from Template via Differentiable Physical Simulations from Monocular Video

T0 review · 3 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Two regularization terms allow a differentiable cloth simulation to reconstruct 3D fabric shape and appearance from a single monocular RGB video.

desk verdict Two regularizers that genuinely fix monocular cloth-SfT depth ambiguity, but the headline 2.64x gain over phi-SfT is measured against a baseline running without its standard final rigid alignment. read the letter →

arxiv 2509.08828 v1 pith:6HXSF6TN submitted 2025-09-10 cs.CV

David Stotko , Reinhard Klein This is my paper
classification cs.CV
keywords shape-from-templateclothsimulationdifferentiablephysicsmonocularvideo3DreconstructionSVBRDFestimationdepthambiguityrendering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper aims to show that a single monocular RGB video is enough to reconstruct both the 3D motion of a piece of fabric and its detailed appearance, provided the geometry is driven by a differentiable physical simulation. The key to the method is two regularization terms: one that uses the cloth's own stretching, bending, and shearing energies to stop the optimizer from crumpling the mesh, and one that suppresses external forces along the camera's viewing direction, where monocular gradients carry no depth information. If the method is right, depth ambiguity in cloth reconstruction is not a dead end — carefully chosen priors can substitute for multi-view data. The paper reports a 2.64-fold reduction in mean Chamfer distance against a physics-based baseline and a 2.92-fold reduction against a neural-surrogate baseline, at roughly 30 minutes per scene, and shows that the recovered motion is accurate enough to estimate spatially varying reflectance and environment lighting from the same video.

What carries the argument

The central mechanism is a differentiable mass-spring cloth simulation with two new regularizers. The energy regularizer sums the cloth's stretching, bending, and shearing energies while stopping gradients with respect to stiffness, preventing crumpling without collapsing the simulation's material parameters. The force regularizer penalizes external forces projected along the camera-to-vertex viewing direction, biasing motion along the depth axis that image gradients alone cannot constrain. Around this sit projective texture mapping, a differentiable rasterizer for geometry optimization, and a differentiable ray tracer for appearance estimation. Ablations show these regularizers are responsi

What would settle it

Run the phi-SfT baseline on the same nine scenes using its standard final rigid alignment and recompute the mean Chamfer distance; if the baseline's error drops substantially below the reported 8.80, the 2.64x claim does not survive the baseline's own protocol. A second check: compare depth maps on a scene with known ground truth after using identical alignment for all methods — the paper's own depth-error metric would then be the adjudicator.

Watch

Extended reading notes

Core claim

The central claim is that the depth-ambiguity problem in monocular shape-from-template for cloth can be resolved by two regularization terms attached to a differentiable mass-spring simulation. The first term penalizes the internal deformation energy of the cloth while stopping gradients with respect to stiffness, so the optimizer cannot 'cheat' by lowering stiffness and crumpling the fabric to fit the image. The second term penalizes the component of each external force that points along the camera-to-vertex viewing direction, directly counteracting the fact that gradients of the image loss with respect to vertex positions are exactly orthogonal to that direction. With these terms, the opti

Load-bearing premise

The headline 2.64x/2.92x error reduction assumes a fair comparison protocol: the authors do not apply the final rigid alignment that the phi-SfT baseline uses in its published evaluation, and if that protocol is restored the reported improvement factor could shrink.

Editorial extensions

If this is right

  • A single monocular RGB video, a template mesh, and per-frame masks are sufficient to reconstruct cloth geometry with sub-centimeter average depth error on the tested scenes.
  • The same video can then be used to estimate SVBRDF textures and an environment map; the recovered diffuse details are sharper than when the appearance pipeline is run on a less accurate geometry.
  • The geometry optimization runs in about 30 minutes per scene, roughly 36 times faster than the physics-based baseline it outperforms.
  • Ablations show the energy regularizer prevents catastrophic crumpling in scenes with wind along the viewing direction, and the force regularizer matters most for exactly those depth-ambiguous motions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The energy regularizer is a general principle: for any deformable object whose internal energy can be differentiated, the same stop-gradient energy penalty should discourage the optimizer from 'cheating' its physics model, so the approach likely transfers to non-fabric soft bodies.
  • The force regularizer's design reveals a testable prediction: scenes in which the dominant motion is perpendicular to the camera axis should show little benefit from the regularizer, while along-axis motion should show the largest improvements; a controlled synthetic experiment varying wind direction could confirm this.
  • Since the paper omits normal maps due to single-view ambiguity, a natural extension is jointly alternating geometry and appearance optimization, which the paper acknowledges as a limitation; if that joint loop converges, it may push monocular material estimation further.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes a three-stage monocular shape-from-template pipeline for dynamic cloth. It first projectively maps a texture onto a template mesh, then optimizes the parameters of a mass-spring cloth simulation plus per-vertex external forces to match RGB and mask observations, introducing two novel regularizers (internal-energy and force-direction) intended to reduce depth ambiguity. In the third stage it optimizes SVBRDF textures and an environment map via differentiable ray tracing. Evaluated on the nine real scenes of the phi-SfT dataset, it reports a mean L2 Chamfer distance of 3.33e-4 m^2 versus 8.80e-4 for phi-SfT and 9.71e-4 for PG-SfT, i.e. a 2.64x/2.92x improvement, in about 30 minutes per scene; ablations show that the regularizers prevent crumpling. Appearance estimation is demonstrated qualitatively on real scenes and on five synthetic scenes with known ground-truth materials.

Significance. The contribution is potentially significant: it couples differentiable physics-based simulation with differentiable rendering for monocular template-based reconstruction, and it goes beyond geometry by showing that the recovered motion is accurate enough to support SVBRDF and environment-light estimation from a single RGB video. The two regularizers are simple, well-motivated by the gradient-orthogonality analysis in the supplementary material, and the ablation study indicates that they prevent pathological crumpling. The source code is released and evaluation uses a public dataset with multiple metrics. However, the headline improvement is weakened by a modified baseline protocol for phi-SfT, and the appearance-estimation claim currently rests on qualitative evidence only.

major comments (3)
  1. [Section 4.1.2, Table 1] The headline 2.64x factor in the Abstract and Section 4.1.2 is computed from Table 1, whose note states that the results for phi-SfT 'differ from the values reported in their paper because we do not perform any rigid alignment at the end [27].' This changes the evaluation protocol for the main baseline. If phi-SfT's published pipeline includes a final rigid alignment to compensate global drift, removing it can only increase its Chamfer error, so the reported factor may overstate the improvement due to the proposed method. The paper does not report phi-SfT with its standard alignment, nor any analysis of how much of the 2.64x improvement is attributable to the protocol change. Please rerun the comparison under phi-SfT's published protocol, or justify in detail why rigid alignment is inappropriate here, and report the effect on the supplementary depth and p2s metrics as well.
  2. [Section 4.1.3, Table 3; Section 3.3.2, Eqs. (7)-(9)] The regularization weights lambda_E=2 and lambda_F=2e-4, as well as other hyperparameters, are stated as fixed values, but the ablation study is performed on the same nine scenes used for the final evaluation. If these weights were selected by observing the same test scenes, the reported mean Chamfer improvements and the ablation differences are optimistically biased. A sensitivity analysis over lambda_E and lambda_F, or a leave-one-scene-out selection procedure, is needed to show that the central gain is not an artifact of per-dataset tuning. At minimum, the manuscript should state explicitly how the weights were chosen.
  3. [Section 4.2.2, Figure 7 and Table 7] Appearance estimation is a central contribution, and the synthetic scenes provide ground-truth SVBRDFs and environment maps, but the evaluation is only qualitative. The comparison between estimates from ground-truth motion and from the reconstructed motion in Figure 7 is informative, yet no quantitative errors (e.g., RMSE, SSIM, or perceptual distance on diffuse, roughness, metallic, and environment-map) are reported. Without such metrics, the claim that the reconstructed geometry is 'sufficient quality' to enable sharp appearance recovery is not fully supported. Please add numeric comparisons for the synthetic scenes and, ideally, an analysis of how appearance quality degrades with geometry error.
minor comments (4)
  1. [Abstract and Section 4.1.2] The abstract states 'reduced the error in the 3D reconstruction by a factor of 2.64' without specifying that this is the L2 Chamfer distance. The L1 Chamfer distance in Table 6 shows a smaller mean improvement of about 1.64x over phi-SfT. Please qualify the headline claim to avoid overgeneralization.
  2. [Section 3.3.2, Eq. (9)] The text says the force regularization 'penalizes external forces along the image direction,' but the formula subtracts the projection onto the viewing direction, i.e. it penalizes the component perpendicular to the viewing direction. Please rephrase to avoid confusion.
  3. [Supplementary, Eq. (28)] The depth metric is computed only over pixels where both ground truth and reconstruction have valid depth. This may exclude systematic errors at the fabric boundary. Please report the coverage fraction or use a full-mask metric to ensure the depth comparison is not biased.
  4. [Section 4.1.1] The qualitative comparison states that 'both phi-SfT and our method yield comparable visual quality,' which is in tension with the large quantitative differences in Table 1. A brief comment on why the visual similarity is not reflected in the metrics would help the reader.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular reduction found; geometry is optimized from RGB/mask losses and evaluated on external depth ground truth, though one comparison-protocol caveat is noted.

full rationale

The central reconstruction pipeline is self-contained and not circular: the shape optimization (Section 3.3.2) minimizes a loss composed of image, silhouette, and two regularization terms, and the ground-truth depth/point clouds are used only for evaluation, not during optimization (Section 4 states 'only the RGB images are used for the reconstruction task'). The two novel regularizers RE and RF are defined from internal cloth energy and external forces, not from the target depth or Chamfer metric, and the supplementary gradient-orthogonality derivation (Eq. 26) independently motivates why such regularization addresses depth ambiguity; none of the equations reduces to the evaluation metric. Hyperparameters λE and λF are reported as set values rather than as fitted predictions, and there is no quoted statement that they were tuned to the reported Chamfer numbers. Self-citations to prior work by the same group ([62] PG-SfT, [29,30] decoder) are used as baselines or implementation components, not as load-bearing uniqueness theorems or ansatz smuggled via citation. The paper itself discloses a comparison-protocol caveat in Section 4.1.2 / Table 1: 'Note that the results for phi-SfT differ from the values reported in their paper because we do not perform any rigid alignment at the end [27].' This is a benchmark-fairness concern that could affect the size of the reported 2.64x improvement, but it is not a circular derivation: the method's output is not defined in terms of the baseline's protocol. A secondary self-referential element exists in Section 4.2.2, where synthetic scenes are rendered using the authors' own reconstructed motion as ground truth ('we reuse the reconstructed motion of our algorithm and save renderings with novel appearance parameters'); this makes the synthetic-scene geometry metrics a self-consistency check rather than an independent external test, but the paper's main quantitative claims rest on the real phi-SfT dataset with external depth ground truth. Overall, no prediction reduces to its input by construction, so the circularity score is 0.

Assumptions & free parameters 9 free parameters · 6 assumptions · 0 invented entities

The central reconstruction rests on a standard physics surrogate with many fitted parameters (stiffnesses and per-vertex forces), plus two hand-designed regularizers. The main auxiliary assumptions are the sufficiency of the mass-spring model and the known-template/camera setup. No new physical entities are postulated.

free parameters (9)
  • Stretching stiffness Y = init log10(200), bounds 10..1000
    Optimized per scene; scales resistance to edge-length changes.
  • Bending stiffness B = init 1e-3, bounds 1e-4..1e-2
    Optimized per scene; controls wrinkle size and fold sharpness.
  • Shearing stiffness S = init 1e-4, bounds 1e-5..1e-2
    Optimized per scene; controls angular resistance in the mesh.
  • Constant external force C = init (0,-1,0)
    A constant per-scene force vector, part of the external force model.
  • Dynamic external forces D_i_n = init 0
    Per-vertex, per-frame force vectors (625 vertices per scene) that drive the deformation fit; the high-dimensional fitting part of the inverse problem.
  • Texture T = init 0.5
    Projective texture map optimized in the first phase to provide appearance for the geometry loss.
  • SVBRDF textures (diffuse, roughness, metallic) and environment map = init gray
    Optimized in the appearance phase via a decoder (16 patches of 128x128) and a 512x256 environment map.
  • Regularization weights lambda_E, lambda_F, lambda_env = 2, 2e-4, 1e-5
    Chosen by hand and tuned through ablations on the same nine evaluation scenes; directly affect the loss balance.
  • Damping delta and area density rho = delta=0.9, rho=0.1 kg/m2
    Fixed constants; motion is scale-invariant under force and mass scaling.
assumptions (6)
  • standard math Newton's second law and backward Euler time integration
    Used in Section 3.1 to derive the velocity and position update rules.
  • domain assumption The mass-spring model with homogeneous stiffness parameters is a sufficient approximation of cloth mechanics
    The method relies on this model to generate physically plausible deformations; real cloth has nonlinear, anisotropic, and heterogeneous behavior.
  • domain assumption Per-vertex external forces can represent wind and manipulation
    External forces are treated as free vectors per vertex per frame (Section 3.3.2, supplementary 6.2), which can also absorb modeling errors in the internal forces.
  • domain assumption Camera intrinsics/extrinsics and a template mesh aligned with the first frame are known
    Assumed throughout the pipeline; the paper states masks can come from Segment Anything but camera and template are inputs.
  • standard math Rendering gradients are orthogonal to the viewing direction
    Derived in supplementary Section 6.4 using the OpenCV perspective projection; used to justify the force regularization.
  • ad hoc to paper Stop-gradient on stiffness parameters in the energy regularization
    Introduced to prevent the optimizer from minimizing stiffness to reduce the energy regularizer; this is a deliberate algorithmic choice specific to this method.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SAFT: Shape and Appearance of Fabrics from Template via Differentiable Physical Simulations from Monocular Video." pith.science (2026). https://pith.science/paper/6HXSF6TN

@misc{pith2026250908828,
  author       = {Pith},
  title        = {Pith review of: SAFT: Shape and Appearance of Fabrics from Template via Differentiable Physical Simulations from Monocular Video},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6HXSF6TN}},
  note         = {Machine review of arXiv:2509.08828}
}
read the original abstract

The reconstruction of three-dimensional dynamic scenes is a well-established yet challenging task within the domain of computer vision. In this paper, we propose a novel approach that combines the domains of 3D geometry reconstruction and appearance estimation for physically based rendering and present a system that is able to perform both tasks for fabrics, utilizing only a single monocular RGB video sequence as input. In order to obtain realistic and high-quality deformations and renderings, a physical simulation of the cloth geometry and differentiable rendering are employed. In this paper, we introduce two novel regularization terms for the 3D reconstruction task that improve the plausibility of the reconstruction by addressing the depth ambiguity problem in monocular video. In comparison with the most recent methods in the field, we have reduced the error in the 3D reconstruction by a factor of 2.64 while requiring a medium runtime of 30 min per scene. Furthermore, the optimized motion achieves sufficient quality to perform an appearance estimation of the deforming object, recovering sharp details from this single monocular RGB video.

Figures

Figures reproduced from arXiv: 2509.08828 by the authors.

Figure 1
Figure 1. Overview of the three phases of our method. First, we perform projective texture mapping to obtain an accurate texture on the [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of the pipeline during one epoch. Initially, we perform a physical simulation to obtain the geometry for all frames [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Qualitative comparison between our method and two [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Depth images and differences for scene R6 frame 39. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 6
Figure 6. Figure 6: Two examples of estimated appearance of real-world [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: One example of estimating the appearance of the syn [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Schematic view of a fabric in its rest state, a bend fabric, [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Projection step in camera coordinates. The gradient of the loss func￾tion with respect to some arbitrary (physical) parame￾ter q can be decomposed us￾ing the chain-rule. We as￾sume the parameter q to have some influence on a vertex at position (x y z) that will be proj…
Figure 10
Figure 10. Figure 10: Renderings and mesh views of all methods for two [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: Qualitative comparison between depth images for two [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 12
Figure 12. Figure 12: Estimated appearance parameters of all real scenes. Major differences are highlighted with a red box. Please zoom in for details. [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]
Figure 13
Figure 13. Figure 13: Appearance parameters of all synthesized scenes. The first row depicts the input material textures or the environment map. [PITH_FULL_IMAGE:figures/full_fig_p019_13.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

84 extracted references · 7 linked inside Pith

  1. [27]

    In CVPR, pages 3948–3958, 2022

    Navami Kairanda, Edith Tretschk, Mohamed Elgharib, Christian Theobalt, and Vladislav Golyanik.ϕ-SfT: Shape- from-Template with a Physics-Based Deformation Model. In CVPR, pages 3948–3958, 2022. 1, 2, 3, 4, 5, 6, 7, 13, 14, 15, 16, 17, 18

    2022

  2. [1]

    Large steps in cloth sim- ulation

    David Baraff and Andrew Witkin. Large steps in cloth sim- ulation. InAnnual Conference on Computer Graphics and Interactive Techniques, page 43–54, 1998. 2, 3, 12

    1998

  3. [2]

    Shape-from-template.IEEE TPAMI, 37(10):2099–2118, 2015

    Adrien Bartoli, Yan G ´erard, Franc ¸ois Chadebecq, Toby Collins, and Daniel Pizarro. Shape-from-template.IEEE TPAMI, 37(10):2099–2118, 2015. 1

  4. [3]

    Pbns: Physically based neural simulator for unsupervised garment pose space deformation.CoRR, abs/2012.11310, 2020

    Hugo Bertiche, Meysam Madadi, and Sergio Escalera. Pbns: Physically based neural simulator for unsupervised garment pose space deformation.CoRR, abs/2012.11310, 2020. 2

    arXiv 2012

  5. [4]

    Neu- ral cloth simulation.ACM TOG, 41, 2022

    Hugo Bertiche, Meysam Madadi, and Sergio Escalera. Neu- ral cloth simulation.ACM TOG, 41, 2022. 2

    2022

  6. [5]

    Lensch, and Jan Kautz

    Mark Boss, Varun Jampani, Kihwan Kim, Hendrik P.A. Lensch, and Jan Kautz. Two-shot spatially-varying brdf and shape estimation. InProceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR),

  7. [6]

    Bregler, A

    C. Bregler, A. Hertzmann, and H. Biermann. Recovering non-rigid 3d shape from image streams. InCVPR, pages 690–696 vol.2, 2000. 1

    2000

  8. [7]

    The isowarp: the template-based visual geometry of isometric surfaces

    David Casillas-Perez, Daniel Pizarro, David Fuentes- Jimenez, Manuel Mazo, and Adrien Bartoli. The isowarp: the template-based visual geometry of isometric surfaces. IJCV, 129(7):2194–2222, 2021. 1

    2021

Show all 84 references
  1. [8]

    Neural ordinary differential equa- tions.Advances in neural information processing systems, 31, 2018

    Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equa- tions.Advances in neural information processing systems, 31, 2018. 2

    2018

  2. [9]

    Gabriel Cirio, Jorge Lopez-Moreno, David Miraut, and Miguel A. Otaduy. Yarn-level simulation of woven cloth. ACM TOG, 33(6), 2014. 2, 3, 12

    2014

  3. [10]

    Efficient simulation of knitted cloth using persistent contacts

    Gabriel Cirio, Jorge Lopez-Moreno, and Miguel A Otaduy. Efficient simulation of knitted cloth using persistent contacts. InProceedings of the 14th ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pages 55–61, 2015. 2, 3

    2015

  4. [11]

    Neural parametric gaussians for monocular non-rigid object reconstruction

    Devikalyan Das, Christopher Wewer, Raza Yunus, Eddy Ilg, and Jan Eric Lenssen. Neural parametric gaussians for monocular non-rigid object reconstruction. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 10715–10725, 2024. 1

    2024

  5. [12]

    Drapenet: Garment generation and self- supervised draping

    Luca De Luigi, Ren Li, Benoit Guillard, Mathieu Salzmann, and Pascal Fua. Drapenet: Garment generation and self- supervised draping. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 1451–1460, 2023. 2

    2023

  6. [13]

    Single-image svbrdf cap- ture with a rendering-aware deep network.ACM Trans

    Valentin Deschaintre, Miika Aittala, Fredo Durand, George Drettakis, and Adrien Bousseau. Single-image svbrdf cap- ture with a rendering-aware deep network.ACM Trans. Graph., 37, 2018. 2

    2018

  7. [14]

    Diffpd: Dif- ferentiable projective dynamics.ACM TOG, 41(2), 2021

    Tao Du, Kui Wu, Pingchuan Ma, Sebastien Wah, Andrew Spielberg, Daniela Rus, and Wojciech Matusik. Diffpd: Dif- ferentiable projective dynamics.ACM TOG, 41(2), 2021. 2

    2021

  8. [15]

    Learning-based bending stiffness parameter estima- tion by a drape tester.ACM TOG, 41(6), 2022

    Xudong Feng, Wenchao Huang, Weiwei Xu, and Huamin Wang. Learning-based bending stiffness parameter estima- tion by a drape tester.ACM TOG, 41(6), 2022. 2

    2022

  9. [16]

    Texture-generic deep shape-from-template.IEEE Access, 9:75211–75230,

    David Fuentes-Jimenez, Daniel Pizarro, David Casillas- Perez, Toby Collins, and Adrien Bartoli. Texture-generic deep shape-from-template.IEEE Access, 9:75211–75230,

  10. [17]

    Deep inverse rendering for high-resolution svbrdf estimation from an arbitrary number of images.ACM Trans

    Duan Gao, Xiao Li, Yue Dong, Pieter Peers, Kun Xu, and Xin Tong. Deep inverse rendering for high-resolution svbrdf estimation from an arbitrary number of images.ACM Trans. Graph., 38(4):134–1, 2019. 2

    2019

  11. [18]

    Modeling the dy- namics of pde systems with physics-constrained deep auto- regressive networks.Journal of Computational Physics, 403: 109056, 2020

    Nicholas Geneva and Nicholas Zabaras. Modeling the dy- namics of pde systems with physics-constrained deep auto- regressive networks.Journal of Computational Physics, 403: 109056, 2020. 2

    2020

  12. [19]

    Hdm-net: Monocular non-rigid 3d recon- struction with learned deformation model

    Vladislav Golyanik, Soshi Shimada, Kiran Varanasi, and Didier Stricker. Hdm-net: Monocular non-rigid 3d recon- struction with learned deformation model. InVirtual Reality and Augmented Reality, pages 51–72. Springer International Publishing, 2018. 1

    2018

  13. [20]

    Fine-grained dif- ferentiable physics: a yarn-level model for fabrics

    D Gong, Z Zhu, A Bulpitt, and H Wang. Fine-grained dif- ferentiable physics: a yarn-level model for fabrics. InICLR,

  14. [21]

    Black, and Otmar Hilliges

    Artur Grigorev, Michael J. Black, and Otmar Hilliges. Hood: Hierarchical graphs for generalized modelling of clothing dynamics. InCVPR, pages 16965–16974, 2023. 2

    2023

  15. [22]

    Real-time geometry, albedo, and mo- tion reconstruction using a single rgb-d camera.ACM Trans

    Kaiwen Guo, Feng Xu, Tao Yu, Xiaoyang Liu, Qionghai Dai, and Yebin Liu. Real-time geometry, albedo, and mo- tion reconstruction using a single rgb-d camera.ACM Trans. Graph., 2017. 2

    2017

  16. [23]

    Hyunho Ha, Seung-Hwan Baek, Giljoo Nam, and Min H. Kim. Progressive acquisition of svbrdf and shape in motion. Computer Graphics Forum, 39(6):480–495, 2020. 2

    2020

  17. [24]

    Shape, light, and material decomposition from images using monte carlo rendering and denoising

    Jon Hasselgren, Nikolai Hofmann, and Jacob Munkberg. Shape, light, and material decomposition from images using monte carlo rendering and denoising. InAdvances in Neural Information Processing Systems, pages 22856–22869. Cur- ran Associates, Inc., 2022. 2, 4, 15

    2022

  18. [25]

    Diff- taichi: Differentiable programming for physical simulation

    Yuanming Hu, Luke Anderson, Tzu-Mao Li, Qi Sun, Nathan Carr, Jonathan Ragan-Kelley, and Fredo Durand. Diff- taichi: Differentiable programming for physical simulation. InICLR, 2020. 2

    2020

  19. [26]

    Estimating cloth simu- lation parameters from a static drape using neural networks

    Eunjung Ju and Myung Geol Choi. Estimating cloth simu- lation parameters from a static drape using neural networks. IEEE Access, 8:195113–195121, 2020. 2

    2020

  20. [28]

    Neuralclothsim: Neural defor- mation fields meet the kirchhoff-love thin shell theory

    Navami Kairanda, Marc Habermann, Christian Theobalt, and Vladislav Golyanik. Neuralclothsim: Neural defor- mation fields meet the kirchhoff-love thin shell theory. arXiv:2308.12970, 2023. 2

    2023 arXiv

  21. [29]

    Uni- fied shape and appearance reconstruction with joint camera parameter refinement.Graphical Models, 129, 2023

    Julian Kaltheuner, Patrick Stotko, and Reinhard Klein. Uni- fied shape and appearance reconstruction with joint camera parameter refinement.Graphical Models, 129, 2023. 2, 4

    2023

  22. [30]

    Rosa: Reconstructing object shape and appearance textures by adaptive detail transfer

    Julian Kaltheuner, Patrick Stotko, and Reinhard Klein. Rosa: Reconstructing object shape and appearance textures by adaptive detail transfer. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2025. 2, 4

    2025

  23. [31]

    Deep svbrdf acquisition and modelling: A survey.Computer Graphics Forum, 43(6):e15199, 2024

    Behnaz Kavoosighafi, Saghi Hajisharif, Ehsan Miandji, Gabriel Baravdish, Wen Cao, and Jonas Unger. Deep svbrdf acquisition and modelling: A survey.Computer Graphics Forum, 43(6):e15199, 2024. 2

    2024

  24. [32]

    Berg, Wan-Yen Lo, Piotr Dollar, and Ross Girshick

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C. Berg, Wan-Yen Lo, Piotr Dollar, and Ross Girshick. Segment anything. InProceedings of the IEEE/CVF International Conference on Computer Vision...

    2023

  25. [33]

    Modular primitives for high-performance differentiable rendering.ACM TOG, 39 (6), 2020

    Samuli Laine, Janne Hellsten, Tero Karras, Yeongho Seol, Jaakko Lehtinen, and Timo Aila. Modular primitives for high-performance differentiable rendering.ACM TOG, 39 (6), 2020. 3, 14

    2020

  26. [34]

    Mod- eling surface appearance from a single photograph using self-augmented convolutional neural networks.ACM Trans

    Xiao Li, Yue Dong, Pieter Peers, and Xin Tong. Mod- eling surface appearance from a single photograph using self-augmented convolutional neural networks.ACM Trans. Graph., 36(4), 2017. 2

    2017

  27. [35]

    Diffcloth: Differentiable cloth simulation with dry frictional contact.ACM TOG, 42(1), 2022

    Yifei Li, Tao Du, Kui Wu, Jie Xu, and Wojciech Matusik. Diffcloth: Differentiable cloth simulation with dry frictional contact.ACM TOG, 42(1), 2022. 2

    2022

  28. [36]

    Learning to recon- struct shape and spatially-varying reflectance from a single image.ACM Trans

    Zhengqin Li, Zexiang Xu, Ravi Ramamoorthi, Kalyan Sunkavalli, and Manmohan Chandraker. Learning to recon- struct shape and spatially-varying reflectance from a single image.ACM Trans. Graph., 37, 2018. 2

    2018

  29. [37]

    Differen- tiable cloth simulation for inverse problems

    Junbang Liang, Ming Lin, and Vladlen Koltun. Differen- tiable cloth simulation for inverse problems. InNeurIPS. Curran Associates, Inc., 2019. 2

    2019

  30. [38]

    Daniel Lichy, Jiaye Wu, Soumyadip Sengupta, and David W. Jacobs. Shape and material capture at home. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 6123–6133, 2021. 2

    2021

  31. [39]

    Better together: Joint reasoning for non- rigid 3d reconstruction with specularities and shading.arXiv preprint arXiv:1708.01654, 2017

    Qi Liu-Yin, Rui Yu, Lourdes Agapito, Andrew Fitzgibbon, and Chris Russell. Better together: Joint reasoning for non- rigid 3d reconstruction with specularities and shading.arXiv preprint arXiv:1708.01654, 2017. 2

    2017 arXiv

  32. [40]

    Warp: A high-performance python frame- work for gpu simulation and graphics

    Miles Macklin. Warp: A high-performance python frame- work for gpu simulation and graphics. https://github.com/ nvidia/warp, 2022. NVIDIA GPU Technology Conference (GTC). 2

    2022

  33. [41]

    Physical simulation layer for accurate 3d modeling

    Mariem Mezghanni, Th ´eo Bodrito, Malika Boulkenafed, and Maks Ovsjanikov. Physical simulation layer for accurate 3d modeling. InCVPR, pages 13514–13523, 2022. 2

    2022

  34. [42]

    Extracting Triangular 3D Models, Materials, and Light- ing From Images

    Jacob Munkberg, Jon Hasselgren, Tianchang Shen, Jun Gao, Wenzheng Chen, Alex Evans, Thomas M¨uller, and Sanja Fi- dler. Extracting Triangular 3D Models, Materials, and Light- ing From Images. InProceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition...

    2022

  35. [43]

    Rahul Narain, Armin Samii, and James F. O’Brien. Adaptive anisotropic remeshing for cloth simulation.ACM TOG, 31 (6), 2012. 2, 14

    2012

  36. [44]

    Local non-rigid structure-from-motion from diffeomorphic map- pings

    Shaifali Parashar, Mathieu Salzmann, and Pascal Fua. Local non-rigid structure-from-motion from diffeomorphic map- pings. InCVPR, 2020. 1

    2020

  37. [45]

    Pytorch: An imperative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Rai- son, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, L...

    2019

  38. [46]

    Learning mesh-based simulation with graph networks

    Tobias Pfaff, Meire Fortunato, Alvaro Sanchez-Gonzalez, and Peter Battaglia. Learning mesh-based simulation with graph networks. InICLR, 2020. 2

    2020

  39. [47]

    Geometry-aware network for non-rigid shape prediction from a single view

    Albert Pumarola, Antonio Agudo, Lorenzo Porzi, Alberto Sanfeliu, Vincent Lepetit, and Francesc Moreno-Noguer. Geometry-aware network for non-rigid shape prediction from a single view. InCVPR, 2018. 1

    2018

  40. [48]

    Scalable differentiable physics for learning and control

    Yi-Ling Qiao, Junbang Liang, Vladlen Koltun, and Ming C Lin. Scalable differentiable physics for learning and control. InICML, pages 7847–7856, 2020. 2

    2020

  41. [49]

    Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics-informed neural networks: A deep learning frame- work for solving forward and inverse problems involving nonlinear partial differential equations.Journal of Computa- tional physics, 378:686–707, 2019. 2

    2019

  42. [50]

    Accelerating 3d deep learning with pytorch3d, 2020

    Nikhila Ravi, Jeremy Reizenstein, David Novotny, Taylor Gordon, Wan-Yen Lo, Justin Johnson, and Georgia Gkioxari. Accelerating 3d deep learning with pytorch3d, 2020. 14

    2020

  43. [51]

    Sam 2: Segment anything in images and videos.arXiv preprint arXiv:2408.00714, 2024

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman R¨adle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos.arXiv preprint arXiv:2408.00714, 2024. 2

    2024 arXiv

  44. [52]

    How will it drape like? capturing fabric mechanics from depth images.Computer Graphics Forum, 42(2):149–160, 2023

    Carlos Rodriguez-Pardo, Melania Prieto-Martin, Dan Casas, and Elena Garces. How will it drape like? capturing fabric mechanics from depth images.Computer Graphics Forum, 42(2):149–160, 2023. 2

    2023

  45. [53]

    Gaussian garments: Reconstruct- ing simulation-ready clothing with photorealistic appearance from multi-view video.arXiv preprint arXiv:2409.08189,

    Boxiang Rong, Artur Grigorev, Wenbo Wang, Michael J Black, Bernhard Thomaszewski, Christina Tsalicoglou, and Otmar Hilliges. Gaussian garments: Reconstruct- ing simulation-ready clothing with photorealistic appearance from multi-view video.arXiv preprint arXiv:2409.08189,

    arXiv

  46. [54]

    De- formable surface tracking ambiguities

    Mathieu Salzmann, Vincent Lepetit, and Pascal Fua. De- formable surface tracking ambiguities. InCVPR, pages 1–8,

  47. [55]

    Otaduy, and Dan Casas

    Igor Santesteban, Miguel A. Otaduy, and Dan Casas. Snug: Self-supervised neural dynamic garments. InCVPR, pages 8140–8150, 2022. 2

    2022

  48. [56]

    Autoclip: Adaptive gradient clipping for source separation networks

    Prem Seetharaman, Gordon Wichern, Bryan Pardo, and Jonathan Le Roux. Autoclip: Adaptive gradient clipping for source separation networks. In2020 IEEE 30th Interna- tional Workshop on Machine Learning for Signal Processing (MLSP). IEEE, 2020. 14

    2020

  49. [57]

    Towards multi- layered 3d garments animation

    Yidi Shao, Chen Change Loy, and Bo Dai. Towards multi- layered 3d garments animation. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 14361–14370, 2023. 2

    2023

  50. [58]

    Robusft: Ro- bust real-time shape-from-template, a c++ library.Image and Vision Computing, 141:104867, 2024

    Mohammadreza Shetab-Bushehri, Miguel Aranda, Erol ¨Ozg¨ur, Youcef Mezouar, and Adrien Bartoli. Robusft: Ro- bust real-time shape-from-template, a c++ library.Image and Vision Computing, 141:104867, 2024. 1

    2024

  51. [59]

    Ismo-gan: Adversarial learning for monoc- ular non-rigid 3d reconstruction

    Soshi Shimada, Vladislav Golyanik, Christian Theobalt, and Didier Stricker. Ismo-gan: Adversarial learning for monoc- ular non-rigid 3d reconstruction. InCVPRW, 2019. 1

    2019

  52. [60]

    Neural dense non- rigid structure from motion with latent space constraints

    Vikramjit Sidhu, Edgar Tretschk, Vladislav Golyanik, An- tonio Agudo, and Christian Theobalt. Neural dense non- rigid structure from motion with latent space constraints. In ECCV, 2020. 1

    2020

  53. [61]

    Sobolevfusion: 3d reconstruction of scenes undergoing free non-rigid motion

    Miroslava Slavcheva, Maximilian Baust, and Slobodan Ilic. Sobolevfusion: 3d reconstruction of scenes undergoing free non-rigid motion. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018. 2

    2018

  54. [62]

    Physics- guided Shape-from-Template: Monocular Video Percep- tion through Neural Surrogate Models

    David Stotko, Nils Wandel, and Reinhard Klein. Physics- guided Shape-from-Template: Monocular Video Percep- tion through Neural Surrogate Models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11895–11904, 2024. 1, 2, 3, 4, 5, 6, 7, ...

    2024

  55. [63]

    Caphy: Cap- turing physical properties for animatable human avatars

    Zhaoqi Su, Liangxiao Hu, Siyou Lin, Hongwen Zhang, Shengping Zhang, Justus Thies, and Yebin Liu. Caphy: Cap- turing physical properties for animatable human avatars. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 14150–14160, 2023. 2

    2023

  56. [64]

    Sullivan.Curves of Finite Total Curvature, pages 137–161

    John M. Sullivan.Curves of Finite Total Curvature, pages 137–161. Birkh ¨auser Basel, Basel, 2008. 12

    2008

  57. [65]

    Dif- fcloud: Real-to-sim from point clouds with differentiable simulation and rendering of deformable objects

    Priya Sundaresan, Rika Antonova, and Jeannette Bohgl. Dif- fcloud: Real-to-sim from point clouds with differentiable simulation and rendering of deformable objects. InIROS, pages 10828–10835, 2022. 2

    2022

  58. [66]

    Improved shape-from-template method with perspective space constraints for disappearing features.Complex & Intelligent Systems, pages 1–14, 2024

    Dongliang Tan, Huamin Yang, Zhengang Jiang, Weili Shi, Jun Qin, and Feng Qu. Improved shape-from-template method with perspective space constraints for disappearing features.Complex & Intelligent Systems, pages 1–14, 2024. 2

    2024

  59. [67]

    State of the art in dense monocular non-rigid 3d reconstruction.Comput

    Edith Tretschk, Navami Kairanda, Mallikarjun B R, Rishabh Dabral, Adam Kortylewski, Bernhard Egger, Marc Haber- mann, Pascal Fua, Christian Theobalt, and Vladislav Golyanik. State of the art in dense monocular non-rigid 3d reconstruction.Comput. Graph. Forum, 42(2):485–520,

  60. [68]

    Huang, Omid Taheri, Michael J

    Shashank Tripathi, Lea M ¨uller, Chun-Hao P. Huang, Omid Taheri, Michael J. Black, and Dimitrios Tzionas. 3d human pose estimation via intuitive physics. InCVPR, pages 4713– 4725, 2023. 2

    2023

  61. [69]

    Un- supervised deep learning of incompressible fluid dynamics

    Nils Wandel, Michael Weinmann, and Reinhard Klein. Un- supervised deep learning of incompressible fluid dynamics. CoRR, abs/2006.08762, 2020. 2

    2006 arXiv

  62. [70]

    Metamizer: a versatile neural optimizer for fast and accurate physics sim- ulations.arXiv preprint arXiv:2410.19746, 2024

    Nils Wandel, Stefan Schulz, and Reinhard Klein. Metamizer: a versatile neural optimizer for fast and accurate physics sim- ulations.arXiv preprint arXiv:2410.19746, 2024. 2

    2024 arXiv

  63. [71]

    Interdiff: Generating 3d human-object interactions with physics-informed diffusion

    Sirui Xu, Zhengyuan Li, Yu-Xiong Wang, and Liang-Yan Gui. Interdiff: Generating 3d human-object interactions with physics-informed diffusion. InICCV, pages 14928–14940,

  64. [72]

    Zhang, Zachary Manchester, and Deva Ramanan

    Gengshan Yang, Shuo Yang, John Z. Zhang, Zachary Manchester, and Deva Ramanan. Ppr: Physically plausi- ble reconstruction from monocular videos. InICCV, pages 3914–3924, 2023. 2

    2023

  65. [73]

    Shan Yang, Junbang Liang, and Ming C. Lin. Learning- based cloth material recovery from video. InICCV, 2017. 2

    2017

  66. [74]

    Single image surface appearance modeling with self- augmented cnns and inexact supervision

    Wenjie Ye, Xiao Li, Yue Dong, Pieter Peers, and Xin Tong. Single image surface appearance modeling with self- augmented cnns and inexact supervision. InComputer Graphics Forum, pages 201–211. Wiley Online Library,

  67. [75]

    Deep reflectance scanning: Recovering spatially-varying mate- rial appearance from a flash-lit video sequence

    Wenjie Ye, Yue Dong, Pieter Peers, and Baining Guo. Deep reflectance scanning: Recovering spatially-varying mate- rial appearance from a flash-lit video sequence. InCom- puter Graphics Forum, pages 409–427. Wiley Online Li- brary, 2021. 2

    2021

  68. [76]

    Rui Yu, Chris Russell, Neill D. F. Campbell, and Lourdes Agapito. Direct, dense, and deformable: Template-based non-rigid 3d reconstruction from rgb video. InICCV, 2015. 1

    2015

  69. [77]

    Physdiff: Physics-guided human motion diffusion model

    Ye Yuan, Jiaming Song, Umar Iqbal, Arash Vahdat, and Jan Kautz. Physdiff: Physics-guided human motion diffusion model. InICCV, pages 16010–16021, 2023. 2

    2023

  70. [78]

    Recent trends in 3d reconstruction of general non-rigid scenes

    Raza Yunus, Jan Eric Lenssen, Michael Niemeyer, Yiyi Liao, Christian Rupprecht, Christian Theobalt, Gerard Pons-Moll, Jia-Bin Huang, Vladislav Golyanik, and Eddy Ilg. Recent trends in 3d reconstruction of general non-rigid scenes. In Computer Graphics Forum, page e15062. Wiley...

    2024

  71. [79]

    Iron: Inverse rendering by optimizing neural sdfs and materials from photometric images

    Kai Zhang, Fujun Luan, Zhengqi Li, and Noah Snavely. Iron: Inverse rendering by optimizing neural sdfs and materials from photometric images. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5565–5574, 2022. 2

    2022

  72. [80]

    Physavatar: Learning the physics of dressed 3d avatars from visual observations

    Yang Zheng, Qingqing Zhao, Guandao Yang, Wang Yi- fan, Donglai Xiang, Florian Dubost, Dmitry Lagun, Thabo Beeler, Federico Tombari, Leonidas Guibas, and Gordon Wetzstein. Physavatar: Learning the physics of dressed 3d avatars from visual observations. InComputer Vision – ECCV ...

    2024

  73. [81]

    Adversarial single-image svbrdf estimation with hybrid training

    Xilong Zhou and Nima Khademi Kalantari. Adversarial single-image svbrdf estimation with hybrid training. InCom- puter Graphics Forum, pages 315–325. Wiley Online Li- brary, 2021. 2 Supplementary Material This supplementary material contains further information about the physic...

    2021

  74. [82]

    Numerical Integration Scheme For simulation, we discretize the time into constant steps of∆t= 5 msresulting in all time-dependent quantities to be discretized accordingly,e.g

    Method 6.1. Numerical Integration Scheme For simulation, we discretize the time into constant steps of∆t= 5 msresulting in all time-dependent quantities to be discretized accordingly,e.g. vertex positions⃗ x n = ⃗ x(n∆t). The update rules when applying the backward Eu- ler sch...

  75. [83]

    Simulation Scheme As a consequence of the spatial discretization into discrete vertices, the energy terms dependent not only on a single vertex but also on neighboring vertices

    Implementation Details 7.1. Simulation Scheme As a consequence of the spatial discretization into discrete vertices, the energy terms dependent not only on a single vertex but also on neighboring vertices. In order to get an accurate update step for all vertices at once, we ha...

  76. [84]

    Evaluation 8.1. Qualitative Comparison In addition to the qualitative comparison in the main paper, we present mesh renderings and a novel view for the same scenes in Figure 10 and for all scenes in the supplemental video. The novel view camera is always rotated by45 ◦ to the ...

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.