pith. sign in

arxiv: 2605.18373 · v1 · pith:BI2LJN4Mnew · submitted 2026-05-18 · 💻 cs.RO · cs.LG· math.DS· math.OC

Dynamic robotic cloth folding with efficient Koopman operator-based model predictive control

Edoardo Caldarelli , Franco Coltraro , Adri\`a Colom\'e , Lorenzo Rosasco , Carme Torras This is my paper

Pith reviewed 2026-05-20 09:33 UTC · model grok-4.3

classification 💻 cs.RO cs.LGmath.DSmath.OC
keywords robotic cloth foldingKoopman operatormodel predictive controldynamic foldingsim-to-real transfercloth dynamics
0
0 comments X

The pith

Koopman operator linear models enable efficient generation of fast, accurate robotic cloth folding trajectories for unseen poses.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes that a linear surrogate obtained via kernel-based Koopman operator regression, trained solely on high-fidelity physics-simulator data of cloth, can replace expensive nonlinear dynamics inside a model predictive controller. This substitution lets the controller plan and execute rapid folding motions that reach new target poses. A sympathetic reader would care because current robotic cloth folding is either slow or imprecise when using fast motions, and this method promises to close the gap by transferring simulation-trained linear models directly to real manipulators without measurable loss in fold quality. Both simulated and physical robot trials confirm that the resulting trajectories fold unseen configurations successfully.

Core claim

The linearization supplied by the Koopman operator-based model can be employed to efficiently generate fast folding trajectories to unseen poses, without sacrificing folding accuracy. The surrogate is trained with data from a high-fidelity, physics-based cloth simulator and inserted into a model predictive control loop in place of the original nonlinear simulator.

What carries the argument

Kernel-based Koopman operator regression that yields a linear surrogate for cloth dynamics, allowing the model predictive controller to run at high speed while still planning dynamic folds.

If this is right

  • Folding trajectories for new cloth poses can be computed orders of magnitude faster than with full nonlinear simulation inside the controller.
  • Dynamic folding remains accurate even though the planner uses a linearized model rather than the original physics simulator.
  • Sim-to-real transfer succeeds for fast motions without requiring multiple corrective attempts on the physical robot.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same linearization step could be applied to other deformable-object tasks such as unfolding or draping if similar simulator data are available.
  • Periodic re-training of the Koopman model from a small set of real-robot measurements might further reduce any residual sim-to-real gap.

Load-bearing premise

The linear surrogate learned only from simulator data stays accurate enough when the planned trajectories are executed on real cloth whose dynamics may differ.

What would settle it

A real-robot trial that produces visibly poorer folds or fails on previously unseen poses, while the same trajectories succeed in the simulator, would falsify the central claim.

Figures

Figures reproduced from arXiv: 2605.18373 by Adri\`a Colom\'e, Carme Torras, Edoardo Caldarelli, Franco Coltraro, Lorenzo Rosasco.

Figure 1
Figure 1. Figure 1: An example of a dynamic cloth folding trajectory [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Summary of the cloth folding strategy proposed in this paper. A cloth simulator [4] simulates synthetic folding [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Some of the training parabolas used for the data [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: (a) Final folding error evaluated across 10 target folds generated with single-handed motions, for different values [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: (a) x-y visualization of the trajectories generated with different values of the prediction horizon T, and m = 158. An initial motion towards the center of the cloth emerges as a consequence of a larger value for T. This type of motion is pivotal for the fold to be successful. The initial position is marked by a red dot. (b) Final folding error evaluated across 10 target folds generated with single-handed … view at source ↗
Figure 6
Figure 6. Figure 6: An illustrative example of a folding trajectory generated by our MPC pipeline, with a woolen piece of cloth. The [PITH_FULL_IMAGE:figures/full_fig_p006_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: A qualitative visualization of the folds obtained with our MPC pipeline, in the sim-to-real evaluation. The top row [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
read the original abstract

Robotic cloth folding is a challenging task, particularly when considering dynamic folding tasks, which aim at folding cloth by fast motions that leverage its dynamics. When subject to such fast motions, the complexity of cloth dynamics hinders both system identification and planning of folding trajectories, resulting in a difficult simulation-to-reality transfer when using physical models of cloth. Compared to the dexterity that humans exhibit when performing folding tasks, robotic approaches usually employ small garments with quite rigid dynamics, and are either too slow, or fast but imprecise, requiring several attempts to achieve a reasonably good fold. In this paper, we tackle these challenges by generating fast folding trajectories with a novel model predictive controller, integrating physics-based simulation of cloth dynamics and efficient, kernel-based Koopman operator regression. Koopman operator regression, an increasingly popular machine learning technique for nonlinear system identification, is used to obtain a linear model for the cloth being folded. Such a surrogate model, trained with data from a high-fidelity, physics-based cloth simulator, can then be employed within a suitable model predictive control algorithm, in place of the costly, nonlinear one, to efficiently generate folding trajectories to be executed by a robotic manipulator. Both in simulated and real-robot experiments, we show how the linearization supplied by the Koopman operator-based model can be employed to efficiently generate fast folding trajectories to unseen poses, without sacrificing folding accuracy.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The manuscript proposes a Koopman operator-based linear surrogate model, obtained via kernel regression on trajectories from a high-fidelity physics simulator, to replace the nonlinear cloth dynamics inside a model predictive controller. This enables efficient generation of fast folding trajectories that are claimed to transfer from simulation to a real robot while maintaining accuracy on cloth poses outside the training distribution.

Significance. If the sim-to-real accuracy claim holds with quantitative support, the work would provide a practical route to real-time MPC for dynamic deformable-object tasks by leveraging a learned linear model in place of expensive nonlinear simulation. The explicit combination of physics-based data generation with Koopman regression for cloth folding is a concrete strength that could be extended to other manipulation problems.

major comments (2)
  1. [Real-robot experiments] Real-robot experiments section: the central claim that the Koopman linearization produces fast trajectories to unseen poses 'without sacrificing folding accuracy' is not supported by reported quantitative metrics, error bars, or standard deviations on final fold quality; without these, it is impossible to verify that closed-loop performance on hardware matches the simulated results.
  2. [Koopman model training and validation] Koopman model training and validation: no ablation on kernel choice or analysis of how the distribution of training trajectories affects prediction error during the high-speed segments of the MPC-generated motions is provided, yet these choices directly determine whether the linear surrogate remains faithful for the fast dynamics required by the central claim.
minor comments (1)
  1. [Method] Notation for the lifted state and kernel functions could be clarified with an explicit equation reference to avoid ambiguity when describing the regression procedure.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive and detailed comments, which help clarify the presentation of our results. We address each major comment below and indicate the revisions we will make to strengthen the quantitative support and analysis in the manuscript.

read point-by-point responses

  1. Referee: [Real-robot experiments] Real-robot experiments section: the central claim that the Koopman linearization produces fast trajectories to unseen poses 'without sacrificing folding accuracy' is not supported by reported quantitative metrics, error bars, or standard deviations on final fold quality; without these, it is impossible to verify that closed-loop performance on hardware matches the simulated results.

    Authors: We acknowledge that the real-robot results would be more convincing with explicit quantitative metrics and variability measures. The original manuscript reports success through visual demonstrations and some aggregate success rates for the hardware trials, while detailed error statistics (such as corner position deviations and fold quality) are provided primarily for the simulated cases. To directly address the concern, we will add tables and plots in the revised version showing mean final fold errors with standard deviations and error bars computed across multiple real-robot trials for both seen and unseen poses. This will allow direct comparison to the simulated performance. revision: yes

  2. Referee: [Koopman model training and validation] Koopman model training and validation: no ablation on kernel choice or analysis of how the distribution of training trajectories affects prediction error during the high-speed segments of the MPC-generated motions is provided, yet these choices directly determine whether the linear surrogate remains faithful for the fast dynamics required by the central claim.

    Authors: We agree that a more thorough validation of the Koopman surrogate would strengthen the paper. The kernel (radial basis function) was selected after initial cross-validation on held-out trajectories because it provided the best trade-off between prediction accuracy and computational efficiency for the cloth state representation used. Training data were generated from physics simulations covering a range of folding speeds and initial configurations to target the relevant dynamics. In the revision we will add an explicit ablation comparing kernel choices (RBF, polynomial, and linear) on prediction error, as well as a time-resolved error analysis that isolates high-speed segments of the MPC trajectories. This will quantify how well the linear model approximates the fast dynamics and will include discussion of the training distribution's coverage. revision: yes

Circularity Check

0 steps flagged

No circularity: Koopman surrogate learned from external simulator data

full rationale

The paper trains a kernel-based Koopman operator regression exclusively on trajectories generated by a high-fidelity physics-based cloth simulator, then substitutes the resulting linear surrogate into an MPC planner to produce folding trajectories for unseen poses. This is a standard data-driven system-identification pipeline whose outputs are not defined in terms of themselves; the linear model is fitted to independent simulation data, and the claim of accurate real-robot execution on novel poses is an empirical generalization tested in both simulation and hardware experiments rather than a tautological renaming or self-referential fit. No load-bearing step reduces by the paper's own equations to a fitted quantity defined in terms of the target result.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the transferability of a data-driven linear model from simulator to real cloth and on the existence of a Koopman operator that sufficiently linearizes the cloth dynamics for the MPC horizon.

axioms (1)
  • domain assumption A Koopman operator exists that yields an accurate finite-dimensional linear representation of the cloth dynamics over the relevant state space and time horizon.
    Invoked when the paper states that the surrogate linear model replaces the costly nonlinear simulator inside the MPC.

pith-pipeline@v0.9.0 · 5798 in / 1243 out tokens · 35694 ms · 2026-05-20T09:33:10.367778+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

24 extracted references · 24 canonical work pages

  1. [1]

    Modeling, learning, perception, and control methods for deformable object manipulation,

    H. Yin, A. Varava, and D. Kragic, “Modeling, learning, perception, and control methods for deformable object manipulation,”Science Robotics, vol. 6, no. 54, p. eabd8803, 2021

    work page 2021

  2. [2]

    Unfolding the literature: A review of robotic cloth manipulation,

    A. Longhini, Y . Wang, I. Garcia-Camacho, D. Blanco-Mulero, M. Mo- letta, M. Welle, G. Aleny `a, H. Yin, Z. Erickson, D. Held,et al., “Unfolding the literature: A review of robotic cloth manipulation,” Annual Review of Control, Robotics, and Autonomous Systems, vol. 8, 2024

    work page 2024

  3. [3]

    An inextensible model for the robotic manipulation of textiles,

    F. Coltraro, J. Amor ´os, M. Alberich-Carrami ˜nana, and C. Torras, “An inextensible model for the robotic manipulation of textiles,”Applied Mathematical Modelling, vol. 101, pp. 832–858, 2022

    work page 2022

  4. [4]

    A novel collision model for inextensible textiles and its experimental validation,

    F. Coltraro, J. Amor ´os, M. Alberich-Carrami ˜nana, and C. Torras, “A novel collision model for inextensible textiles and its experimental validation,”Applied Mathematical Modelling, vol. 128, pp. 287–308, 2024

    work page 2024

  5. [5]

    Flingbot: The unreasonable effectiveness of dynamic manipulation for cloth unfolding,

    H. Ha and S. Song, “Flingbot: The unreasonable effectiveness of dynamic manipulation for cloth unfolding,” inConference on Robot Learning, pp. 24–33, PMLR, 2022

    work page 2022

  6. [6]

    Speedfolding: Learning efficient bimanual folding of garments,

    Y . Avigal, L. Berscheid, T. Asfour, T. Kr ¨oger, and K. Goldberg, “Speedfolding: Learning efficient bimanual folding of garments,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1–8, IEEE, 2022

    work page 2022

  7. [7]

    Learning visual feedback control for dynamic cloth folding,

    J. Hietala, D. Blanco-Mulero, G. Alcan, and V . Kyrki, “Learning visual feedback control for dynamic cloth folding,” in2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1455–1462, IEEE, 2022

    work page 2022

  8. [8]

    Adafold: Adapting folding trajectories of cloths via feedback-loop manipula- tion,

    A. Longhini, M. C. Welle, Z. Erickson, and D. Kragic, “Adafold: Adapting folding trajectories of cloths via feedback-loop manipula- tion,”IEEE Robotics and Automation Letters, 2024

    work page 2024

  9. [9]

    Dynamic manipulation,

    M. T. Mason and K. M. Lynch, “Dynamic manipulation,” inProceed- ings of 1993 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’93), vol. 1, pp. 152–159, IEEE, 1993

    work page 1993

  10. [10]

    QDP: Learning to sequentially optimise quasi-static and dynamic manipu- lation primitives for robotic cloth manipulation,

    D. Blanco-Mulero, G. Alcan, F. J. Abu-Dakka, and V . Kyrki, “QDP: Learning to sequentially optimise quasi-static and dynamic manipu- lation primitives for robotic cloth manipulation,” in2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 984–991, IEEE, 2023

    work page 2023

  11. [11]

    Benchmarking the sim-to-real gap in cloth manipulation,

    D. Blanco-Mulero, O. Barbany, G. Alcan, A. Colom ´e, C. Torras, and V . Kyrki, “Benchmarking the sim-to-real gap in cloth manipulation,” IEEE Robotics and Automation Letters, vol. 9, no. 3, pp. 2981–2988, 2024

    work page 2024

  12. [12]

    Linear quadratic control of nonlinear systems with Koopman operator learning and the Nystr ¨om method,

    E. Caldarelli, A. Chatalic, A. Colom ´e, C. Molinari, C. Ocampo- Martinez, C. Torras, and L. Rosasco, “Linear quadratic control of nonlinear systems with Koopman operator learning and the Nystr ¨om method,”Automatica, vol. 177, p. 112302, 2025

    work page 2025

  13. [13]

    Hamiltonian systems and transformation in Hilbert space,

    B. O. Koopman, “Hamiltonian systems and transformation in Hilbert space,”Proceedings of the National Academy of Sciences, vol. 17, no. 5, pp. 315–318, 1931

    work page 1931

  14. [14]

    Linear predictors for nonlinear dynamical systems: Koopman operator meets model predictive control,

    M. Korda and I. Mezi ´c, “Linear predictors for nonlinear dynamical systems: Koopman operator meets model predictive control,”Auto- matica, vol. 93, pp. 149–160, 2018

    work page 2018

  15. [15]

    Koopman operator dy- namical models: Learning, analysis and control,

    P. Bevanda, S. Sosnowski, and S. Hirche, “Koopman operator dy- namical models: Learning, analysis and control,”Annual Reviews in Control, vol. 52, pp. 197–212, 2021

    work page 2021

  16. [16]

    A practical aerodynamic model for dynamic textile manipulation in robotics,

    F. Coltraro, J. Amor ´os, C. Torras, and M. Alberich-Carrami ˜nana, “A practical aerodynamic model for dynamic textile manipulation in robotics,”Mechanism and Machine Theory, vol. 209, p. 105993, 2025

    work page 2025

  17. [17]

    A quadratic bending model for inextensible surfaces,

    M. Bergou, M. Wardetzky, D. Harmon, D. Zorin, and E. Grinspun, “A quadratic bending model for inextensible surfaces,” inProceedings of the Fourth Eurographics Symposium on Geometry Processing, SGP ’06, pp. 227–230, Eurographics Association, 2006

    work page 2006

  18. [18]

    O. C. Zienkiewicz, R. L. Taylor, and J. Z. Zhu,The Finite Element Method: Its Basis and Fundamentals, Sixth Edition. Butterworth- Heinemann, May 2005

    work page 2005

  19. [19]

    ¨Uber die praktische Aufl ¨osung von Integralgleichun- gen mit Anwendungen auf Randwertaufgaben,

    E. J. Nystr ¨om, “ ¨Uber die praktische Aufl ¨osung von Integralgleichun- gen mit Anwendungen auf Randwertaufgaben,” 1930

    work page 1930

  20. [20]

    Using the Nystr ¨om method to speed up kernel machines,

    C. Williams and M. Seeger, “Using the Nystr ¨om method to speed up kernel machines,”Advances in neural information processing systems, vol. 13, 2000

    work page 2000

  21. [21]

    Tracking cloth deformation: A novel dataset for closing the sim-to- real gap for robotic cloth manipulation learning,

    F. Coltraro, J. Borr `as, M. Alberich-Carrami ˜nana, and C. Torras, “Tracking cloth deformation: A novel dataset for closing the sim-to- real gap for robotic cloth manipulation learning,”The International Journal of Robotics Research, p. 02783649251317617, 2025

    work page 2025

  22. [22]

    Folding deformable objects using predictive simulation and trajectory opti- mization,

    Y . Li, Y . Yue, D. Xu, E. Grinspun, and P. K. Allen, “Folding deformable objects using predictive simulation and trajectory opti- mization,” in2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 6000–6006, IEEE, 2015

    work page 2015

  23. [23]

    Model predictive control for dynamic cloth manipulation: Parameter learning and experimental validation,

    A. Luque, D. Parent, A. Colom ´e, C. Ocampo-Martinez, and C. Torras, “Model predictive control for dynamic cloth manipulation: Parameter learning and experimental validation,”IEEE Transactions on Control Systems Technology, 2024

    work page 2024

  24. [24]

    Quadratic dynamic matrix control for fast cloth manipulation,

    E. Caldarelli, A. Colom ´e, C. Ocampo-Martinez, and C. Torras, “Quadratic dynamic matrix control for fast cloth manipulation,” in 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 8178–8185, IEEE, 2023

    work page 2023