pith. sign in

arxiv: 2605.23847 · v1 · pith:GR3VT2DJnew · submitted 2026-05-22 · 💻 cs.RO

Instrumentation for Imitation Learning: Enhancing Training Datasets for Clothes Hanger Insertion

Remko Proesmans , Thomas Lips , Francis wyffels This is my paper

Pith reviewed 2026-05-25 03:51 UTC · model grok-4.3

classification 💻 cs.RO
keywords imitation learninginstrumentationrobotic manipulationdiffusion policyclothes hanger insertionteleoperationstate information
0
0 comments X

The pith

Instrumenting objects with sensors lets imitation learning policies for hanger insertion outperform vision-only versions by 14-25 points.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper shows that adding sensors to objects supplies state information vision cannot reliably recover, enabling more effective imitation learning for robotic manipulation with limited demonstrations. Policies trained on this instrumentation data achieve 14-25 percentage point higher success rates than vision-only policies and display clearer task awareness. Black-box policies learn to prioritize the sensor signals without explicit instruction. Augmenting the original teleoperated dataset with rollouts from an instrumented expert policy then allows a purely vision-based student policy to reach the performance level of the instrumented expert.

Core claim

Using 180 teleoperated demonstrations, diffusion policies with access to instrumentation data outperform vision-only counterparts by 14-25 percentage points and exhibit greater task awareness. A black-box imitation learning policy learns to prioritise instrumentation signals without explicit guidance. Enhancing the teleoperation dataset with rollouts from an instrumented expert policy enables a vision-only student policy to achieve performance comparable to the instrumented expert, thereby surpassing the original vision-only policy.

What carries the argument

Instrumentation, defined as sensor integration in objects, that supplies direct state information for the insertion task.

If this is right

  • Black-box policies can learn to prioritize instrumentation signals without any explicit guidance during training.
  • Vision-only policies reach performance comparable to instrumented experts after training on datasets that include rollouts from instrumented experts.
  • Instrumentation produces policies with measurably greater task awareness during manipulation.
  • The approach improves success rates by 14-25 percentage points over standard vision-only imitation learning on the same base dataset.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same instrumentation strategy could reduce data requirements for other contact-rich insertion or assembly tasks where vision is occluded or ambiguous.
  • Object sensors used only during data collection could serve as a temporary scaffold to bootstrap stronger vision-only policies for deployment.
  • Minimal sensor suites on common objects might generalize across multiple manipulation tasks without per-task redesign.

Load-bearing premise

The added instrumentation supplies state information that cannot be reliably recovered from vision alone and that the 180 teleoperated demonstrations plus generated rollouts form a representative training distribution.

What would settle it

A vision-only policy trained on the augmented dataset from instrumented rollouts failing to match the instrumented expert's success rate would falsify the dataset-enhancement claim.

Figures

Figures reproduced from arXiv: 2605.23847 by Francis wyffels, Remko Proesmans, Thomas Lips.

Figure 1
Figure 1. Figure 1: Instrumented clothes hanger insertion. distribution toward states where the policy is already profi￾cient [12]. Alternatively, in interactive imitation learning [13] expert interventions can get a struggling policy rollout back on track [14]–[16]. This allows for deliberate data collection to efficiently overcome data mismatches between the train data and deployment settings [15]. In addition to larger dat… view at source ↗
Figure 2
Figure 2. Figure 2: Instrumented clothes hanger with integrated infrared [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Visualisation of each test set. The curves are traces [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: End-to-end success rates for diffusion policies trained [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Policy errors and behaviours related to task awareness. [PITH_FULL_IMAGE:figures/full_fig_p005_6.png] view at source ↗
read the original abstract

Large behaviour models have transformed the field of robotic manipulation, but prohibitive data requirements have thus far prevented a revolution similar to vision language models. We believe that instrumentation, i.e. sensor integration in objects, can provide invaluable state information and enable efficient learning for robotic manipulation. In this paper, we present instrumented imitation learning of clothes hanger insertion. Using 180 teleoperated demonstrations, we train diffusion policies with and without access to instrumentation data. Results show that policies leveraging instrumentation outperform vision-only counterparts by 14-25 %pt and exhibit greater task awareness. Crucially, a black-box imitation learning policy learns to prioritise instrumentation signals without explicit guidance. In addition, enhancing the teleoperation dataset with rollouts from an instrumented expert policy, enables a vision-only student policy to achieve performance comparable to the instrumented expert, thereby surpassing the original vision-only policy. These findings establish instrumentation as a promising strategy to enhance imitation learning for robotic manipulation. Datasets are available on Zenodo.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The paper claims that instrumenting objects with sensors supplies state information that improves imitation learning for robotic clothes hanger insertion. Using 180 teleoperated demonstrations, diffusion policies with access to instrumentation data outperform vision-only policies by 14-25 percentage points and show greater task awareness. A black-box policy learns to prioritize instrumentation signals without explicit supervision. Augmenting the dataset with rollouts from an instrumented expert policy allows a vision-only student policy to match the instrumented expert's performance, surpassing the original vision-only baseline. Datasets are released on Zenodo.

Significance. If the empirical gains hold under scrutiny, the work demonstrates a practical route to more data-efficient imitation learning for manipulation by leveraging direct state signals from instrumentation. The public dataset release on Zenodo supports reproducibility and further research. The finding that policies can implicitly learn to use these signals is noteworthy, though the significance depends on confirming that the performance delta arises from information not recoverable from vision alone.

major comments (3)
  1. [Results] Results (performance comparison): the 14-25 percentage point gains are reported without accompanying statistical significance tests, run-to-run variance, or details on exact sensor placement and failure mode analysis, making it impossible to verify that the outperformance is robustly attributable to instrumentation rather than other experimental factors.
  2. [Abstract / Results] Abstract and results discussion: the central attribution that instrumentation supplies state information (e.g., contact or pose) unrecoverable from vision lacks a controlled ablation comparing instrumented signals against standard vision-based recovery methods such as pose estimation or depth tracking on the same visual observations.
  3. [Methods / Evaluation] Dataset and evaluation: the 180 teleoperated demonstrations plus generated rollouts are treated as representative without reported coverage analysis, out-of-distribution testing, or characterization of the insertion task distribution, weakening the claim that the student policy generalizes comparably.
minor comments (2)
  1. [Methods] Clarify in the methods how the diffusion policy architecture ingests the mixed instrumentation and vision inputs (e.g., concatenation details or separate encoders).
  2. [Results] The abstract states policies 'exhibit greater task awareness'—provide a concrete metric or qualitative example in the results to support this.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for their constructive feedback. We address each major comment below, indicating where revisions will be made to strengthen the manuscript.

read point-by-point responses

  1. Referee: [Results] Results (performance comparison): the 14-25 percentage point gains are reported without accompanying statistical significance tests, run-to-run variance, or details on exact sensor placement and failure mode analysis, making it impossible to verify that the outperformance is robustly attributable to instrumentation rather than other experimental factors.

    Authors: We agree that reporting run-to-run variance and statistical tests would improve verifiability. In the revised manuscript we will add performance statistics across multiple training seeds (with standard deviations) and apply paired t-tests to the success rate differences. Exact sensor placement diagrams and a failure mode breakdown (e.g., contact loss vs. misalignment) will be included in the methods and results sections. These additions directly address the concern that gains may stem from uncontrolled factors. revision: yes

  2. Referee: [Abstract / Results] Abstract and results discussion: the central attribution that instrumentation supplies state information (e.g., contact or pose) unrecoverable from vision lacks a controlled ablation comparing instrumented signals against standard vision-based recovery methods such as pose estimation or depth tracking on the same visual observations.

    Authors: We acknowledge the absence of an explicit ablation against pose estimation or depth tracking. Our central evidence instead rests on the student-teacher result: a vision-only policy trained on instrumented-expert rollouts matches the instrumented expert while the original vision-only baseline does not. This indicates that the instrumentation signals contain task-relevant information not present in the original visual demonstrations. Adding a full pose-estimation baseline would require new perception pipelines and is beyond the scope of the current study; we will, however, expand the discussion to clarify this distinction and note the limitation. revision: partial

  3. Referee: [Methods / Evaluation] Dataset and evaluation: the 180 teleoperated demonstrations plus generated rollouts are treated as representative without reported coverage analysis, out-of-distribution testing, or characterization of the insertion task distribution, weakening the claim that the student policy generalizes comparably.

    Authors: We will add a quantitative characterization of the demonstrated task distribution (hanger pose ranges, insertion angles) and a coverage analysis of the 180 trajectories in the revised methods section. The evaluation protocol already includes randomized initial conditions drawn from the same distribution; we will explicitly label these as in-distribution and discuss the lack of dedicated out-of-distribution testing as a limitation. These changes will better support the generalization claim for the student policy. revision: yes

Circularity Check

0 steps flagged

No circularity: purely empirical comparisons with no derivations or self-referential fits

full rationale

The paper reports direct empirical results from training diffusion policies on 180 teleoperated demonstrations, comparing instrumented vs. vision-only conditions and testing dataset augmentation via rollouts. No equations, fitted parameters, predictions derived from inputs, or load-bearing self-citations appear in the provided text. All performance claims (14-25 %pt gains, prioritization of signals) rest on measured outcomes rather than any reduction to definitions or prior author work. This matches the default expectation of a non-circular empirical study.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The abstract introduces no mathematical derivations, free parameters, or new entities; it applies existing imitation learning methods with added sensor data under standard robotics assumptions about demonstration quality and task repeatability.

pith-pipeline@v0.9.0 · 5701 in / 1120 out tokens · 25771 ms · 2026-05-25T03:51:49.375387+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

37 extracted references · 37 canonical work pages · 6 internal anchors

  1. [1]

    A roadmap for ai in robotics,

    A. Billard,et al., “A roadmap for ai in robotics,”Nature Machine Intelligence, vol. 7, no. 6, p. 818–824, June 2025. [Online]. Available: http://dx.doi.org/10.1038/s42256-025-01050-6

    work page doi:10.1038/s42256-025-01050-6 2025

  2. [2]

    Good old-fashioned engineering can close the 100,000- year “data gap

    K. Goldberg, “Good old-fashioned engineering can close the 100,000- year “data gap” in robotics,”Science Robotics, vol. 10, no. 105, p. eaea7390, 2025. [Online]. Available: https://www.science.org/doi/abs/ 10.1126/scirobotics.aea7390

    work page doi:10.1126/scirobotics.aea7390 2025

  3. [3]

    A Careful Examination of Large Behavior Models for Multitask Dexterous Manipulation

    J. Barreiros,et al., “A careful examination of large behavior models for multitask dexterous manipulation,”arXiv preprint arXiv:2507.05331, 2025

    work page internal anchor Pith review arXiv 2025

  4. [4]

    A survey of robot learning from demonstration,

    B. D. Argall, S. Chernova, M. Veloso, and B. Browning, “A survey of robot learning from demonstration,”Robotics and Autonomous Systems, vol. 57, no. 5, pp. 469–483, 2009. [Online]. Available: https: //www.sciencedirect.com/science/article/pii/S0921889008001772

    work page 2009

  5. [5]

    Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware,

    T. Z. Zhao, V . Kumar, S. Levine, and C. Finn, “Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware,” inProceedings of Robotics: Science and Systems, Daegu, Republic of Korea, July 2023

    work page 2023

  6. [7]

    $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control

    K. Black,et al., “π 0: A vision-language-action flow model for general robot control,” 2024. [Online]. Available: https: //arxiv.org/abs/2410.24164

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  7. [8]

    OpenVLA: An Open-Source Vision-Language-Action Model

    M. J. Kim,et al., “Openvla: An open-source vision-language-action model,”2024 Conference on Robot Learning, vol. abs/2406.09246,

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  8. [9]

    Available: https://api.semanticscholar.org/CorpusID: 270440391

    [Online]. Available: https://api.semanticscholar.org/CorpusID: 270440391

    work page

  9. [10]

    Open X-Embodiment: Robotic Learning Datasets and RT-X Models

    E. Collaboration,et al., “Open x-embodiment: Robotic learning datasets and rt-x models,” 2024. [Online]. Available: https://arxiv.org/ abs/2310.08864

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  10. [11]

    DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset

    A. Khazatsky,et al., “Droid: A large-scale in-the-wild robot manipulation dataset,” inProceedings of Robotics: Science and Systems, 2024. [Online]. Available: https://arxiv.org/abs/2403.12945

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  11. [12]

    Bridgedata v2: A dataset for robot learning at scale,

    H. Walke,et al., “Bridgedata v2: A dataset for robot learning at scale,” 2024. [Online]. Available: https://arxiv.org/abs/2308.12952

    work page arXiv 2024

  12. [13]

    So you think you can scale up autonomous robot data collection?

    S. Mirchandani, S. Belkhale, J. Hejna, E. Choi, M. S. Islam, and D. Sadigh, “So you think you can scale up autonomous robot data collection?” 2024. [Online]. Available: https://arxiv.org/abs/2411. 01813

    work page 2024

  13. [14]

    Interactive imitation learning in robotics: A survey,

    C. Celemin, R. P ´erez-Dattari, E. Chisari, G. Franzese, L. de Souza Rosa, R. Prakash, Z. Ajanovi ´c, M. Ferraz, A. Valada, and J. Kober, “Interactive imitation learning in robotics: A survey,”

    work page

  14. [15]

    Available: https://arxiv.org/abs/2211.00600

    [Online]. Available: https://arxiv.org/abs/2211.00600

    work page arXiv

  15. [16]

    Real-time operator takeover for visuomotor diffusion policy training,

    N. Ingelhag, J. Munkeby, M. C. Welle, M. Moletta, and D. Kragic, “Real-time operator takeover for visuomotor diffusion policy training,”

    work page

  16. [17]

    Available: https://arxiv.org/abs/2502.02308

    [Online]. Available: https://arxiv.org/abs/2502.02308

    work page arXiv

  17. [18]

    HG-DAgger: Interactive Imitation Learning with Human Experts

    M. Kelly, C. Sidrane, K. Driggs-Campbell, and M. J. Kochenderfer, “Hg-dagger: Interactive imitation learning with human experts,” 2019. [Online]. Available: https://arxiv.org/abs/1810.02890

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  18. [19]

    Racer: Rich language-guided failure recovery policies for imitation learning,

    Y . Dai, J. Lee, N. Fazeli, and J. Chai, “Racer: Rich language-guided failure recovery policies for imitation learning,” 2024. [Online]. Available: https://arxiv.org/abs/2409.14674

    work page arXiv 2024

  19. [20]

    Quantifying demonstration quality for robot learning and generalization,

    M. Sakr, Z. J. Li, H. F. M. Van der Loos, D. Kuli ´c, and E. A. Croft, “Quantifying demonstration quality for robot learning and generalization,”IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 9659–9666, 2022

    work page 2022

  20. [21]

    Beyond success: Quantifying demonstration quality in learning from demonstration,

    M. Bilal, N. Lipovetzky, D. Oetomo, and W. Johal, “Beyond success: Quantifying demonstration quality in learning from demonstration,” in2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024, pp. 5120–5127

    work page 2024

  21. [22]

    Rldg: Robotic generalist policy distillation via reinforcement learning,

    C. Xu, Q. Li, J. Luo, and S. Levine, “Rldg: Robotic generalist policy distillation via reinforcement learning,” 2024

    work page 2024

  22. [23]

    Lab2field transfer of a robotic raspberry harvester enabled by a soft sensorized physical twin,

    K. Junge, C. Pires, and J. Hughes, “Lab2field transfer of a robotic raspberry harvester enabled by a soft sensorized physical twin,” Communications Engineering, vol. 2, no. 1, p. 40, Jun 2023

    work page 2023

  23. [24]

    Solving rubik’s cube with a robot hand,

    OpenAI,et al., “Solving rubik’s cube with a robot hand,” 2019

    work page 2019

  24. [25]

    Simpler learning of robotic manipulation of clothing by utilizing diy smart textile technology,

    A. Verleysen, T. Holvoet, R. Proesmans, C. Den Haese, and F. wyffels, “Simpler learning of robotic manipulation of clothing by utilizing diy smart textile technology,”Applied Sciences, vol. 10, no. 12, 2020. [Online]. Available: https://www.mdpi.com/2076-3417/10/12/4088

    work page 2020

  25. [26]

    Modular piezoresistive smart textile for state estima- tion of cloths,

    R. Proesmans, A. Verleysen, R. Vleugels, P. Veske, V .-L. De Gusseme, and F. wyffels, “Modular piezoresistive smart textile for state estima- tion of cloths,”Sensors, vol. 22, no. 1, 2022

    work page 2022

  26. [27]

    Learning quadrupedal locomotion over challenging terrain,

    J. Lee, J. Hwangbo, L. Wellhausen, V . Koltun, and M. Hutter, “Learning quadrupedal locomotion over challenging terrain,”Science Robotics, vol. 5, no. 47, Oct. 2020. [Online]. Available: http: //dx.doi.org/10.1126/scirobotics.abc5986

    work page doi:10.1126/scirobotics.abc5986 2020

  27. [28]

    Visual dexterity: In-hand reorientation of novel and complex object shapes,

    T. Chen, M. Tippur, S. Wu, V . Kumar, E. Adelson, and P. Agrawal, “Visual dexterity: In-hand reorientation of novel and complex object shapes,”Science Robotics, vol. 8, no. 84, p. eadc9244,

    work page

  28. [29]

    Available: https://www.science.org/doi/abs/10.1126/ scirobotics.adc9244

    [Online]. Available: https://www.science.org/doi/abs/10.1126/ scirobotics.adc9244

    work page

  29. [30]

    Unfolding the literature: A review of robotic cloth manipulation,

    A. Longhini,et al., “Unfolding the literature: A review of robotic cloth manipulation,”Annual Review of Control, Robotics, and Autonomous Systems, vol. 8, 2025

    work page 2025

  30. [31]

    Robohanger: Learning generalizable robotic hanger insertion for diverse garments,

    Y . Chen, S. Wei, B. Xiao, J. Lyu, J. Chen, F. Zhu, and H. Wang, “Robohanger: Learning generalizable robotic hanger insertion for diverse garments,”IEEE Robotics and Automation Letters, vol. 10, no. 9, pp. 8922–8929, 2025

    work page 2025

  31. [32]

    Hanging work of t-shirt in consideration of deformability and stretchability,

    Y . Koishihara, S. Arnold, K. Yamazaki, and T. Matsubara, “Hanging work of t-shirt in consideration of deformability and stretchability,” in 2017 IEEE International Conference on Information and Automation (ICIA), 2017, pp. 130–135

    work page 2017

  32. [33]

    Gello: A general, low- cost, and intuitive teleoperation framework for robot manipulators,

    P. Wu, Y . Shentu, Z. Yi, X. Lin, and P. Abbeel, “Gello: A general, low- cost, and intuitive teleoperation framework for robot manipulators,” 2024

    work page 2024

  33. [34]

    Diffusion policy: Visuomotor policy learning via action diffusion,

    C. Chi, Z. Xu, S. Feng, E. Cousineau, Y . Du, B. Burchfiel, R. Tedrake, and S. Song, “Diffusion policy: Visuomotor policy learning via action diffusion,”The International Journal of Robotics Research, 2024

    work page 2024

  34. [35]

    Robot learning as an empirical science: Best practices for policy evaluation,

    H. Kress-Gazit, K. Hashimoto, N. Kuppuswamy, P. Shah, P. Horgan, G. Richardson, S. Feng, and B. Burchfiel, “Robot learning as an empirical science: Best practices for policy evaluation,” 2024. [Online]. Available: https://arxiv.org/abs/2409.09491

    work page arXiv 2024

  35. [36]

    Instrumentation for better demonstrations: A case study,

    R. Proesmans, T. Lips, and F. wyffels, “Instrumentation for better demonstrations: A case study,” 2025. [Online]. Available: https://arxiv.org/abs/2504.18481

    work page arXiv 2025

  36. [37]

    The role of artificial intelligence-driven soft sensors in advanced sustainable process industries: A critical review,

    Y . S. Perera, D. Ratnaweera, C. H. Dasanayaka, and C. Abeykoon, “The role of artificial intelligence-driven soft sensors in advanced sustainable process industries: A critical review,”Engineering Applications of Artificial Intelligence, vol. 121, p. 105988, 2023. [Online]. Available: https://www.sciencedirect.com/science/article/pii/ S0952197623001720

    work page 2023

  37. [38]

    Vital pretraining: Visuo-tactile pretraining for tactile and non- tactile manipulation policies,

    A. George, S. Gano, P. Katragadda, and A. B. Farimani, “Vital pretraining: Visuo-tactile pretraining for tactile and non- tactile manipulation policies,” 2024. [Online]. Available: https: //arxiv.org/abs/2403.11898

    work page arXiv 2024