Pith. sign in

REVIEW 3 major objections 5 minor 19 references

This paper extends a touchscreen-based haptic calibration from 2D reaching to tabletop grasping on the NICO humanoid, and shows that a stereo-vision hand-alignment loop outperforms calibration across the full workspace.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 15:39 UTC pith:SSEDQQCT

load-bearing objection A solid, plainly reported extension of haptic calibration from reaching to grasping on NICO; the results are believable but the paper under-specifies where the task-specific corrections come from, and the headline M4 numbers have an asterisk. the 3 major comments →

arxiv 2607.18210 v1 pith:SSEDQQCT submitted 2026-07-20 cs.RO

Optimization of sim-to-real transfer in the humanoid robot NICO

Juraj Gavura , Igor Farka\v{s} This is my paper
classification cs.RO
keywords sim-to-real transferrobotic graspinghaptic calibrationhumanoid robotstereo visionvisual feedbackYOLO object detectiontabletop grasping
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that a calibration model trained on 2D touchscreen reaching can also correct grasp positions for a humanoid robot, and that a simple stereo-vision feedback loop can outperform calibration outside the trained region. On a 22-point tabletop grid, the fully nonlinear calibration model M3 reached 96.7% success inside the neural-network calibration area, while the visual feedback model M4 reached 72.7% across the whole workspace, and 94.1% when stereo hand localization succeeded. The pipeline uses only the robot's built-in low-resolution fisheye cameras, avoiding depth cameras, motion capture, and external tracking. If correct, this means a cheap, explicit calibration plus a simple feedback loop can close much of the sim-to-real gap for grasping.

Core claim

The central claim is that the authors' earlier haptic-calibration models, built for 2D reaching, transfer to tabletop grasping when combined with object detection, stereo-vision localization, and task-specific grasp corrections. The fully nonlinear model M3, which predicts a complete corrected 3D target, succeeds in 29 of 30 attempts inside the calibration area, while the visual feedback model M4, which uses no calibration, reaches 48 of 66 attempts across the full workspace, and 48 of 51 when stereo hand localization is correct. The paper interprets this as evidence that calibration-based correction is strongest where training data exist, while simple closed-loop visual alignment is more ro

What carries the argument

Three correction models map real-world table coordinates to simulated inverse-kinematics targets: M1 is piecewise-linear, M2 predicts horizontal coordinates with a neural network while interpolating height, and M3 predicts all three coordinates with a fully nonlinear neural network. The visual feedback model M4 replaces calibration with an iterative hand-alignment loop that shifts the IK target by p_k = p_{k-1} + (p_object - p_hand)/2^{k-1} until the horizontal distance falls below 1 cm. Task-specific z-height and palm-yaw corrections, manually measured at irregular reference points and interpolated with radial-basis functions, are applied for grasp execution, and a stereo-vision pivot corre

Load-bearing premise

The load-bearing premise is that the manually measured z-height and palm-yaw corrections (Section 3.5) and the stereo pivot correction (Section 3.3) were not tuned on the same 22 test positions used to report success; if they were, the reported success rates partly reflect hand-tuning rather than the calibration or feedback models themselves.

What would settle it

Repeat the M3 and M4 evaluations using a leave-one-out protocol: derive the z/yaw and pivot corrections from reference points that exclude each evaluated grid position, then compare the success rates. If the inside-area success of M3 drops substantially, the reported rates depend on corrections selected with knowledge of the test grid.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Grasping can be improved without depth cameras, motion capture, or external tracking; the robot's built-in fisheye stereo is sufficient.
  • Inside the region where neural-network training data exist, a fully nonlinear correction reaches near-reliable grasping performance.
  • Outside that region, visual feedback beats calibration, showing the two approaches complement each other.
  • Stereo-vision failures in hand localization are the main bottleneck for visual feedback; fixing them would lift overall success from 72.7% toward the conditional 94.1%.
  • The gap between M2 and M3 shows that predicting the full 3D target, including height, matters for grasp success.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A direct head-to-head comparison with domain-randomization methods on the same robot and grid would test whether explicit calibration plus simple feedback is competitive without learning an end-to-end policy; the reported numbers are suggestive but the evaluation setups differ.
  • The observed drift after ten minutes of camera operation suggests the fisheye calibration is time-varying; an online recalibration or a learned drift model could make the manual pivot correction unnecessary.
  • Because the z-height and yaw corrections are task-specific and manually measured, applying the pipeline to other object shapes or sizes would require re-measuring them; an automated measurement procedure could preserve the low-cost advantage.
  • A stereo-depth method with stronger background rejection, such as temporal filtering, would likely convert the 94.1% conditional success of the visual feedback model into a full-workspace rate.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper extends a previous haptic-calibration method for 2D reaching to tabletop grasping on the NICO humanoid robot, using YOLO-based object/hand detection, stereo vision, and task-specific corrections. It compares three calibration models (M1, M2, M3) with a visual-feedback hand-alignment model (M4) over 66 attempts per model on a 22-point grid. The main reported results are that M3 achieves 96.7% success inside the neural-network correction area (29/30) and M4 achieves 72.7% success over the full workspace (48/66), with 94.1% success (48/51) when stereo hand localization is successful.

Significance. If the claims hold, the paper demonstrates that a calibration trained for reaching can transfer to grasping and that a simple stereo-vision hand-alignment loop can outperform calibration outside the trained region—all using only the robot's built-in cameras, without external sensors. The paper is honest about failure modes, explicitly acknowledging M3's poor extrapolation and stereo hand-localization failures, and it reports per-position grid results rather than only aggregate numbers. However, the evidence is statistically thin: 66 attempts per model, no confidence intervals, a post-hoc conditional success rate, and several manually fitted corrections whose selection process is not disclosed. These issues weaken the strength of the central comparative claims.

major comments (3)
  1. [Sec. 4.2, Table 2] The headline comparison (M4 72.7% vs M3 57.6% vs M2 48.5%) is reported without any measure of uncertainty. With 66 attempts per model, a two-proportion test for the M4–M3 difference is not statistically significant (approximate p≈0.07), so the claim that M4 is 'best across the full workspace' is not supported by conventional significance. Provide Wilson confidence intervals, and use a paired test (e.g., McNemar or Wilcoxon signed-rank on per-position success counts) since the same 22 grid positions are used. Adjust the qualitative claims accordingly.
  2. [Sec. 3.3 and Sec. 3.5] The pivot correction (Sec. 3.3) and the RBF-interpolated z and yaw corrections (Sec. 3.5) are manually fitted, but the paper does not specify how the reference points were chosen or whether they overlap the 22 evaluation positions, nor the criterion for selecting the pivot point. If these corrections were tuned using the same test grid, the reported success rates partly reflect hand-tuning rather than the calibration or feedback models. Please report the reference-point coordinates, the pivot-selection procedure, and a sensitivity analysis (e.g., results without the pivot correction, or with a validation split of the reference points).
  3. [Sec. 4.2, Fig. 8] The 'Grasp Rate (SV OK)' of 94.1% (48/51) excludes 15 of 66 attempts (5 positions) where stereo hand localization failed. This is a post-hoc exclusion, and the unconditional full-workspace rate is 72.7%. The paper should clearly label the 94.1% as conditional and should map the failed positions to examine whether they cluster in a particular workspace region or with particular background depths. As it stands, the robust full-workspace claim rests on 72.7%, which itself is not shown to be statistically distinct from M3's 57.6% (see first comment).
minor comments (5)
  1. [Sec. 4.1, Fig. 7] The text states mean displacement of 2.14 cm inside and 5.45 cm outside the NN area, while the figure caption reports 2.13 cm and 5.35 cm. Please reconcile the numbers.
  2. [Sec. 4.1] M2 is only reported in Table 2; there is no per-grid visualization or discussion of why it underperforms M3. A sentence or two would help the reader verify the consistency with reaching results.
  3. [Sec. 3.5] The pipeline description does not explicitly specify whether M4 also uses the z and yaw corrections. If it does, that should be stated; if not, the description of the common lower/close phase is ambiguous.
  4. [Sec. 4.2, Fig. 8] The figure lists 'Hand V OK' and 'Grasp Rate ( V OK)' without defining what 'V OK' stands for; the text uses 'SV OK'. Please standardize the terminology.
  5. [General] No confidence intervals are reported anywhere; even for the key 29/30 and 48/51 results, Wilson intervals would be a useful addition.

Circularity Check

0 steps flagged

No significant circularity: the grasping results are independent transfer experiments and the cited prior calibration models are not retrained on grasp outcomes.

full rationale

The core claims of the paper are empirical transfer results rather than derivations from fitted quantities. The calibration models M1–M3 are taken from prior work [6] and explicitly not retrained for grasping (“In this paper, the models are not retrained, but are used as correction modules within the grasping pipeline”), so the grasp success rates are not fitted to the reported test outcomes. The z/yaw and stereo pivot corrections are task-specific calibrations, but the paper does not state that their reference points coincide with the 22 evaluated grid positions, and the success metric (object grasping) was not used to select those corrections; without evidence of overlap, this is a possible overfitting/leakage concern rather than a definitional circularity. The M3 in-area result (96.7%) is measured inside the region where training data existed, but the outcome variable is grasping success, which was not part of the model’s training target, so it is an in-domain evaluation, not a prediction of the model’s own training labels. Self-citations to [6] are references to prior methodology and are not invoked to forbid alternatives or to define the target result. The visual feedback law is explicitly stated and is a straightforward servoing update, not a renamed known result. No step in the paper reduces by construction to its own inputs.

Axiom & Free-Parameter Ledger

6 free parameters · 6 axioms · 0 invented entities

The central claim rests on several fitted quantities: the pre-existing NN calibration models, hand-measured z/yaw corrections, and a pivot-based stereo drift correction. These are honest engineering calibrations, but they mean the reported grasp rates are not parameter-free predictions.

free parameters (6)
  • M1-M3 calibration model parameters = not reported (from [6])
    Piecewise-linear mapping, MLP weights, and interpolation tables trained on touchscreen-reaching data in prior work; used unchanged as correction modules and determine grasp behavior.
  • Vertical z correction values = manual measurements at reference points (values not listed)
    Hand-measured z offsets for the palm at reference points, RBF-interpolated across workspace; directly affects grasp success.
  • Palm yaw correction values = manual measurements at reference points (values not listed)
    Hand-measured wrist yaw angles for top-down grasp; RBF-interpolated; used for all calibration models.
  • Stereo pivot correction = pivot point and shift magnitude chosen by observation (not quantified)
    Compensates localization drift by moving predicted positions toward a fixed pivot; fitted to observed drift.
  • Visual feedback threshold and step schedule = 1 cm; a_k=1/(2k-1)
    Hand-set stopping condition and shrinking step size; not fitted but design choices that affect success.
  • YOLO model weights (YOLO12s) = trained on 442 custom images
    Detection of object/hands; mAP reported, but weights not released.
axioms (6)
  • domain assumption The robot's built-in fisheye cameras, after calibration and pivot correction, give 3D positions accurate enough for grasping (mean 0.76 cm on a 9x4 grid).
    Section 3.3: stereo localization is used without external ground truth; the only pseudo-ground-truth is a paper grid.
  • domain assumption Correction models trained for 2D reaching transfer to 3D grasping without retraining.
    Section 3.2: models are used as-is; the paper tests this transfer.
  • domain assumption RBF interpolation of manually measured z/yaw corrections generalizes across the workspace.
    Section 3.5: arbitrary targets receive interpolated z/yaw values.
  • domain assumption A single plush tomato is a representative grasp object for evaluating the pipeline.
    Section 3.3: only the tomato is tested; claims of 'object grasping' are made from one object.
  • domain assumption PyBullet IK with the modified URDF and NicoMotion execution is a faithful enough sim-to-real substrate.
    Section 3.1: this is the control backbone; sim-to-real gap is the thing being calibrated.
  • ad hoc to paper The localization drift can be compensated by moving predicted positions toward a fixed pivot point.
    Section 3.3; introduced specifically to fix observed drift, no independent validation.

pith-pipeline@v1.3.0-alltime-deepseek · 8675 in / 13295 out tokens · 106494 ms · 2026-08-01T15:39:27.607807+00:00 · methodology

0 comments
read the original abstract

Robotic grasping requires accurate coordination between visual perception, object localization, inverse kinematics, and hand control. However, when movements planned in simulation are executed on a physical robot, the sim-to-real gap can cause small positioning errors that prevent successful grasping. In our previous work, we introduced a low-cost haptic calibration method that improved 2D reaching accuracy of the humanoid robot NICO. In this paper, we extend this approach from reaching to tabletop object grasping by adding YOLO-based object and hand detection, stereo vision-based localization using the robot's built-in low-resolution fisheye cameras, and task-specific corrections for grasp execution. Together, these components form a novel calibration-based grasping pipeline that does not require RGB-D cameras, motion capture, or external tracking systems. We also implemented a visual feedback model that aligns the robot hand with the detected object before grasping. Our results show that the fully nonlinear calibration model achieved the best performance inside the calibrated area, while the visual feedback model achieved the highest overall grasping success across the full tabletop workspace.

Figures

Figures reproduced from arXiv: 2607.18210 by Igor Farka\v{s}, Juraj Gavura.

Figure 1
Figure 1. Figure 1: NICO robot setup equipped with a paper grid used as ground truth for object placement in the grasping experiments. The experiments were performed with the humanoid robot NICO, using its right arm and four-finger child-sized hand. A paper grid ( [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Example of object detection results using the YOLO12s model. Left: the original image; Middle: ground truth annotations from the dataset; Right: predictions by the trained model with confidence scores [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Evaluation of the corrected stereovision localization method on a 9×4 grid of target positions. The unfocused version uses the initial head pose, where NICO looks toward the center of the table. Green squares indicate target positions, blue points show the estimated positions, and the values inside the green squares represent the estimated z coordinate in centimeters. The pivot point indicates the directio… view at source ↗
Figure 3
Figure 3. Figure 3: Example of a stereo-vision calibra￾tion image with detected chessboard cor￾ners and their projected positions. Using a fixed head position, where the robot kept its initial head pose during inference, the final stereo vi￾sion module achieved a mean 2D lo￾calization error 0.76 cm. Only a few outliers reached errors >2 cm, mostly at edge positions, many of which were outside the reachable area of the robot a… view at source ↗
Figure 5
Figure 5. Figure 5: Grasping pipeline of our approach. 4 Results 4.1 Calibration-based grasping We first evaluated the grasping performance of the calibration-based models to test whether the calibration also reduces the reality gap in the grasping task. The baseline model M1 is a purely linear model without additional correction in the horizontal x and y axes. As shown in the upper half of [PITH_FULL_IMAGE:figures/full_fig_… view at source ↗
Figure 6
Figure 6. Figure 6: Visualization of 2D grasping success rate for grid-based targets spread across the whole reachable area using the baseline model M1 (top) and NN correction model M3 (bottom). The plot shows the number of successful grasp attempts for each target position, together with the touchscreen area, NN correction area, and NICO arm reach. The box "NICO" denotes the robot position (to be shifted 10 cm up). 4.2 Visua… view at source ↗
Figure 7
Figure 7. Figure 7: Visualization of predictions produced by M3 for input points distributed across the whole table. Red points represent input target positions and blue points show the predicted positions. The darker rectangle marks the region covered by training data used for the NN, whereas points outside this region require extrapolation. The box "NICO" denotes the robot position. −  −  −  −       #&7-4(/ −  … view at source ↗
Figure 8
Figure 8. Figure 8: Visualization of 2D grasping success rate for grid-based targets spread across the whole reachable area using the visual feedback hand alignment model. The plot shows the number of successful grasp attempts for each target position. Gray targets indicate positions where stereo vision substantially overestimated the hand position. However, the performance of this model was limited by the reliability of ster… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

19 extracted references · 2 linked inside Pith

  1. [1]

    http://pybullet.org (2016–2021)

    Coumans, E., Bai, Y.: PyBullet, a python module for physics simulation for games, robotics and machine learning. http://pybullet.org (2016–2021)

    2016

  2. [2]

    Artificial Intelligence Review 54(3), 1677–1734 (2021)

    Du, G., Wang, K., Lian, S., Zhao, K.: Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: A review. Artificial Intelligence Review 54(3), 1677–1734 (2021)

    2021

  3. [3]

    In: IEEE/CVF CVPR (2020)

    Fang, H.S., Wang, C., Gou, M., Lu, C.: Graspnet-1billion: A large-scale benchmark for general object grasping. In: IEEE/CVF CVPR (2020)

    2020

  4. [4]

    In: IJCNN (2024)

    Gäde, C., Habekost, J.G., Wermter, S.: Domain adaptation as auxiliary task for sim-to-real transfer in vision-based neuro-robotic control. In: IJCNN (2024)

    2024

  5. [5]

    In: ICANN (2022)

    Gäde, C., Kerzel, M., Strahl, E., Wermter, S.: Sim-to-real neural learning with domain randomisation for humanoid robot grasping. In: ICANN (2022)

    2022

  6. [6]

    In: ICANN (2025)

    Gavura, J., Vavrečka, M., Farkaš, I., Gäde, C.: Robotic calibration based on haptic feedback improves sim-to-real transfer. In: ICANN (2025)

    2025

  7. [7]

    Haviland, J., Dayoub, F., Corke, P.: Control of the final-phase of closed-loop visual grasping using image-based visual servoing (2020), arXiv:2001.05650

    Pith/arXiv arXiv 2020

  8. [8]

    In: CVPR (2019)

    James, S., et al.: Sim-to-real via sim-to-sim: Data-efficient robotic grasping via randomized-to-canonical adaptation networks. In: CVPR (2019)

    2019

  9. [9]

    Electronics 12(18), 3981 (2023)

    Jin, Y., et al.: Target localization and grasping of NAO robot based on yolov8 network and monocular ranging. Electronics 12(18), 3981 (2023)

    2023

  10. [10]

    In: IEEE RO-MAN

    Kerzel, M., et al.: NICO - Neuro-Inspired COmpanion: A developmental humanoid robot platform for multimodal interaction. In: IEEE RO-MAN. pp. 113–120 (2017)

    2017

  11. [11]

    Current Robotics Reports 1(4), 239–249 (2020)

    Kleeberger, K., Bormann, R., Kraus, W., Huber, M.F.: A survey on learning-based robotic grasping. Current Robotics Reports 1(4), 239–249 (2020)

    2020

  12. [12]

    Science Robotics 4(26) (2019)

    Mahler, J., et al.: Learning ambidextrous robot grasping policies. Science Robotics 4(26) (2019)

    2019

  13. [13]

    In: Robotics: Science and Systems (2018)

    Morrison, D., Leitner, J., Corke, P.: Closing the loop for robotic grasping: A real-time, generative grasp synthesis approach. In: Robotics: Science and Systems (2018)

    2018

  14. [14]

    In: IEEE ICRA

    Piacenza, P., Yuan, J., Huh, J., Isler, V.: VF AS-grasp: Closed loop grasping with visual feedback and adaptive sampling. In: IEEE ICRA. pp. 4126–4132 (2024)

    2024

  15. [15]

    Rameshbabu, B., et al.: A benchmarking study of vision-based robotic grasping algorithms: A comparative analysis (2025), arXiv:2307.11622

    arXiv 2025

  16. [16]

    Sensors 24(1) (2024)

    Sun, R., Wu, C., Zhao, X., Zhao, B., Jiang, Y.: Object recognition and grasping for collaborative robots based on vision. Sensors 24(1) (2024)

    2024

  17. [17]

    Tian, Y., Ye, Q., Doermann, D.S.: Yolov12: Attention-centric real-time object de- tectors (2025), arXiv:2502.12524

    Pith/arXiv arXiv 2025

  18. [18]

    In: IEEE-RAS Int

    Vahrenkamp, N., et al.: Visual servoing for humanoid grasping and manipulation tasks. In: IEEE-RAS Int. Conf. on Humanoid Robots. pp. 406–412 (2008)

    2008

  19. [19]

    In: ICRA (2024)

    Vuong, A.D., et al.: Grasp-anything: Large-scale grasp dataset from foundation models. In: ICRA (2024)

    2024