Hand-Object Contact Detection using Grasp Quality Metrics

Akansel Cosgun; Thanh Vinh Nguyen

REVIEW 3 major objections 5 minor 13 references

Hand-Object Contact Detection using Grasp Quality Metrics

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Under ground-truth poses, simulator grasp-quality scores detect hand-object contact in 89.3% of frames.

desk verdict A small, honest proof-of-concept: GraspIt's built-in contact state is used as a contact detector, and it agrees with manual stable-grasp labels ~89% of the time, but the evaluation lacks a baseline and the labels are not contact labels. read the letter →

arxiv 2501.06987 v2 pith:6KMCRQNP submitted 2025-01-13 cs.RO

Thanh Vinh Nguyen , Akansel Cosgun This is my paper · ORCID

classification cs.RO

keywords contactdetectiongraspqualitymetricsFerrari-Cannymeasuresscenereconstructionrobot-to-humanhandoverMANOhandmodelDexYCB

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Under idealized conditions, with ground-truth hand and object poses fed directly into a grasp simulator, this paper tries to show that hand-object contact can be detected from grasp quality scores rather than from force or tactile sensors. Reconstructing the interaction scene and reading two Ferrari-Canny stability measures, the system declares contact whenever either measure rises above its no-contact baseline. Compared with manually annotated stable-grasp frames from a human-object interaction benchmark, this rule matches the human label in 89.3% of frames, with a 15.7% false-positive rate and an 8.0% false-negative rate over 22,659 frames. The authors argue this matters for robot-to-human handover, where deciding when a person has actually taken an object is safety-critical and force-based methods can release too early if the hand merely touches the object. It would also give a way to test whether a hand pose is graspable at all.

What carries the argument

The load-bearing object is the pair of Ferrari-Canny grasp quality measures $\epsilon$ and $v$ computed by the GraspIt! simulator: $\epsilon$ is the worst-case grasp stability and $v$ is the average-case stability. The simulator returns sentinel values $\epsilon = -1.0$ and $v = 0.0$ when there is no contact, and at least one of the two becomes positive when contact is made, which the paper turns into the decision rule contact $\Longleftrightarrow$ ($\epsilon > 0$ or $v > 0$). The hand is represented with a MANO mesh (shape vector $\beta$ and pose vector $\theta$), converted into a joint-based kinematic model in the simulator, and the object's VRML model is placed according to its ground-truth pose. The machinery converts arbitrary hand and object poses into scalar stability signals without any learned classifier.

What would settle it

Run the system on frames with synchronized tactile or force ground truth instead of visual annotation: if a significant number of frames with physical hand-object contact but non-force-closure hand poses have $\epsilon = -1.0$ and $v = 0.0$, then the method detects grasps rather than contact, and if many no-contact near-miss poses yield $\epsilon > 0$ or $v > 0$, then the simulator's contact model overreports; either outcome would refute the claim that the metric criterion measures hand-object contact.

Watch

Extended reading notes

Core claim

The paper's central claim, stated in Section III.A, is that the system correctly detects hand-object contact in 89.3% of evaluated frames. The detection criterion is exactly that the simulator reports no-contact sentinel values ($\epsilon = -1.0$ and $v = 0.0$) when there is no contact, and contact is asserted when $\epsilon > 0$ or $v > 0$. Against manual labels of when a subject appears to achieve a stable grasp, this criterion yields object-wise accuracy from 83.9% (a bowl) to 93.2% (a pudding box) and subject-wise accuracy from 85.1% to 91.1%. The two grasp quality measures have overlapping distributions for grasped and non-grasped frames, so they are not cleanly separable by a threshold, but higher values are slightly more common in stable-grasp frames and both measures jump when the ground-truth state transitions into grasping.

Load-bearing premise

The load-bearing premise is that the manually annotated labels, marking frames where a subject appeared to achieve a stable grasp, are reliable enough to serve as ground truth; the paper does not describe the annotation protocol or inter-annotator agreement and itself attributes lower accuracy for two subjects to annotation errors.

Editorial extensions

If this is right

In a robot-to-human handover, object release can be triggered by the simulator's contact signal instead of by tactile sensors, potentially avoiding premature release when the human touches but does not securely grasp the object.
The same pipeline can flag whether a hand is currently in a graspable pose, extending the output from a binary contact label to a continuous stability estimate.
Because hand shape parameters are person-specific but estimable once per subject, a real-time vision-based version is plausible using a single RGB image for hand reconstruction and object pose estimation.
Accuracy depends on object geometry: small objects that are easily grasped on their sides, such as a pudding box, are detected best, while a bowl that can be pinched or cupped is detected worst, so the method is most trustworthy for power-grasped objects.
False positives correlate with the time between first detected contact and the manual stable-grasp label, meaning the system tends to report contact earlier than a human's 'stable grasp' annotation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the manual labels encode 'stable grasp' rather than physical touch, then the reported 89.3% agreement conflates contact detection with grasp-stability classification; evaluating the same pipeline against synchronized tactile-sensor ground truth would likely show different accuracy on non-grasp touches.
The strong overlap in quality-metric distributions suggests that a simple threshold on $\epsilon$ or $v$ will not separate contact from non-contact in general, so a temporal detector that scores the jump in both metrics at the grasping transition may outperform the static rule.
The method could be extended to estimate grasp readiness continuously during handover by treating the rising $\epsilon$ and $v$ time series as evidence accumulation, and this could be tested without new hardware by replaying the benchmark sequences under different decision rules.
A testable implication of the false-positive correlation is that if the ground-truth labels were shifted earlier to the moment of first physical contact, the reported accuracy would rise; re-annotating a subset of frames with contact sensors would settle whether the offset is labeling lag or detector lead.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a clean little experiment that basically checks whether GraspIt's contact detector, thresholded by its Ferrari-Canny quality metrics, lines up with manual stable-grasp annotations in DexYCB. It mostly does, 89.3% overall. That's a useful sanity check, but the paper oversells it slightly by calling the manual labels 'ground truth' for contact.

What's new: the specific pipeline — MANO hand reconstruction, GraspIt, Ferrari-Canny metrics, DexYCB — appears to be new. The authors are honest about the idealized setup (ground truth poses) and about their own annotation issues for subjects 8 and 9.

What it does well: the evaluation is per-object and per-subject, the false positive / false negative breakdown is informative, and the time-series observation (metrics jump at the transition) is a genuine insight worth following up.

Soft spots, in order of importance. First, the labels are 'appeared to achieve a stable grasp', not contact. A hand can touch an object without a stable grasp, and the authors even interpret early detections as contact before the stable grasp is confirmed. So the 89.3% is agreement with a stable-grasp heuristic, not a clean contact detection rate. Second, no annotation protocol, no inter-annotator agreement, no count of excluded or ambiguous frames. The authors' own comment about subjects 8 and 9 suggests label noise is real. Third, no baseline. A simple mesh-distance threshold might do as well or better; without it, we don't know what the quality metrics add over a geometric check. Fourth, no error bars; the per-object range (83.9–93.2) suggests real variation but we can't tell what's signal.

The core idea is sound and the paper is transparent. It's a proof-of-concept, not a definitive study. The numbers will hold up as a sanity check, but the central claim should be read as 'our thresholded simulator contact detector agrees with manual stable-grasp labels in most frames', not as a general contact detection accuracy.

Who this is for: robotics people working on human-robot handover or contact detection without force sensors. It's a fine workshop/short paper. For a full journal, it needs a baseline, a proper annotation study, and ideally an error analysis separating label noise from detector error.

I'd send it to peer review — it's honest and reproducible — but I'd push for major revision on the evaluation. If the authors add a baseline and tighten the label definition, it becomes a solid short paper.

Referee Report

3 major / 5 minor

Summary. The paper proposes a hand-object contact detection system that reconstructs the human hand (MANO model) and the grasped object in the GraspIt! simulator, then uses the Ferrari-Canny grasp quality metrics epsilon and v to decide whether contact has occurred. The detection rule is simple: contact is declared if either metric is greater than zero, based on the observed simulator outputs of epsilon=-1.0 and v=0.0 for no-contact frames. The method is evaluated on the DexYCB dataset using ground-truth hand and object poses, and the authors report an overall accuracy of 89.3% against manual annotations of frames in which a subject appeared to achieve a stable grasp. Per-object and per-subject accuracy tables are provided, along with distribution plots of the quality metrics and a correlation analysis between detection timing and false-positive rate.

Significance. The central idea is interesting and potentially useful: using grasp-quality metrics as a contact indicator could enable contact detection without tactile or force sensors, which is relevant for robot-to-human handovers. The paper is honest about relying on idealized ground-truth poses and about the limitations of the manual annotation. If the evaluation were made rigorous, the approach could be a practical contribution. However, the current evidence does not yet support the headline accuracy claim, because the ground truth is not contact annotation, no baseline is compared, and no uncertainty measures are given. The paper does provide a reproducible pipeline description and uses a public dataset, which are strengths, but the assessment is not yet at the level required for a journal publication.

major comments (3)

[Section III, Section III.A] The ground-truth labels are manual annotations of frames where subjects 'appeared to achieve a stable grasp,' not annotations of hand-object contact. The system, by contrast, detects contact using grasp-quality metrics. Contact and stable grasp are different constructs: a hand can contact an object before a stable grasp is achieved, and the paper itself interprets early detections as false positives in Fig. 5. Therefore the reported 89.3% accuracy (Section III.A) is not a clean measure of contact detection; it measures agreement with a stable-grasp label whose temporal alignment is not defined. The authors must either use a contact ground truth (e.g., contact labels in DexYCB, if available) or explicitly model and report the expected mismatch. At minimum, the annotation protocol, inter-annotator agreement, and the number/type of excluded frames must be reported, especially because the authors attribute accuracy dips for subjects 8 and 9 to 'errors in our manual ground truth annotation process.' This issue is load-bearing because the entire evaluation depends on this comparison.
[Section III.A, Tables I and II] No baseline method is evaluated. The paper's central claim is that the system achieves high accuracy, but without comparing against a trivial geometric baseline (e.g., distance between hand and object meshes) or an existing contact-detection method, the 89.3% figure cannot be interpreted as an improvement or even as a calibrated result. The introduction contrasts force-based methods with the proposed approach, but no quantitative comparison to any alternative is provided. Adding at least one simple baseline is necessary to support the claim that grasp-quality metrics provide a useful contact signal beyond what a distance threshold would give.
[Section III.A and Figure 5] No statistical uncertainty or significance measures are reported. The per-object accuracy ranges from 83.9% to 93.2% and per-subject from 85.1% to 91.1%, but these are point estimates without confidence intervals, and the differences are discussed without any significance testing. Figure 5 is described as showing a 'correlation' between detection offset and false-positive rate, but no correlation coefficient, p-value, or goodness-of-fit is reported, and the regression line is presented with no error bounds. As a result, the reader cannot assess whether the observed patterns are meaningful or how stable the headline 89.3% accuracy is across the dataset.

minor comments (5)

[Section III.B] The text states that both quality metrics have 'roughly similar distributions regardless of ground truth,' which appears to undermine the discriminative power of the metrics; this tension should be discussed more explicitly, perhaps by quantifying the overlap or by showing that only the temporal change (Fig. 7) is informative.
[Section III] The evaluation section mentions 'statistical analysis ... of processing times,' but no processing-time results appear anywhere in the paper; either report them or remove the mention.
[Section II] The conversion from MANO parameters to the URDF robot description used by GraspIt! is described only briefly; a short explanation or a reference to the exact procedure would improve reproducibility.
[Abstract and Conclusion] The abstract and conclusion state accuracy 'approaching 90%' while the body reports 89.3%; for a paper that emphasizes numbers, the abstract should state the precise figure.
[References] Reference [11] is a URL with an access date; consider replacing it with a standard citation or adding a formal title.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the contact rule is a simulator readout, and the 89.3% accuracy is anchored to external manual labels rather than to fitted parameters.

full rationale

The paper's detection pipeline takes DexYCB ground-truth hand and object poses, reconstructs them in GraspIt, and flags contact when either Ferrari-Canny metric exceeds zero. No parameter is fitted to the manual stable-grasp labels, and the >0 threshold is not learned: it follows directly from GraspIt's reported no-contact sentinel values (epsilon=-1.0 and v=0.0), which the paper states explicitly in Section III. The empirical claim is therefore an external comparison between a deterministic simulator readout and manually annotated frames, so the 89.3% figure is not forced by construction. The paper's own stated limitations—the undefined annotation protocol, the absence of inter-annotator agreement, and the Section III.A attribution of subject 8/9 accuracy drops to 'errors in our manual ground truth annotation process'—are measurement-validity concerns about label quality, not circularity, because those labels are not an input to the detector. The only self-citation, reference [1], appears as background on robot-to-human handover and is not load-bearing for any technical step. Thus no circular step can be exhibited from the paper's equations or definitions.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The system introduces no new physical entities or fitted constants beyond a hand-set threshold of zero on the simulator's quality metrics. The central claim rests on the reliability of the simulator's contact model, the accuracy of the MANO/DexYCB pose inputs, and the quality of the manual grasp labels used as ground truth.

free parameters (1)

Grasp quality metric threshold = 0.0
Contact is declared when epsilon > 0 or v > 0; the 0 threshold is chosen because the simulator reports epsilon=-1.0 and v=0.0 in no-contact frames. It is not optimized against labels, but it is a hand-chosen rule that directly encodes the simulator's contact state.

assumptions (3)

domain assumption GraspIt's Ferrari-Canny quality metrics and contact detection are reliable.
The system uses GraspIt's built-in contact and quality computation as the source of truth for contact; if the simulator's collision or contact model is wrong, the system's output is wrong.
domain assumption MANO hand model and DexYCB pose data accurately represent the human hand and object.
The reconstruction relies on these inputs, so errors in pose or shape parameters propagate to contact detection.
domain assumption Manual annotations of stable grasping are accurate.
Ground truth labels are manually created; no protocol or inter-annotator agreement is reported.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Hand-Object Contact Detection using Grasp Quality Metrics." pith.science (2026). https://pith.science/paper/6KMCRQNP

@misc{pith2026250106987,
  author       = {Pith},
  title        = {Pith review of: Hand-Object Contact Detection using Grasp Quality Metrics},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6KMCRQNP}},
  note         = {Machine review of arXiv:2501.06987}
}

read the original abstract

We propose a novel hand-object contact detection system based on grasp quality metrics extracted from object and hand poses, and evaluated its performance using the DexYCB dataset. Our evaluation demonstrated the system's high accuracy (approaching 90%). Future work will focus on a real-time implementation using vision-based estimation, and integrating it to a robot-to-human handover system.

Figures

Figures reproduced from arXiv: 2501.06987 by the authors.

**Figure 2.** Hand 3D mesh generated by the MANO model. Joints’ positions are [PITH_FULL_IMAGE:figures/full_fig_p001_2.png] view at source ↗

**Figure 3.** The system diagram for our hand-object contact detection. [PITH_FULL_IMAGE:figures/full_fig_p002_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 5.** Figure 5: The average offset between the first contact detection frame and the [PITH_FULL_IMAGE:figures/full_fig_p003_5.png]

**Figure 4.** Figure 4: 3D models of the (a) least and (b) most accurately detected objects. [PITH_FULL_IMAGE:figures/full_fig_p003_4.png]

**Figure 6.** Figure 6: Distribution plots of grasp quality measures for grasped and non [PITH_FULL_IMAGE:figures/full_fig_p004_6.png]

**Figure 7.** Figure 7: Grasp quality measures v (top) and ϵ (bottom) for a typical grasp sequence from the DexYCB dataset. Frames without hand-object contact as detected by the system are omitted. The X-axis represents the frame number’s offset from the first grasping frame in the ground tru…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

13 extracted references · 12 canonical work pages

[1]

Ob- ject handovers: a review for robotics,

V . Ortenzi, A. Cosgun, T. Pardi, W. P. Chan, E. Croft and D. Kuli´c, “Ob- ject handovers: a review for robotics,” IEEE Transactions on Robotics, 2021

work page 2021
[2]

”A human-inspired object handover controller,” The International Journal of Robotics Research, 2013

Chan, Wesley P., et al. ”A human-inspired object handover controller,” The International Journal of Robotics Research, 2013

work page 2013
[3]

A. T. Miller and P. K. Allen, ”GraspIt!: a versatile simulator for robotic grasping,” IEEE Robotics and Automation Magazine, 2004

work page 2004
[4]

Romero, D

J. Romero, D. Tzionas and M. J. Black, ”Embodied hands: modeling and capturing hands and bodies together,” ACM Transactions on Graphics, 2017

work page 2017
[5]

A. T. Miller and P. K. Allen, ”Examples of 3D grasp quality compu- tations,” IEEE International Conference on Robotics and Automation, 1999

work page 1999
[6]

Hasson et al., ”Learning joint reconstruction of hands and manipu- lated objects,” IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019

Y . Hasson et al., ”Learning joint reconstruction of hands and manipu- lated objects,” IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019

work page 2019
[7]

Y . W. Chao et al., ”DexYCB: a benchmark for capturing hand grasping of objects,” IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021

work page 2021
[8]

Calli et al., ”Yale-CMU-Berkeley dataset for robotic manipulation research,” The International Journal of Robotics Research, 2017

B. Calli et al., ”Yale-CMU-Berkeley dataset for robotic manipulation research,” The International Journal of Robotics Research, 2017

work page 2017

Show all 13 references

[9]

Boukhayma, R

A. Boukhayma, R. de Bem and P. H. S. Torr, ”3D hand shape and pose from images in the wild,” IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019

2019
[10]

R. A. Potamias, J. Zhang, J. Deng and S. Zafeiriou, ”WiLoR: end-to-end 3D hand localization and reconstruction in-the-wild,” arXiv:2409.12259 [cs.CV], 2024

2024 arXiv
[11]

https://ai.google.dev/ edge/mediapipe/solutions/vision/hand landmarker (accessed January 8, 2025)

”Hand landmarks detection guide,” ai.google.dev. https://ai.google.dev/ edge/mediapipe/solutions/vision/hand landmarker (accessed January 8, 2025)

2025
[12]

Y . He, Y . Wang, H. Fan, J. Sun and Q. Chen, ”FS6D: few-shot 6D pose estimation of novel objects,” IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022

2022
[13]

Hampali, M

S. Hampali, M. Rad, M. Oberweger and V . Lepetit, ”HOnnotate: a method for 3D annotation of hand and object poses,” IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020

2020

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

Ob- ject handovers: a review for robotics,

V . Ortenzi, A. Cosgun, T. Pardi, W. P. Chan, E. Croft and D. Kuli´c, “Ob- ject handovers: a review for robotics,” IEEE Transactions on Robotics, 2021

work page 2021

[2] [2]

”A human-inspired object handover controller,” The International Journal of Robotics Research, 2013

Chan, Wesley P., et al. ”A human-inspired object handover controller,” The International Journal of Robotics Research, 2013

work page 2013

[3] [3]

A. T. Miller and P. K. Allen, ”GraspIt!: a versatile simulator for robotic grasping,” IEEE Robotics and Automation Magazine, 2004

work page 2004

[4] [4]

Romero, D

J. Romero, D. Tzionas and M. J. Black, ”Embodied hands: modeling and capturing hands and bodies together,” ACM Transactions on Graphics, 2017

work page 2017

[5] [5]

A. T. Miller and P. K. Allen, ”Examples of 3D grasp quality compu- tations,” IEEE International Conference on Robotics and Automation, 1999

work page 1999

[6] [6]

Hasson et al., ”Learning joint reconstruction of hands and manipu- lated objects,” IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019

Y . Hasson et al., ”Learning joint reconstruction of hands and manipu- lated objects,” IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019

work page 2019

[7] [7]

Y . W. Chao et al., ”DexYCB: a benchmark for capturing hand grasping of objects,” IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021

work page 2021

[8] [8]

Calli et al., ”Yale-CMU-Berkeley dataset for robotic manipulation research,” The International Journal of Robotics Research, 2017

B. Calli et al., ”Yale-CMU-Berkeley dataset for robotic manipulation research,” The International Journal of Robotics Research, 2017

work page 2017