Pith. sign in

REVIEW 5 major objections 8 minor 98 references

Multimodal Human-Intent Modeling for Contextual Robot-to-Human Handovers of Arbitrary Objects

T0 review · 5 major / 8 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A robot can infer a distant handover target—and how to grasp it—from gaze plus a spoken phrase.

desk verdict The gaze+language selection result is the real contribution; the explicit-preference claim is undermined by circular validation. read the letter →

arxiv 2508.02982 v1 pith:GCG2NX6L submitted 2025-08-05 cs.RO

Lucas Chen , Guna Avula , Hanwen Ren , Zixing Wang , Ahmed H. Qureshi This is my paper
classification cs.RO
keywords human-robotobjecthandovergaze-basedselectionlanguage-guidedgraspingmultimodalintentmodelinghuman-centeredroboticsgraspgenerationphysicalinteractionassistive
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that a robot can fetch and hand over an arbitrary object from a cluttered table when the user identifies it the way people naturally do: by looking at it and saying what they want. Its unified pipeline turns a facial-camera gaze stream and a spoken instruction into a target-object bounding box, extracts any stated grasp preference, generates a stable robot grasp on the allowed region, and produces a smooth handover motion. In real-world experiments with randomly placed household objects, the combined gaze-plus-language selection identified the correct object in 96.23 percent of trials, while language alone reached 48 percent and gaze alone 82.67 percent; generated grasps were stable in 95.6 percent of trials without a specified part and 90 percent with one. The aim is to close the gap between natural human instruction and robot handover without pre-selected targets or 3D object models.

What carries the argument

Two coupled mechanisms carry the argument. The first is the fusion selector: gaze is smoothed into a 2D Gaussian heatmap, and each language-derived candidate object box receives a score equal to the gaze probability mass inside it, so the spoken name resolves gaze misses while the gaze resolves ambiguity between same-name objects. The second is the part-constrained grasp proposer: the holding part, if stated, is localized as the bounding-box overlap with the object box, and the grasp generator is restricted to that overlap when the robot holds the part or to its complement when the human does; when no part is stated, candidate robot grasps are rescored against a predicted human hand by a distance measure and an approach-angle measure. These mechanisms make both selection and grasping work on arbitrary, unseen objects.

What would settle it

Set up a scene in which the user says "hand me the mug and I want to hold the handle," but the mug's handle is adjacent to another object so the handle's bounding box overlaps both objects; a successful run requires the robot's grasp to sample only mug geometry. If the grasp point cloud comes from the neighboring object or the handle is not left free, the part-localization assumption fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that human intent for a handover can be modeled well enough, from one facial camera and one spoken sentence, to drive the whole robot action sequence. The selection step computes a gaze heatmap from the user's face, then scores every candidate box produced by a language-grounded object detector by the gaze probability inside it, choosing the box with the highest score. When the user names a part such as a handle, tip, or stem, the part is localized as the intersection of its detected box with the object box; the robot is allowed to grasp either that intersection, if the robot should hold the part, or the complement, if the human should. Grasp candidates are then generated on the allowed point cloud and ranked by stability confidence, or, when no part is named, by distance and approach angle away from a predicted human hand. The reported success rates, 96.23 percent object selection and 95.6 percent or 90 percent grasp stability, are the empirical backing for this claim.

Load-bearing premise

The explicit-preference branch assumes the named holding part can be located by intersecting its bounding box with the object's bounding box, and that this rectangle, or its complement, is a valid region for the grasp generator; if the part box bleeds onto neighboring geometry or the object mask is coarse, the robot will grasp the wrong surface even when selection succeeds.

Editorial extensions

If this is right

  • Household and assistive robots could fetch and hand over objects named on the fly, without a pre-registered object database or 3D models.
  • Users can steer the robot's grasp with ordinary phrases like "hold the tip" or "leave the handle," which matters for sharp, fragile, or bulky objects.
  • The combined selection result gives a concrete baseline: language alone 48 percent, gaze alone 82.67 percent, and fused 96.23 percent on cluttered scenes with multiple instances of the same object.
  • Because the pipeline is modular, improving any single component, such as gaze estimation, language grounding, grasp proposal, or motion policy, should improve the integrated handover rather than requiring a new architecture.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper reports component accuracies but not an end-to-end success rate; a direct extension would measure how often a full trial ends with the user holding the requested part, since that multiplies selection, part localization, grasp stability, and motion success.
  • The explicit-preference path treats the part as a 2D rectangle in image space; propagating the part constraint into the completed 3D point cloud before grasp generation is a testable change that would likely improve the 90 percent stability rate on narrow objects.
  • Because language alone is weak at 48 percent while gaze alone reaches 82.67 percent, the fusion gain comes mostly from language filtering gaze misses; a cheap extension is confidence-weighted fusion, letting a highly confident object name override a stray gaze point.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 8 minor

Summary. The paper proposes an integrated pipeline for robot-to-human handover of arbitrary objects selected from a distance. The user indicates an object by gaze and natural language; a gaze module (MPIIFaceGaze with MediaPipe head-direction blending, EMA smoothing, and a Gaussian heatmap) is combined with a language parser (RoBERTa on SQuAD plus SpaCy dependency analysis) to extract the object name, holding part, and holder (robot or human). Multimodal object selection uses GLIP bounding boxes scored by gaze heatmap overlap, grasp generation uses PoinTr for point-cloud completion and Contact-GraspNet constrained either by the parsed part region or by GraspTTA/CoGrasp hand prediction, and motion generation uses Riemannian Motion Policies. The paper reports real-world subsystem experiments, a 10-participant user study, and headline numbers of 96.23% object-selection accuracy for gaze+language, 92% grasp-preference compliance, and 95.6%/90% grasp stability without/with a specified part.

Significance. If the claims hold, the paper addresses a genuinely underexplored combination: remote target selection via gaze and free-form language together with explicit and implicit grasp-preference modeling for handover. The integration of published components (GLIP, PoinTr, Contact-GraspNet, GraspTTA, CoGrasp) into a working real-robot system with a user study is a useful engineering contribution, and the explicit-preference branch is a clear extension over the authors' prior CoGrasp work. The paper does not provide machine-checked proofs or released code/data, and the evidence for the explicit-preference branch and for the motion module is currently weaker than the headline claims suggest. The central qualitative conclusions are plausible and worth pursuing, but the quantitative support needs correction and strengthening.

major comments (5)
  1. [§4.2.4, Algorithm 1, Algorithm 2] The preference-compliance metric in §4.2.4 is self-referential. Success for specified-part trials is declared when the grasp falls within (or outside) BOp, but BOp is computed by Algorithm 1 and then used in Algorithm 2 to extract Ppc, the exact point cloud fed to Contact-GraspNet. Consequently Table 5's 92% measures consistency with the algorithm's own region, not semantic compliance with the user's stated part. A grasp on the wrong part of the object can pass this check whenever the computed region is wrong in the same way. Please evaluate compliance with independent part-level ground truth, e.g., per-trial human coding or 3D part annotations, and report whether the robot actually contacted the intended part.
  2. [§3.4, Algorithm 1 Lines 4-10] The 2D bounding-box algebra used to localize the holding part is not a reliable 3D graspable region. For TH=Human, the complement BOu − (BOu ∩ BOp) includes, after depth projection, table and background points that lie inside the object's 2D box; Contact-GraspNet can therefore propose grasps on the table or off the object. For TH=Robot, the intersection inherits any GLIP part-box misalignment and is not aligned to the 3D part boundary. The paper provides no evaluation of part localization accuracy against 3D part labels, and the reported 92% cannot detect this failure mode. Please replace or augment the box-intersection step with depth-aware or mask-based part segmentation and validate it separately.
  3. [§4.2.3 Table 4 and §4.2.5 Table 6] Several reported statistics are internally inconsistent. For n=150 per condition, gaze 82.67% vs. gaze+language 96.23% gives a two-proportion z of about 3.82, not -3.06, and language 48% vs. gaze+language 96.23% gives z≈9.31, not -5.94; also 96.23% of 150 is 144.345 trials, which is impossible for integer success counts. In Table 6, the 'No Part Specified' column sums to 44/45 = 97.8%, while the text reports 95.6% (43/45). These discrepancies make the headline numbers unreliable as stated. Please correct the values, report exact trial counts, and release the underlying trial-level data so the statistics can be verified.
  4. [§3.6, §4] The handover motion generation module, one of the four stated contributions, has no quantitative evaluation. The RMP formulation in §3.6 is described, but there are no module-level metrics for trajectory smoothness, jerk, collision avoidance, or handover success, and the user-study questions address subjective time and comfort rather than motion quality. Please add a dedicated evaluation or explicitly reposition this component as an implementation detail rather than a validated contribution.
  5. [§3.3, §4.2] The language parser is never assessed in isolation. The explicit-preference branch depends on correctly extracting (TO, TP, TH), but the paper reports no accuracy for the roBERTa/SpaCy parser on annotated commands. End-to-end selection and stability results can mask parser errors, and a misassigned TH would send the grasp to the wrong region before any grasp generator runs. Please include a small benchmark with ground-truth (TO, TP, TH) annotations for the test utterances.
minor comments (8)
  1. [§4.2.1] The sentence 'This allows for additional language-based input from the user to boost object selection accuracy, as shown later in Figure 3' should refer to the user-study results figure (Figure 6), not the environment setup figure.
  2. [§4.2.1, Table 2] The text states gaze success ranges from 76% to 96%, but the maximum in Table 2 is 94%; please correct the text or the table.
  3. [§4.2.2] The size categories leave gaps: small is defined as less than 1 cm, medium as 2-4 cm, and large as greater than 8 cm, with no ranges for 1-2 cm, 4-8 cm, or exactly 8 cm. Please specify exhaustive intervals and clarify how object width is measured.
  4. [Algorithm 1, Line 2] The score SOi is described as an IoU score but is computed as the sum of heatmap probability density inside the bounding box; please rename it (e.g., 'heatmap score') to avoid confusion with intersection-over-union.
  5. [Algorithm 2, caption and Line 1] The algorithm caption says the input is color and depth image, but the input list shows only Id; please align the caption, input list, and comments.
  6. [§4.2.5] The failure-case discussion says failures involved Scissors, Screwdriver, and Small Clamp, but in the no-part column only the Scissors has a failed trial; please state which rows correspond to which condition.
  7. [Table 7] The total time with part specification (43 s) is shorter than without (55 s) even though part selection adds time; this is presumably because human-grasp prediction is skipped, but the reason should be stated explicitly in the text.
  8. [Throughout] Please proofread for typos and formatting issues, including 'Departmant', 'computiation', 'multimodel', 'accpetable', 'socially-complaint', and the duplicated reference entry [90].

Circularity Check

1 steps flagged · score 6.0 of 10

Explicit-preference grasp success reduces to box arithmetic: the same computed part region both constrains grasp generation and defines the success metric.

  1. self definitional [Section 4.2.4 (Grasp Generation), with Algorithm 1 Lines 6-10 and Algorithm 2 Lines 3-6]
    "Only the robot's grasping preference is stated. A success is counted whenever the robot grasp falls within the region of the robot's preference part. ... the language instruction eliminates a great source of variance in the grasp generation output since the grasp trial declares success as long as the gripper obeys the part constraint."

    The success metric for the explicit-preference branch is defined over the same region that the method itself computes and then uses to constrain grasp generation. Algorithm 1 (Lines 6-10) produces BOp as a GLIP part box intersected with (or subtracted from) the object box. Algorithm 2 (Line 4) crops the completed object point cloud with exactly that BOp ('Ppc <- PC Extractor(BOp, pc')') and Contact-GraspNet receives only Ppc (Line 6). Section 4.2.4 then counts success precisely when a grasp falls inside/outside BOp.

full rationale

The only load-bearing circularity is in the explicit-preference grasp validation. The paper's own definitions tie the success criterion to Algorithm 1's computed part box BOp, which is also the crop used to build the grasp generator's input Ppc. Consequently Table 5's 'with human/with robot' percentages cannot certify that the robot grasped the semantically correct part; they certify only that the grasp stayed inside the box the system itself drew. This is a partial circularity because it collapses the paper's main novelty relative to its prior CoGrasp work. The rest of the pipeline is not circular: combined gaze+language object selection (96.23%) is scored against independently specified target objects, and grasp stability (95.6% and 90%) is a physical lift-and-hold test that is blind to part semantics but is a real, independent measurement. The use of the authors' prior CoGrasp scoring function is a normal citation to a published, externally evaluated system; the paper does not fit CoGrasp to these trials, and no uniqueness theorem is imported, so it does not raise the score. Some reporting inconsistencies (e.g., Table 6 no-part column summing to 44/45 while the text reports 95.6%, and the Z-statistics not reproducing from the stated trial counts) are correctness risks but not circularity. Overall, one central 'prediction' reduces by construction to its computed input, giving a partial circularity score of 6.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The system does not introduce new physical entities; it combines pre-trained models. The free parameters listed are tuning choices that affect the gaze fusion, heatmap, and motion modules.

free parameters (4)
  • Alpha, gaze-head fusion weight = not reported
    Used in Eq. 1 to blend head direction and MPIIFaceGaze gaze direction; the value is chosen by the authors but never given in the paper.
  • Beta, EMA smoothing factor for gaze = not reported
    Used in Eq. 4 to smooth the gaze point sequence; the value is not specified.
  • Gaussian standard deviation for gaze heatmap = 57 pixels per axis
    Set via cross-validation and qualitative analysis in Section 3.2; it affects which object wins the heatmap score.
  • Kappa and Omega, RMP scaling hyperparameters = not reported
    Appear in Eq. 7 for motion generation; the values are not given, so the motion behavior is not fully reproducible from the text.
assumptions (5)
  • domain assumption A user's gaze direction, estimated from head pose and eye gaze, reliably points at the intended object on the monitor while the user speaks.
    The fusion in Section 3.2 and Algorithm 1 depends on this; gaze alone only reaches 82.67 percent in the paper's own test.
  • domain assumption GLIP can generate correct candidate bounding boxes for arbitrary user-phrased object names and part names from a single image.
    Algorithm 1 feeds TO and TP to GLIP; language-only selection is 48 percent, showing GLIP by itself is often insufficient.
  • ad hoc to paper The intersection or set-difference of object and part bounding boxes accurately defines the robot graspable region.
    Algorithm 1, Lines 7-10, uses bounding-box intersection or subtraction as the grasp region for explicit preferences, which is an approximation not validated against part segmentation.
  • domain assumption PoinTr point cloud completion produces geometry accurate enough for Contact-GraspNet to generate stable grasps.
    Algorithm 2, Lines 2-4, uses PoinTr to complete unseen object backsides; no ablation or error analysis for completion is given.
  • domain assumption GraspTTA predicted human hand grasps approximate where a person would naturally hold the object.
    Used in Section 3.5, case 3, to score robot grasps; the paper itself notes hand predictions are not well aligned with standard human grasping areas in Section 4.2.4.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multimodal Human-Intent Modeling for Contextual Robot-to-Human Handovers of Arbitrary Objects." pith.science (2026). https://pith.science/paper/GCG2NX6L

@misc{pith2026250802982,
  author       = {Pith},
  title        = {Pith review of: Multimodal Human-Intent Modeling for Contextual Robot-to-Human Handovers of Arbitrary Objects},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GCG2NX6L}},
  note         = {Machine review of arXiv:2508.02982}
}
read the original abstract

Human-robot object handover is a crucial element for assistive robots that aim to help people in their daily lives, including elderly care, hospitals, and factory floors. The existing approaches to solving these tasks rely on pre-selected target objects and do not contextualize human implicit and explicit preferences for handover, limiting natural and smooth interaction between humans and robots. These preferences can be related to the target object selection from the cluttered environment and to the way the robot should grasp the selected object to facilitate desirable human grasping during handovers. Therefore, this paper presents a unified approach that selects target distant objects using human verbal and non-verbal commands and performs the handover operation by contextualizing human implicit and explicit preferences to generate robot grasps and compliant handover motion sequences. We evaluate our integrated framework and its components through real-world experiments and user studies with arbitrary daily-life objects. The results of these evaluations demonstrate the effectiveness of our proposed pipeline in handling object handover tasks by understanding human preferences. Our demonstration videos can be found at https://youtu.be/6z27B2INl-s.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

98 extracted references · 58 canonical work pages

  1. [1]

    Object Handovers: a Review for Robotics

    V. Ortenzi, A. Cosgun, T. Pardi, W. Chan, E. Croft, D. Kulic, Object handovers: a review for robotics, IEEE Transactions on robotics (2022). arXiv:2007.12952

    work page Pith review arXiv 2022

  2. [2]

    H. Duan, Y. Yang, D. Li, P. Wang, Human–robot object handover: Recent progress and future direction, Biomimetic Intelligence and Robotics 4 (1) (2024) 100145. doi:https://doi.org/10.1016/j.birob.2024.100145. URL https://www.sciencedirect.com/science/article/pii/ S2667379724000032

    arXiv 2024

  3. [3]

    Aleotti, V

    J. Aleotti, V. Micelli, S. Caselli, An affordance sensitive system for robot to human object handover, International Journal of Social Robotics 6 (2014) 653 – 666. URL https://api.semanticscholar.org/CorpusID:45666433

    2014

  4. [4]

    H. N. Dang, P. K. Allen, Semantic grasping: Planning robotic grasps functionally suitable for an object manipulation task, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (2012) 1311–1317. URL https://api.semanticscholar.org/CorpusID:8860232

    2012

  5. [5]

    Huber, H

    M. Huber, H. Radrich, C. Wendt, M. Rickert, A. Knoll, T. Brandt, S. Glasauer, Evaluation of a novel biologically in- spired trajectory generator in human-robot interaction, in: RO- MAN 2009 - The 18th IEEE International Symposium on Robot and Human Interactive Communication, 2009, pp. 639–644. doi:10.1109/ROMAN.2009.5326233

    work page arXiv 2009

  6. [6]

    Y. Liu, Y. Liu, C. Jiang, K. Lyu, W. Wan, H. Shen, B. Liang, Z. Fu, H. Wang, L. Yi, Hoi4d: A 4d egocentric dataset for category-level human-object interaction, Computer Vision and Pattern Recognition (2024). arXiv:2203.01577

    arXiv 2024

  7. [7]

    Grasp as You Say: Language-guided Dexterous Grasp Generation

    Y.-L. Wei, J.-J. Jiang, C. Xing, X. Tan, X.-M. Wu, H. Li, M. Cutkosky, W.-S. Zheng, Grasp as you say: Language-guided dexterous grasp generation (2024). arXiv:2405.19291

    work page Pith review arXiv 2024

  8. [8]

    I. Kim, H. Inooka, Hand-over of an object between human and robot, [1992] Proceedings IEEE International Workshop on Robot and Human Communication (1992) 199–203. URL https://api.semanticscholar.org/CorpusID:62094396

    1992

Show all 98 references
  1. [9]

    A. K. Keshari, H. Ren, A. H. Qureshi, Cograsp: 6-dof grasp generation for human-robot collaboration, in: 2023 IEEE In- ternational Conference on Robotics and Automation (ICRA), 2023, pp. 9829–9836. doi:10.1109/ICRA48891.2023.10160623

    2023

  2. [10]

    Parastegari, B

    S. Parastegari, B. Abbasi, E. Noohi, M. Zefran, Modeling hu- man reaching phase in human-human object handover with ap- plication in robot-human handover, in: 2017 IEEE/RSJ Inter- national Conference on Intelligent Robots and Systems (IROS), 2017, pp. 3597–3602. doi:10.1109/IR...

    2017

  3. [11]

    Parastegari, E

    S. Parastegari, E. Noohi, B. Abbasi, M. ˇZefran, Fail- ure recovery in robot–human object handover, IEEE Transactions on Robotics 34 (3) (2018) 660–673. doi:10.1109/TRO.2018.2819198

    2018

  4. [12]

    J. R. Wolpaw, D. J. McFarland, G. W. Neat, C. A. Forneris, An eeg-based brain-computer interface for cursor control, Elec- troencephalography and Clinical Neurophysiology 78 (3) (1991) 252–259. doi:https://doi.org/10.1016/0013-4694(91)90040-B. URL https://www.sciencedirect.com...

    1991 doi

  5. [13]

    Jeong, K.-H

    J.-H. Jeong, K.-H. Shim, D.-J. Kim, S.-W. Lee, Brain-controlled robotic arm system based on multi-directional cnn-bilstm net- work using eeg signals, IEEE Transactions on Neural Sys- tems and Rehabilitation Engineering 28 (5) (2020) 1226–1238. doi:10.1109/TNSRE.2020.2981659

    2020

  6. [14]

    Devlin, M.-W

    J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, Bert: Pre- training of deep bidirectional transformers for language under- standing, North American Chapter of the Association for Com- putational Linguistics (2019). arXiv:1810.04805. 14

    2019 arXiv

  7. [15]

    Raffel, N

    C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, P. J. Liu, Exploring the limits of transfer learning with a unified text-to-text transformer, Jour- nal of machine learning research (2023). arXiv:1910.10683

    2023 arXiv

  8. [16]

    Clark, M.-T

    K. Clark, M.-T. Luong, Q. V. Le, C. D. Manning, Electra: Pre- training text encoders as discriminators rather than generators, International Conference on Learning Representations (ICLR) (2020). arXiv:2003.10555

    2020 arXiv

  9. [17]

    T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. ...

    2020 arXiv

  10. [18]

    X. Lai, Z. Tian, Y. Chen, Y. Li, Y. Yuan, S. Liu, J. Jia, Lisa: Reasoning segmentation via large language model, 2024 IEEE/CVF Conference on Computer Vision and Pattern Recog- nition (CVPR) (2023) 9579–9589. URL https://api.semanticscholar.org/CorpusID:260351258

    2023

  11. [19]

    Huang, X

    R. Huang, X. Pan, H. Zheng, H. Jiang, Z. Xie, S. Song, G. Huang, Joint representation learning for text and 3d point cloud, Pattern Recognition (2023). arXiv:2301.07584

    2023 arXiv

  12. [20]

    Rozenberszki, O

    D. Rozenberszki, O. Litany, A. Dai, Language-grounded indoor 3d semantic segmentation in the wild, European Conference on Computer Vision (2022). arXiv:2204.07761

    2022 arXiv

  13. [22]

    Ghosh, A

    S. Ghosh, A. Dhall, M. Hayat, J. Knibbe, Q. Ji, Automatic gaze analysis: A survey of deep learning based approaches, IEEE Transactions on Pattern Analysis and Machine Intelli- gence (2022). arXiv:2108.05479

    2022 arXiv

  14. [23]

    Zhang, Y

    X. Zhang, Y. Sugano, M. Fritz, A. Bulling, Mpiigaze: Real- world dataset and deep appearance-based gaze estimation, IEEE Transactions on Pattern Analysis and Machine Intelli- gence (2017). arXiv:1711.09017

    2017 arXiv

  15. [24]

    Zhang, Y

    X. Zhang, Y. Sugano, M. Fritz, A. Bulling, Mpiigaze: Real-world dataset and deep appearance-based gaze es- timation, IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 41 (1) (2019) 162–175. doi:10.1109/TPAMI.2017.2778103

    2019

  16. [25]

    C ¸ akır, A

    M. C ¸ akır, A. Huckauf, Reviewing the social function of eye gaze in social interaction, in: Proceedings of the 2023 Symposium on Eye Tracking Research and Applications, 2023, pp. 1–3

    2023

  17. [26]

    Strabala, M

    K. Strabala, M. K. Lee, A. D. Dragan, J. Forlizzi, S. S. Srini- vasa, M. Cakmak, V. Micelli, Toward seamless human-robot handovers, Journal of Human-Robot Interaction 2 (2013) 112 – 132. URL https://api.semanticscholar.org/CorpusID:15247009

    2013

  18. [27]

    Y. Qiao, C. Deng, Q. Wu, Referring expression comprehension: A survey of methods and datasets, IEEE transactions on mul- timedia (2020). arXiv:2007.09554

    2020 arXiv

  19. [28]

    Langer, F

    D. Langer, F. Legler, P. Kotsch, A. Dettmann, A. C. Bullinger- Hoffmann, I let go now! towards a voice-user interface for han- dovers between robots and users with full and impaired sight, Robotics 11 (2022) 112. URL https://api.semanticscholar.org/CorpusID:252973599

    2022

  20. [29]

    Zhang, Y

    H. Zhang, Y. Lu, C. Yu, D. Hsu, X. Lan, N. Zheng, Invigorate: Interactive visual grounding and grasping in clutter, Robotics: Science and Systems (2024). arXiv:2108.11092

    2024 arXiv

  21. [30]

    P. Chen, Q. Li, S. Biaz, T. Bui, A. Nguyen, gscorecam: What objects is clip looking at?, in: Proceedings of the Asian Confer- ence on Computer Vision, 2022, pp. 1959–1975

    2022

  22. [31]

    L¨ uddecke, A

    T. L¨ uddecke, A. Ecker, Image segmentation using text and im- age prompts, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 7086–7096

    2022

  23. [32]

    Decatur, I

    D. Decatur, I. Lang, R. Hanocka, 3d highlighter: Localizing re- gions on 3d shapes via text descriptions, 2023 IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR) (2022) 20930–20939. URL https://api.semanticscholar.org/CorpusID:254926514

    2022

  24. [33]

    Y. Song, P. Sun, Y. Ren, Y. Zheng, Y. Zhang, Learning 6-dof fine-grained grasp detection based on part affordance grounding, CoRR (2023). arXiv:2301.11564

    2023 arXiv

  25. [34]

    X. Zhu, R. Zhang, B. He, Z. Guo, Z. Zeng, Z. Qin, S. Zhang, P. Gao, Pointclip v2: Prompting clip and gpt for powerful 3d open-world learning, 2023 IEEE/CVF International Conference on Computer Vision (ICCV) (2022) 2639–2650. URL https://api.semanticscholar.org/CorpusID:261241594

    2022

  26. [35]

    Ngyen, M

    T. Ngyen, M. N. Vu, A. Vuong, D. Nguyen, T. D. Vo, N. T. H. Le, A. M. Nguyen, Open-vocabulary affordance detection in 3d point clouds, 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2023) 5692–5698. URL https://api.semanticscholar.org/CorpusID...

    2023

  27. [36]

    C. Tang, D. Huang, L. Meng, W. Liu, H. Zhang, Task-oriented grasp prediction with visual-language inputs, 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2023) 4881–4888. URL https://api.semanticscholar.org/CorpusID:257233075

    2023

  28. [37]

    H. Ding, S. D. Cohen, B. L. Price, X. Jiang, Phraseclick: To- ward achieving flexible interactive segmentation by phrase and click, in: European Conference on Computer Vision, 2020. URL https://api.semanticscholar.org/CorpusID:224801564

    2020

  29. [38]

    D. Leal, O. Leal, Initiating object handover in human-robot collaboration using a multi-modal wearable and deep learning visual system, 2020. URL https://api.semanticscholar.org/CorpusID:231853694

    2020

  30. [39]

    Laplaza, A

    J. Laplaza, A. Pumarola, F. Moreno-Noguer, A. Sanfeliu, At- tention deep learning based model for predicting the 3d hu- man body pose using the robot human handover phases, in: 2021 30th IEEE International Conference on Robot and Hu- man Interactive Communication (RO-MAN), 202...

    2021

  31. [40]

    A. Choi, M. K. Jawed, J. Joo, Preemptive motion planning for human-to-robot indirect placement handovers, in: 2022 Interna- tional Conference on Robotics and Automation (ICRA), IEEE, 2022, pp. 4743–4749

    2022

  32. [41]

    Cosgun, A

    A. Cosgun, A. J. B. Trevor, H. I. Christensen, Did you mean this object?: Detecting ambiguity in pointing gesture targets, 2015. URL https://api.semanticscholar.org/CorpusID:211567531

    2015

  33. [42]

    Trick, D

    S. Trick, D. Koert, J. Peters, C. A. Rothkopf, Multimodal uncer- tainty reduction for intention recognition in human-robot inter- action, in: 2019 IEEE/RSJ International Conference on Intelli- gent Robots and Systems (IROS), IEEE, 2019, pp. 7009–7016

    2019

  34. [43]

    Rajabi, P

    N. Rajabi, P. Khanna, S. U. D. Kanik, E. Yadollahi, M. Vasco, M. Bj¨ orkman, C. Smith, D. Kragic, Detecting the intention of object handover in human-robot collaborations: An eeg study, in: 2023 32nd IEEE International Conference on Robot and Human Interactive Communication (R...

    2023

  35. [44]

    Sharma, M

    M. Sharma, M. Rekrut, J. Alexandersson, A. Kr¨ uger, Towards improving eeg-based intent recognition in visual search tasks, in: International Conference on Neural Information Processing, Springer, 2022, pp. 604–615

    2022

  36. [45]

    Cooper, S

    S. Cooper, S. F. Fensome, D. Kourtis, S. Gow, M. Dragone, An eeg investigation on planning human-robot handover tasks, in: 2020 IEEE International Conference on Human-Machine Systems (ICHMS), IEEE, 2020, pp. 1–6

    2020

  37. [46]

    A. B. Vasudevan, D. Dai, L. V. Gool, Object referring in videos with language and human gaze, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018) 4129–4138. URL https://api.semanticscholar.org/CorpusID:4576781

    2018

  38. [47]

    J. Tian, L. Yang, R. Ji, Y. Ma, L. Xu, J. Yu, Y. Shi, J. Wang, Gaze-guided hand-object interaction synthesis: Benchmark and method, ArXiv abs/2403.16169 (2024). URL https://api.semanticscholar.org/CorpusID:268681107 15

    2024

  39. [48]

    Staudte, M

    M. Staudte, M. Crocker, The utility of gaze in spoken human- robot interaction, in: Proceedings of Workshop on Metrics for Human-Robot Interaction 2008, March 12th, 2008, pp. 53–59

    2008

  40. [49]

    A. Moon, D. M. Troniak, B. T. Gleeson, M. K. X. J. Pan, M. Zheng, B. A. Blumer, K. E. Maclean, E. A. Croft, Meet me where i’m gazing: How shared attention gaze affects human- robot handover timing, 2014 9th ACM/IEEE International Conference on Human-Robot Interaction (HRI) (20...

    2014

  41. [50]

    Gharbi, P.-V

    M. Gharbi, P.-V. Paubel, A. Clodic, O. Carreras, R. Alami, J.-M. Cellier, Toward a better understanding of the commu- nication cues involved in a human-robot object transfer, in: 2015 24th IEEE International Symposium on Robot and Hu- man Interactive Communication (RO-MAN), 20...

    2015

  42. [51]

    R. M. Aronson, T. Santini, T. C. K¨ ubler, E. Kasneci, S. S. Srini- vasa, H. Admoni, Eye-hand behavior in human-robot shared manipulation, 2018 13th ACM/IEEE International Conference on Human-Robot Interaction (HRI) (2018) 4–13. URL https://api.semanticscholar.org/CorpusID:3707577

    2018

  43. [52]

    Y. T. Paulus, G. B. Remijn, Usability of vari- ous dwell times for eye-gaze-based object selec- tion with eye tracking, Displays 67 (2021) 101997. doi:https://doi.org/10.1016/j.displa.2021.101997. URL https://www.sciencedirect.com/science/article/pii/ S0141938221000123

    2021

  44. [53]

    Schweigert, V

    R. Schweigert, V. Schwind, S. Mayer, Eyepointing: A gaze-based selection technique, in: Proceedings of Mensch Und Computer 2019, MuC ’19, Association for Comput- ing Machinery, New York, NY, USA, 2019, p. 719–723. doi:10.1145/3340764.3344897. URL https://doi.org/10.1145/334076...

    2019

  45. [54]

    Y.-F. Shao, C. Wang, C.-S. Fuh, Eyelasso: Real-world object selection using gaze-based gestures, in: 28th IPPR Conference on Computer Vision, Graphics, and Image Processing, 2015

    2015

  46. [55]

    Krafka, A

    K. Krafka, A. Khosla, P. Kellnhofer, H. Kannan, S. Bhandarkar, W. Matusik, A. Torralba, Eye tracking for everyone, Computer Vision and Pattern Recognition (2016). arXiv:1606.05814

    2016 arXiv

  47. [56]

    H. Deng, W. Zhu, Monocular free-head 3d gaze tracking with deep learning and geometry constraints, 2017 IEEE Interna- tional Conference on Computer Vision (ICCV) (2017) 3162– 3171. URL https://api.semanticscholar.org/CorpusID:32485021

    2017

  48. [57]

    Fischer, H

    T. Fischer, H. J. Chang, Y. Demiris, RT-GENE: Real-Time Eye Gaze Estimation in Natural Environments, in: European Conference on Computer Vision, 2018, pp. 339–357

    2018

  49. [58]

    Kellnhofer, A

    P. Kellnhofer, A. Recasens, S. Stent, W. Matusik, A. Tor- ralba, Gaze360: Physically unconstrained gaze estimation in the wild, IEEE International Conference on Computer Vision (2019). arXiv:1910.10088

    2019 arXiv

  50. [59]

    B. A. Smith, Q. Yin, S. K. Feiner, S. K. Nayar, Gaze locking: passive eye contact detection for human-object interaction, Pro- ceedings of the 26th annual ACM symposium on User interface software and technology (2013). URL https://api.semanticscholar.org/CorpusID:16104238

    2013

  51. [60]

    C. D. McMurrough, V. Metsis, J. Rich, F. Makedon, An eye tracking dataset for point of gaze detection, Proceedings of the Symposium on Eye Tracking Research and Applications (2012). URL https://api.semanticscholar.org/CorpusID:466361

    2012

  52. [61]

    K. A. F. Mora, F. Monay, J.-M. Odobez, Eyediap: a database for the development and evaluation of gaze estimation algo- rithms from rgb and rgb-d cameras, Proceedings of the Sympo- sium on Eye Tracking Research and Applications (2014). URL https://api.semanticscholar.org/Corpus...

    2014

  53. [62]

    Weidenbacher, G

    U. Weidenbacher, G. Layher, P.-M. Strauss, H. Neumann, A comprehensive head pose and gaze database, in: 2007 3rd IET International Conference on Intelligent Environments, 2007, pp. 455–458. doi:10.1049/cp:20070407

    2007 doi

  54. [63]

    Hennessey, B

    C. Hennessey, B. Noureddin, P. D. Lawrence, A single camera eye-gaze tracking system with free head motion, Proceedings of the 2006 symposium on Eye tracking research & applications (2006). URL https://api.semanticscholar.org/CorpusID:3238368

    2006

  55. [64]

    M. K. X. J. Pan, V. Skjervøy, W. P. Chan, M. Inaba, E. A. Croft, Automated detection of handovers using kinematic fea- tures, The International Journal of Robotics Research 36 (2017) 721 – 738. URL https://api.semanticscholar.org/CorpusID:43614132

    2017

  56. [65]

    Micelli, K

    V. Micelli, K. Strabala, S. S. Srinivasa, Perception and control challenges for effec tive human-robot handoff s, 2011. URL https://api.semanticscholar.org/CorpusID:15607569

    2011

  57. [66]

    W. Wang, R. Li, Z. M. Diekel, Y. Chen, Z. Zhang, Y. Jia, Con- trolling object hand-over in human–robot collaboration via nat- ural wearable sensing, IEEE Transactions on Human-Machine Systems 49 (1) (2019) 59–71. doi:10.1109/THMS.2018.2883176

    2019

  58. [67]

    Cakmak, S

    M. Cakmak, S. S. Srinivasa, M. K. Lee, S. Kiesler, J. Forl- izzi, Using spatial and temporal contrast for fluent robot-human hand-overs, in: 2011 6th ACM/IEEE International Confer- ence on Human-Robot Interaction (HRI), 2011, pp. 489–496. doi:10.1145/1957656.1957823

    2011

  59. [68]

    W. Yang, C. Paxton, M. Cakmak, D. Fox, Human grasp classification for reactive human-to-robot handovers, in: 2020 IEEE/RSJ International Conference on Intelli- gent Robots and Systems (IROS), 2020, pp. 11123–11130. doi:10.1109/IROS45743.2020.9341004

    2020

  60. [69]

    W. Yang, C. Paxton, A. Mousavian, Y.-W. Chao, M. Cakmak, D. Fox, Reactive human-to-robot handovers of arbitrary objects (2021). arXiv:2011.08961

    2021 arXiv

  61. [70]

    Aleotti, V

    J. Aleotti, V. Micelli, S. Caselli, Comfortable robot to human object hand-over, in: 2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication, 2012, pp. 771–776. doi:10.1109/ROMAN.2012.6343845

    2012

  62. [71]

    Edsinger, C

    A. Edsinger, C. C. Kemp, Human-robot interaction for cooper- ative manipulation: Handing objects to one another, in: RO- MAN 2007 - The 16th IEEE International Symposium on Robot and Human Interactive Communication, 2007, pp. 1167–1172. doi:10.1109/ROMAN.2007.4415256

    2007

  63. [72]

    S. J. Christen, L. Feng, W. Yang, Y.-W. Chao, O. Hilliges, J. Song, Synh2r: Synthesizing hand-object motions for learn- ing human-to-robot handovers, 2024 IEEE International Con- ference on Robotics and Automation (ICRA) (2023) 3168–3175. URL https://api.semanticscholar.org/Co...

    2023

  64. [73]

    Antanas, P

    L. Antanas, P. Moreno, M. Neumann, R. P. de Figueiredo, K. Kersting, J. Santos-Victor, L. D. Raedt, Semantic and geo- metric reasoning for robotic grasping: a probabilistic logic ap- proach, Autonomous Robots 43 (2018) 1393 – 1418. URL https://api.semanticscholar.org/CorpusID:69354340

    2018

  65. [74]

    Detry, C

    R. Detry, C. H. Ek, M. Madry, J. H. Piater, D. Kragic, Gen- eralizing grasps across partly similar objects, 2012 IEEE Inter- national Conference on Robotics and Automation (2012) 3791– 3797. URL https://api.semanticscholar.org/CorpusID:12062344

    2012

  66. [75]

    D. Song, K. Huebner, V. Kyrki, D. Kragic, Learning task constraints for robot grasping using graphical models, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (2010) 1579–1585. URL https://api.semanticscholar.org/CorpusID:10337393

    2010

  67. [76]

    W. Liu, A. Daruna, S. Chernova, Cage: Context-aware grasp- ing engine, IEEE International Conference on Robotics and Au- tomation (2020). arXiv:1909.11142

    2020 arXiv

  68. [77]

    Zheng, X

    Y. Zheng, X. Chen, Y. Zheng, S. Gu, R. Yang, B. Jin, P. Li, C. Zhong, Z. Wang, L. Liu, C. Yang, D. Wang, Z. Chen, X. Long, M. Wang, Gaussiangrasper: 3d language gaussian splatting for open-vocabulary robotic grasping (2024). arXiv:2403.09637

    2024 arXiv

  69. [78]

    Mainprice, E

    J. Mainprice, E. A. Sisbot, T. Sim´ eon, R. Alami, Planning safe and legible hand-over motions for human-robot interaction, 2010. URL https://api.semanticscholar.org/CorpusID:6846660

    2010

  70. [79]

    E. A. Sisbot, L. F. Mar ´ ın-Ur ´ ıas, X. Broqu` ere, D. Sidobre, R. Alami, Synthesizing robot motions adapted to human pres- 16 ence, International Journal of Social Robotics 2 (2010) 329–343. URL https://api.semanticscholar.org/CorpusID:4498998

    2010

  71. [80]

    Kajikawa, E

    S. Kajikawa, E. Ishikawa, Trajectory planning for hand-over between human and robot, Proceedings 9th IEEE International Workshop on Robot and Human Interactive Communication. IEEE RO-MAN 2000 (Cat. No.00TH8499) (2000) 281–287. URL https://api.semanticscholar.org/CorpusID:60727860

    2000

  72. [81]

    N. D. Ratliff, J. Issac, D. Kappler, Riemannian motion policies, ArXiv abs/1801.02854 (2018). URL https://api.semanticscholar.org/CorpusID:3707764

    2018 arXiv

  73. [82]

    Y. I. Nakano, R. Ishii, Estimating user’s engagement from eye- gaze behaviors in human-agent conversations, in: Proceedings of the 15th international conference on Intelligent user inter- faces, 2010, pp. 139–148

    2010

  74. [83]

    Lugaresi, J

    C. Lugaresi, J. Tang, H. Nash, C. McClanahan, E. Uboweja, M. Hays, F. Zhang, C.-L. Chang, M. G. Yong, J. Lee, W.- T. Chang, W. Hua, M. Georg, M. Grundmann, Mediapipe: A framework for building perception pipelines, Third Workshop on Computer Vision for AR/VR at IEEE Computer Vi...

    2019 arXiv

  75. [84]

    Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, V. Stoyanov, Roberta: A robustly optimized bert pretraining approach, International Conference on Learning Representations (ICLR) (2019). arXiv:1907.11692

    2019 arXiv

  76. [85]

    T. Wu, S. He, J. Liu, S. Sun, K. Liu, Q.-L. Han, Y. Tang, A brief overview of chatgpt: The history, status quo and potential future development, IEEE/CAA Journal of Automatica Sinica 10 (5) (2023) 1122–1136

    2023

  77. [86]

    G. Team, R. Anil, S. Borgeaud, Y. Wu, J.-B. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauth, et al., Gemini: a family of highly capable multimodal models, arXiv preprint arXiv:2312.11805 (2023)

    2023 arXiv

  78. [87]

    S. Wu, M. Koo, L. Blum, A. Black, L. Kao, F. Scalzo, I. Kurtz, A comparative study of open-source large language models, gpt-4 and claude 2: Multiple-choice test taking in nephrology, NEJM AI Journal (2023)

    2023

  79. [88]

    Honnibal, I

    M. Honnibal, I. Montani, spaCy 2: Natural language under- standing with Bloom embeddings, convolutional neural net- works and incremental parsing, to appear (2017)

    2017

  80. [89]

    L. H. Li, P. Zhang, H. Zhang, J. Yang, C. Li, Y. Zhong, L. Wang, L. Yuan, L. Zhang, J.-N. Hwang, et al., Grounded language- image pre-training, in: Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, 2022, pp. 10965–10975

    2022

  81. [90]

    Z. L. Lin, X. Peng, P. Cong, Y. Hou, X. Zhu, S. Yang, Y. Ma, Wildrefer: 3d object localization in large-scale dynamic scenes with multi-modal visual data and natural language, in: Euro- pean Conference on Computer Vision, 2023. URL https://api.semanticscholar.org/CorpusID:258079297

    2023

  82. [91]

    Margffoy-Tuay, J

    E. Margffoy-Tuay, J. C. P´ erez, E. Botero, P. Arbel´ aez, Dy- namic multimodal instance segmentation guided by natural language queries, European Conference on Computer Vision (2018). arXiv:1807.02257

    2018 arXiv

  83. [92]

    X. Yu, Y. Rao, Z. Wang, Z. Liu, J. Lu, J. Zhou, Pointr: Di- verse point cloud completion with geometry-aware transform- ers, IEEE International Conference on Computer Vision (2021). arXiv:2108.08839

    2021 arXiv

  84. [93]

    Sundermeyer, A

    M. Sundermeyer, A. Mousavian, R. Triebel, D. Fox, Contact-graspnet: Efficient 6-dof grasp generation in clut- tered scenes, in: 2021 IEEE International Conference on Robotics and Automation (ICRA), 2021, pp. 13438–13444. doi:10.1109/ICRA48506.2021.9561877

    2021

  85. [94]

    Jiang, S

    H. Jiang, S. Liu, J. Wang, X. Wang, Hand-object con- tact consistency reasoning for human grasps genera- tion, in: 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 11087–11096. doi:10.1109/ICCV48922.2021.01092

    2021

  86. [95]

    Calli, A

    B. Calli, A. Singh, J. Bruce, A. Walsman, K. Kono- lige, S. Srinivasa, P. Abbeel, A. M. Dollar, Yale-cmu- berkeley dataset for robotic manipulation research, The In- ternational Journal of Robotics Research 36 (3) (2017) 261–268. arXiv:https://doi.org/10.1177/0278364917700714,...

    2017 doi

  87. [96]

    T. Zhu, R. Wu, X. Lin, Y. Sun, Toward human-like grasp: Dexterous grasping via semantic representation of object-hand, 2021 IEEE/CVF International Conference on Computer Vision (ICCV) (2021) 15721–15731. URL https://api.semanticscholar.org/CorpusID:244468621

    2021

  88. [97]

    Y. Lin, Y. Sun, Robot grasp planning based on demonstrated grasp strategies, The International Journal of Robotics Re- search 34 (2015) 26 – 42. URL https://api.semanticscholar.org/CorpusID:10178250 17

    2015

  89. [166]

    doi:10.1109/RO-MAN50785.2021.9515402

    2021

  90. [324]

    doi:10.1109/ROMAN.2015.7333626

    2015

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.