REVIEW 6 cited by
Reasoning Grasping via Multimodal Large Language Model
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Despite significant progress in robotic systems for operation within human-centric environments, existing models still heavily rely on explicit human commands to identify and manipulate specific objects. This limits their effectiveness in environments where understanding and acting on implicit human intentions are crucial. In this study, we introduce a novel task: reasoning grasping, where robots need to generate grasp poses based on indirect verbal instructions or intentions. To accomplish this, we propose an end-to-end reasoning grasping model that integrates a multimodal Large Language Model (LLM) with a vision-based robotic grasping framework. In addition, we present the first reasoning grasping benchmark dataset generated from the GraspNet-1 billion, incorporating implicit instructions for object-level and part-level grasping. Our results show that directly integrating CLIP or LLaVA with the grasp detection model performs poorly on the challenging reasoning grasping tasks, while our proposed model demonstrates significantly enhanced performance both in the reasoning grasping benchmark and real-world experiments.
Forward citations
Cited by 6 Pith papers
-
Beyond Description: Cognitively Benchmarking Fine-Grained Action for Embodied Agents
CFG-Bench uses 19,562 questions across four cognitive tiers to show that vision-language models are weak at fine-grained physical action understanding, and that SFT on its data improves scores on external embodied benchmarks.
-
Boosting Embodied AI Agents through Perception-Generation Disaggregation and Asynchronous Pipeline Execution
Auras, a perception-generation disaggregation framework with a public context buffer and asynchronous pipeline executor, raises embodied-agent throughput by 2.54x on average without losing accuracy (102.7%).
-
Attribute-based Object Grounding and Robot Grasp Detection with Spatial Reasoning
A bidirectional vision-language-depth fusion model, OGRG, outperforms prior baselines in grounding and grasping objects described by spatial language, including with duplicate objects and weak grasp labels.
-
GraspMAS: Zero-Shot Language-driven Grasp Detection with Multi-Agent System
A multi-agent system with planner, coder, and observer agents achieves zero-shot language-driven grasp detection that outperforms existing baselines on benchmarks and robots.
-
Transforming Remanufacturing Automation with Large Language Models: A Forward-Looking Analysis with Case Studies
The authors propose ReManGPT, a conceptual orchestration framework for applying LLMs to remanufacturing, and illustrate it with case studies in disassembly planning, repair guidance, and robotic execution.
-
Foundation Model Driven Robotics: A Comprehensive Review
A review of foundation-model-driven robotics that synthesizes recent work across perception, planning, control, HRI, simulation, and sim-to-real transfer, and highlights open challenges.
Discussion (0). Sign in to comment.