REVIEW 4 cited by
Self-Explainable Affordance Learning with Embodied Caption
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
In the field of visual affordance learning, previous methods mainly used abundant images or videos that delineate human behavior patterns to identify action possibility regions for object manipulation, with a variety of applications in robotic tasks. However, they encounter a main challenge of action ambiguity, illustrated by the vagueness like whether to beat or carry a drum, and the complexities involved in processing intricate scenes. Moreover, it is important for human intervention to rectify robot errors in time. To address these issues, we introduce Self-Explainable Affordance learning (SEA) with embodied caption. This innovation enables robots to articulate their intentions and bridge the gap between explainable vision-language caption and visual affordance learning. Due to a lack of appropriate dataset, we unveil a pioneering dataset and metrics tailored for this task, which integrates images, heatmaps, and embodied captions. Furthermore, we propose a novel model to effectively combine affordance grounding with self-explanation in a simple but efficient manner. Extensive quantitative and qualitative experiments demonstrate our method's effectiveness.
Forward citations
Cited by 4 Pith papers
-
Exo2EgoPose: Leveraging Exocentric Demonstrations for Vision-Language guided Egocentric 3D Hand Pose Forecasting
A transformer-based hand-pose forecaster that reconstructs exocentric video features at video and frame levels from egocentric inputs, then uses them to modulate egocentric pose queries, beating re-implemented baselin...
-
Weakly-Supervised Affordance Grounding Guided by Part-Level Semantic Priors
A pseudo-supervised pipeline with an affordance-to-part mapping, label refinement, cross-view alignment, and a reasoning module achieves state-of-the-art weakly supervised affordance grounding on AGD20K.
-
Context-Dependent Affordance Computation in Vision-Language Models
Seven different agent personas made a vision-language model describe the same COCO image with under 10% lexical overlap, which the paper interprets as >90% context-dependent affordance computation.
-
Bridging Perspectives: A Survey on Cross-view Collaborative Intelligence with Egocentric-Exocentric Vision
A comprehensive review of cross-view video understanding that uses both first-person and third-person cameras, organized into a three-direction taxonomy with a dataset catalog and future research gaps.
Discussion (0). Sign in to comment.