Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

GREAT: Geometry-Intention Collaborative Inference for Open-Vocabulary 3D Object Affordance Grounding

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Mining invariant geometry and interaction intentions lets a model ground 3D object affordances for categories never seen in training.

desk verdict A solid method-and-dataset contribution whose open-vocabulary claim is plausible but needs external benchmarks and direct checks of the MLLM's reasoning outputs before the SOTA claim carries weight. read the letter →

arxiv 2411.19626 v2 pith:P2NOY5PS submitted 2024-11-29 cs.CV cs.AI

Yawen Shao , Wei Zhai , Yuhang Yang , Hongchen Luo , Yang Cao , Zheng-Jun Zha This is my paper · ORCID
classification cs.CVcs.AI
keywords 3Dobjectaffordancegroundingopen-vocabularychain-of-thoughtreasoningmultimodallargelanguagemodelscross-modalfusionpointcloudPIADv2datasetgeneralization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

GREAT addresses open-vocabulary 3D object affordance grounding: given a 3D point cloud and a 2D interaction image, locate the region that supports an arbitrary action. The paper's central claim is that the bottleneck is not cross-modal alignment but the semantic space of interaction priors, and that this space can be expanded by having a multimodal language model reason in steps about which part interacts, why the geometry supports the interaction, what action is shown, and what other actions the object affords. GREAT implements this reasoning with a Multi-Head Affordance Chain-of-Thought fine-tuned on interaction images, then fuses the resulting text knowledge with point cloud and image features to predict point-wise affordance heatmaps. The paper also introduces PIADv2, a dataset with 15K interaction images and more than 38K point clouds covering 24 affordances and 43 object categories, and reports that GREAT outperforms prior methods on seen, unseen-object, and unseen-affordance partitions. This matters because it is a route toward robots that can reason about how to use objects they were not explicitly trained on.

What carries the argument

The load-bearing component is the Multi-Head Affordance Chain-of-Thought (MHACoT): a fine-tuned multimodal large language model prompted through four steps: object interaction perception, geometric structure reasoning, interaction detailed description, and interactive analogical reasoning. The object head's first two prompts extract which part interacts and why its geometry permits the interaction; the affordance head's last two prompts extract the shown interaction and brainstorm two additional interactions. These outputs are encoded, cross-attended to align geometry with intention, and compressed into two knowledge dictionaries. The Cross-Modal Adaptive Fusion Module (CMAFM) then injects the geometric dictionary into the deepest point cloud encoder layer and fuses the intention dictionary with image features, so the decoder sees point geometry and interaction context together. The mechanism's job is to make unseen affordances reachable through known geometric primitives and analogical interaction intentions instead of through memorized label mappings.

What would settle it

Run GREAT on unseen-affordance test images and compare the MLLM's four chain-of-thought answers against human annotations of the interaction part, its geometry, the shown action, and the alternative interactions. If the generated text is frequently wrong or generic for unseen affordances while grounding accuracy stays high, the chain-of-thought is not the mechanism carrying the gain; alternatively, if substituting fixed random text for the chain-of-thought answers preserves unseen-affordance AUC, then the chain-of-thought content is not the active ingredient.

Watch

Extended reading notes

Core claim

The discovery GREAT argues for is that explicitly separating geometric-structure reasoning from interaction-intention reasoning, and letting the two heads cross-reference each other, transfers to affordances whose categories were absent from training. The object head identifies the interacting part and explains its geometry; the affordance head describes the shown interaction and brainstorms two additional plausible interactions. After encoding these descriptions and aligning them via cross- and self-attention into two knowledge features, GREAT injects the geometric knowledge into point cloud features and the intention knowledge into image features, then decodes the fused features into a point-wise affordance map. On the unseen-affordance partition this design reaches an AUC of 69.81 and an aIOU of 12.05, outperforming the strongest baseline by 5.74 AUC and 3.68 aIOU. The authors present this as evidence that invariant geometry plus analogical intention reasoning, rather than a wider vocabulary of labels, is what enables open-vocabulary affordance grounding.

Load-bearing premise

The MLLM's chain-of-thought, fine-tuned on PIADv2 training images, produces accurate geometric attributes and interaction intentions for affordance categories never seen in training; if that reasoning is unreliable or hallucinated, the open-vocabulary gain the paper reports would not transfer.

Editorial extensions

If this is right

  • A robot controller using this pipeline could hypothesize an interaction part and a manipulation strategy for an object category it has never seen, as long as the object's geometry resembles shapes whose affordances were described during training.
  • Reasoning output is not tied to a fixed label vocabulary: the same framework can accept open-ended instructions and produce heatmaps for interactions beyond the 24 affordance categories in PIADv2.
  • The dataset's three partitions turn open-vocabulary affordance grounding into a measurable benchmark, so future methods can be compared on generalization rather than only on seen-category accuracy.
  • Removing either chain-of-thought head degrades performance, with the larger drop coming from removing the affordance head on unseen affordances, so both geometry and intention reasoning contribute to the open-vocabulary gain.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An untested implication is that the four-step reasoning text itself is the active ingredient: one could replace the MLLM's answers with the same template filled with random or generic text while keeping fusion fixed, and check whether unseen-affordance accuracy drops.
  • The geometry-intention decomposition could transfer to robotic grasping-point selection or tool-use planning, where the geometric step names the contact region and the intention step filters candidate actions; that application is outside the paper's experiments.
  • Because the authors identify multi-step inference cost as the bottleneck, distilling the MLLM's chain-of-thought into a small text encoder or a lightweight reasoning head is the most direct path toward real-time use, and PIADv2's scale would support such distillation.
  • The multi-to-multi relation between affordances and object categories in PIADv2 suggests the dataset could support compositional or few-shot splits beyond the three partitions explored here.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes GREAT, a framework for open-vocabulary 3D object affordance grounding. Given a point cloud and an interaction image, GREAT first fine-tunes an MLLM (InternVL) with a Multi-Head Affordance Chain-of-Thought (MHACoT) procedure to produce object geometric attributes and underlying interaction intentions. These textual outputs are encoded with RoBERTa and aggregated into two knowledge dictionaries, which are then fused into point-cloud features and image features through a Cross-Modal Adaptive Fusion Module. The final fused features are decoded into a point-wise affordance heatmap. The paper also introduces PIADv2, a large dataset of paired interaction images and 3D point clouds, and reports experiments on three partitions (Seen, Unseen Object, Unseen Affordance) showing substantial gains over reimplemented baselines. Ablations demonstrate the contribution of each component, and qualitative visualizations support the method's behavior.

Significance. If the reported results are sound, the paper makes a useful contribution to 3D affordance grounding by showing that MLLM-based chain-of-thought reasoning can provide transferable knowledge for unseen affordances. The proposed dataset, PIADv2, is substantially larger than prior benchmarks and could benefit future research. The paper also provides code and dataset release, and the ablations are fairly comprehensive. However, the significance is currently bounded by three limitations: all quantitative evaluation is on the authors' own dataset, the MLLM reasoning outputs are never directly validated for correctness or hallucination, and the fine-tuning data for the MLLM is not described in sufficient detail. These gaps make the central open-vocabulary claim less certain than the paper's wording suggests.

major comments (4)
  1. [Sec. 3.2] The fine-tuning of InternVL is described only in terms of adapters, epochs, learning rate, and LoRA rank; the paper does not state what data the MHACoT prompt-answer pairs come from, whether the answers are human-annotated or MLLM-generated, or whether any of those training pairs involve affordance categories or object categories that appear in the Unseen Affordance or Unseen Object test partitions. This matters because the method's open-vocabulary capability is attributed to this fine-tuned MLLM, and any overlap between the fine-tuning set and the evaluation partitions would be a form of leakage. Please specify the construction, size, and partition membership of the fine-tuning data, and verify that it is disjoint from the evaluation splits.
  2. [Sec. 5.2 and Table 2] All quantitative comparisons are carried out exclusively on PIADv2, a dataset introduced in the same paper, and the baselines are reimplemented with 'the same feature extractor and settings' rather than taken from official reported results. The claim of state-of-the-art performance therefore lacks external anchoring: a reader cannot tell whether the gains reflect the method's superiority or favorable properties of the new dataset and splits. Please add at least one external benchmark (for example, evaluation on the original PIAD or on 3D AffordanceNet) or, at minimum, report official baseline numbers and cross-dataset transfer experiments. Without such evidence, the SOTA claim is not fully supported.
  3. [Sec. 3.2, Sec. 5.3, Table 3] The paper states that the multi-step chain-of-thought strategy 'eliminates the hallucination and ambiguities raised by MLLMs when reasoning interactions,' but no direct measurement of MLLM output quality is reported. There is no hallucination rate, no accuracy metric for the predicted geometric attributes or interaction intentions, and no failure-case analysis, especially for affordance categories unseen in training. The Table 3 ablations show that removing the affordance head lowers Unseen Affordance AUC from 69.81 to 67.18 and removing the object head lowers it to 64.69; these deltas are modest and could stem from changes in the text-feature distribution or from reduced auxiliary supervision rather than from the factual correctness of the extracted knowledge. Please report direct evaluations of the MLLM-generated knowledge (e.g., human-rated accuracy or automatic comparison against annotations) and show representative successes and failures for the unseen-affordance setting.
  4. [Sec. 4, Data Partitions] The definition of the Unseen Affordance partition is under-specified. The paper says only that 'affordances in the test set are not present in the training set, and so does certain objects,' but it does not clarify whether the interaction images and the fine-tuned MLLM training data also exclude those affordances, nor how object categories and image sources are controlled across partitions. If, for example, images depicting 'pour' are present in the MLLM fine-tuning set even though the affordance is absent from the main training set, the open-vocabulary evaluation would be confounded. Please provide the exact partition construction rules and a per-category overlap analysis between training, fine-tuning, and test sets.
minor comments (5)
  1. [Sec. 4] The annotation description says 'each affordance category possesses such annotation of an instance,' which is ambiguous. Please clarify whether each point-cloud instance is annotated with one affordance heatmap per affordance category or only with the affordance category depicted in the paired image.
  2. [Sec. 5.3 and Table 3] The ablation labeled '✗ CMAFM' is not described in detail; please specify what replaces the Cross-Modal Adaptive Fusion Module when it is removed (e.g., simple concatenation or a fixed cross-attention layer).
  3. [Sec. 5.2 and Table 2] The phrase 'significantly outperforms' is used without statistical support. Reporting standard deviations across multiple seeds or a significance test would make the comparison more convincing.
  4. [Sec. 3.2] The full MHACoT prompts are said to be in the appendix, but the appendix is not included in the reviewed manuscript. Please ensure the full prompts are part of the submission or provide them in the main text.
  5. [Sec. 3.2, Eq. (1)] The cross-attention operation fm and the self-attention operation fδ are not defined in terms of their inputs and output shapes. In particular, when To and Ta have different sequence lengths, the implementation of Eq. (1) should be clarified so that the output dimensions of ¯To and ¯Ta are unambiguous.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: the pipeline is a standard supervised image-point-cloud fusion; the only adjacent issues are a self-constructed benchmark and unmeasured MLLM hallucination claims, neither of which reduces a prediction to its inputs.

full rationale

The claimed derivation chain is explicit and non-circular. Inputs P and I are processed by PointNet++/ResNet into Fp, Fi; MHACoT fine-tunes InternVL on training interaction images to produce textual geometry and intention knowledge, encoded by RoBERTa as To, Ta; Eq. 1 aligns them via cross-attention; Eqs. 2-4 fuse these knowledge features into point and image features; Eq. 5 decodes the fused features into affordance heatmaps; Eq. 6 supervises with ground-truth point heatmaps. No equation uses test labels as an input or defines a predicted quantity as a fitted parameter renamed as a result. The MLLM's CoT outputs at test time are not the PIADv2 affordance annotations; they are derived from an externally pretrained model and from prompts, so the open-vocabulary claim rests on transfer rather than on an identity with the benchmark. The main caveats are benchmark-level, not derivation-level: PIADv2 and its Seen/Unseen partitions are constructed and partitioned by the same authors, and the assertion that multi-step reasoning 'eliminates the hallucination and ambiguities raised by MLLMs' (Sec. 3.2) is not supported by any hallucination-rate or failure-case measurement. These are evidence-quality concerns, and the self-citations to the authors' own PIAD/IAGNet work are not load-bearing for the mathematical derivation. Under the stated rules, an unsupported but non-circular assumption should be scored as low circularity, not as a definitional reduction.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central approach rests on the reliability of MLLM-generated affordance knowledge and the quality of a self-built dataset. Neither is independently verified. Hyperparameters are standard but unreported in terms of sensitivity.

free parameters (4)
  • LoRA rank for MLLM fine-tuning = 16
    Chosen hyperparameter; not swept in the paper, affects MLLM adaptation quality.
  • MLLM fine-tuning learning rate = 4e-5
    Chosen for adapter training; not ablated.
  • Main model learning rate = 1e-4
    Chosen for GREAT training; not ablated.
  • Training epochs = 65 (GREAT), 10 (MLLM)
    Stopping points chosen by the authors; no convergence analysis.
assumptions (4)
  • domain assumption InternVL with LoRA fine-tuning produces accurate and generalizable geometric attributes and interaction intentions from interaction images.
    Relies on the MLLM's world knowledge and the quality of the fine-tuning; this is the core enabler of the open-vocabulary behavior (Sec. 3.2, ablation Tab. 3).
  • domain assumption RoBERTa text embeddings preserve the affordance-relevant semantics of the MHACoT outputs.
    The knowledge dictionaries To and Ta are derived from RoBERTa embeddings; if the text encoder discards geometric or interactive cues, the fusion cannot recover them (Sec. 3.2).
  • domain assumption PIADv2 affordance annotations and the three data partitions are clean and correctly constructed, with no category leakage between splits.
    All evaluation is on this self-built dataset; annotation and partition quality are not externally verified (Sec. 4).
  • domain assumption PointNet++ and ResNet18 provide sufficient 3D geometry and 2D image representations for affordance grounding.
    The framework does not pretrain or adapt these encoders for affordance; their capacity is taken as given (Sec. 3.1).

how reviews work

0 comments
Cite this review

Pith. "Pith review of GREAT: Geometry-Intention Collaborative Inference for Open-Vocabulary 3D Object Affordance Grounding." pith.science (2026). https://pith.science/paper/P2NOY5PS

@misc{pith2026241119626,
  author       = {Pith},
  title        = {Pith review of: GREAT: Geometry-Intention Collaborative Inference for Open-Vocabulary 3D Object Affordance Grounding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/P2NOY5PS}},
  note         = {Machine review of arXiv:2411.19626}
}
read the original abstract

Open-Vocabulary 3D object affordance grounding aims to anticipate ``action possibilities'' regions on 3D objects with arbitrary instructions, which is crucial for robots to generically perceive real scenarios and respond to operational changes. Existing methods focus on combining images or languages that depict interactions with 3D geometries to introduce external interaction priors. However, they are still vulnerable to a limited semantic space by failing to leverage implied invariant geometries and potential interaction intentions. Normally, humans address complex tasks through multi-step reasoning and respond to diverse situations by leveraging associative and analogical thinking. In light of this, we propose GREAT (GeometRy-intEntion collAboraTive inference) for Open-Vocabulary 3D Object Affordance Grounding, a novel framework that mines the object invariant geometry attributes and performs analogically reason in potential interaction scenarios to form affordance knowledge, fully combining the knowledge with both geometries and visual contents to ground 3D object affordance. Besides, we introduce the Point Image Affordance Dataset v2 (PIADv2), the largest 3D object affordance dataset at present to support the task. Extensive experiments demonstrate the effectiveness and superiority of GREAT. The code and dataset are available at https://yawen-shao.github.io/GREAT/.

Figures

Figures reproduced from arXiv: 2411.19626 by the authors.

Figure 1
Figure 1. Difference and Motivation. (a) Object affordance grounding on seen setting. (b) Open-Vocabulary Affordance Grounding (OVAG) with previous paradigms. (c) Observing interaction images, people engage in brainstorming through memory representations, drawing on prior interaction experiences to perform analogical reasoning and infer appropriate actions. (d) OVAG with our geometry￾intention collaborative inference with cha… view at source ↗
Figure 2
Figure 2. GREAT pipeline. Initially, it extracts the respective features Fi, Fp through modality-specific backbones, then results of MHACoT inference are encoded and aggregated to form object/affordance knowledge features T¯ o, T¯ a (Sec. 3.2). Next, GREAT utilizes CMAFM to inject knowledge into Fp and Fi is directly fused to obtain fusion features Ftp, Fti (Sec. 3.3). Eventually, Ftp and Fti are sent to the decoder to ground… view at source ↗
Figure 3
Figure 3. PIADv2 Dataset. (a) Extensive data examples from PIADv2, the red region in point clouds is the affordance annotation. (b) Category distribution in PIADv2. (c) Confusion matrix between affordance and object categories, where the horizontal axis represents object category and the vertical axis represents affordance category. (d) Ratio of images and point clouds in each object category. map, enabling effective cross-mo… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Visualization Results. The first row is the interaction image and the last row is the ground truth of 3D object affordance in point cloud. The left-middle-right partitions correspond to the visual comparison results for different 3D object affordance in the Seen, Unsee…
Figure 7
Figure 7. Figure 7: The results demonstrate that, for the same object, the model outputs different results depending on the inter￾action, and the localized 3D object affordance regions are consistent with the interactions depicted in the 2D images. Multiple Instances. To assess the genera…
Figure 8
Figure 8. Figure 8: Multiple Instances. (a) Similar geometric instances. (b) Different geometric instances. 6. Conclusion We present grounding 3D object affordance in an open￾vocabulary fashion, which reasons from interaction images, extrapolating from predefined sample space and general￾…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SeqAfford: Sequential 3D Affordance Reasoning via Multimodal Large Language Model

    cs.CV 2024-12 conditional novelty 7.0 of 10

    SeqAfford combines a 3D multimodal large language model with special segmentation tokens to decompose complex instructions into ordered affordance masks, and the authors present a 180K-pair benchmark for this task.

Reference graph

Works this paper leans on

68 extracted references · 49 canonical work pages · cited by 1 Pith paper

  1. [1]

    Cross- modal learning for image-guided point cloud shape comple- tion

    Emanuele Aiello, Diego Valsesia, and Enrico Magli. Cross- modal learning for image-guided point cloud shape comple- tion. In Advances in Neural Information Processing Systems,

    work page

  2. [2]

    Partafford: Part-level affordance discov- ery from 3d objects

    Xu Chao, Yixin Chen, He Wang, songchun Zhu, Yixin Zhu, and Siyuan Huang. Partafford: Part-level affordance discov- ery from 3d objects. arXiv preprint arXiv:2202.13519, 2022. 1

    arXiv 2022

  3. [3]

    Worldaf- ford: Affordance grounding based on natural language in- structions

    Changmao Chen, Yuren Cong, and Zhen Kan. Worldaf- ford: Affordance grounding based on natural language in- structions. arXiv preprint arXiv:2405.12461, 2024. 3

    arXiv 2024

  4. [4]

    How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites

    Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhang- wei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites. arXiv preprint arXiv:2404.16821, 2024. 1, 3

    arXiv 2024

  5. [5]

    Obja- verse: A universe of annotated 3d objects

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Obja- verse: A universe of annotated 3d objects. arXiv preprint arXiv:2212.08051, 2022. 3, 5

    arXiv 2022

  6. [6]

    3d affordancenet: A benchmark for visual object affordance understanding

    Shengheng Deng, Xun Xu, Chaozheng Wu, Ke Chen, and Kui Jia. 3d affordancenet: A benchmark for visual object affordance understanding. In Proceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition , 2021. 1, 2, 3, 5

    work page 2021

  7. [7]

    Gick and Keith J

    Mary L. Gick and Keith J. Holyoak. Analogical problem solving. Cognitive Psychology, 12(3):306–355, 1980. 1

    work page 1980

  8. [8]

    Vi- sual affordance and function understanding

    Mohammed Hassanin, Salman Khan, and Murat Tahtali. Vi- sual affordance and function understanding. ACM Comput- ing Surveys (CSUR), 54:1 – 35, 2018. 1

    work page 2018

Show all 68 references
  1. [9]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, 2016. 3, 6

    2016

  2. [10]

    LoRA: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. InIn- ternational Conference on Learning Representations , 2022. 3

    2022

  3. [11]

    Diffusion- based generation, optimization, and planning in 3d scenes

    Siyuan Huang, Zan Wang, Puhao Li, Baoxiong Jia, Tengyu Liu, Yixin Zhu, Wei Liang, and Song-Chun Zhu. Diffusion- based generation, optimization, and planning in 3d scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 1

    2023

  4. [12]

    Rekep: Spatio-temporal reasoning of rela- tional keypoint constraints for robotic manipulation

    Wenlong Huang, Chen Wang, Yunzhu Li, Ruohan Zhang, and Li Fei-Fei. Rekep: Spatio-temporal reasoning of rela- tional keypoint constraints for robotic manipulation. arXiv preprint arXiv:2409.01652, 2024. 1

    2024 arXiv

  5. [13]

    Imitation learning: A survey of learning methods

    Ahmed Hussein, Mohamed Medhat Gaber, Eyad Elyan, and Chrisina Jayne. Imitation learning: A survey of learning methods. ACM Comput. Surv., 50(2), 2017. 1

    2017

  6. [14]

    The ecological approach to visual percep- tion: classic edition

    James J Gibson. The ecological approach to visual percep- tion: classic edition. Psychology press, 2014. 1

    2014

  7. [15]

    Visionllm v2: An end-to-end generalist multimodal large language model for hundreds of vision-language tasks

    Wu Jiannan, Zhong Muyan, Xing Sen, Lai Zeqiang, Liu Zhaoyang, Chen Zhe, Wang Wenhai, Zhu Xizhou, Lu Lewei, Lu Tong, Luo Ping, Qiao Yu, and Dai Jifeng. Visionllm v2: An end-to-end generalist multimodal large language model for hundreds of vision-language tasks. arXiv preprint a...

    2024 arXiv

  8. [16]

    How We Reason

    Philip Johnson-Laird. How We Reason . Oxford University Press, 2008. 1

    2008

  9. [17]

    Sukhatme

    David Inkyu Kim and Gaurav S. Sukhatme. Semantic la- beling of 3d point clouds with object affordance for robot manipulation. In 2014 IEEE International Conference on Robotics and Automation (ICRA) , pages 5578–5584, 2014. 1

    2014

  10. [18]

    Openimages: A public dataset for large-scale multi- label and multi-class image classification

    Ivan Krasin, Tom Duerig, Neil Alldrin, Vittorio Ferrari, Sami Abu-El-Haija, Alina Kuznetsova, Hassan Rom, Jasper Ui- jlings, Stefan Popov, Andreas Veit, Serge Belongie, Vic- tor Gomes, Abhinav Gupta, Chen Sun, Gal Chechik, David Cai, Zheyun Feng, Dhyanesh Narayanan, and Kevin ...

    2017

  11. [19]

    Ram: Retrieval-based affordance transfer for generalizable zero-shot robotic manipulation, 2024

    Yuxuan Kuang, Junjie Ye, Haoran Geng, Jiageng Mao, Con- gyue Deng, Leonidas Guibas, He Wang, and Yue Wang. Ram: Retrieval-based affordance transfer for generalizable zero-shot robotic manipulation, 2024. 1

    2024

  12. [20]

    Boosting logical reasoning in large language models through a new framework: The graph of thought, 2023

    Bin Lei, pei Hung Lin, Chunhua Liao, and Caiwen Ding. Boosting logical reasoning in large language models through a new framework: The graph of thought, 2023. 3

    2023

  13. [21]

    One-shot open affordance learning with foundation models

    Gen Li, Deqing Sun, Laura Sevilla-Lara, and Varun Jampani. One-shot open affordance learning with foundation models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. 3

    2024

  14. [22]

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation, 2022

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation, 2022. 1

    2022

  15. [23]

    Ag2manip: Learning novel manipulation skills with agent- agnostic visual and action representations

    Puhao Li, Tengyu Liu, Yuyang Li, Muzhi Han, Haoran Geng, Shu Wang, Yixin Zhu, Song-Chun Zhu, and Siyuan Huang. Ag2manip: Learning novel manipulation skills with agent- agnostic visual and action representations. arXiv preprint arXiv:2404.17521, 2024. 1

    2024 arXiv

  16. [24]

    Manipllm: Embodied multimodal large language model for object-centric robotic manipulation

    Xiaoqi Li, Mingxu Zhang, Yiran Geng, Haoran Geng, Yux- ing Long, Yan Shen, Renrui Zhang, Jiaming Liu, and Hao Dong. Manipllm: Embodied multimodal large language model for object-centric robotic manipulation. In CVPR,

  17. [25]

    Y . Li, N. Zhao, J. Xiao, C. Feng, X. Wang, and T. Chua. Laso: Language-guided affordance segmentation on 3d ob- ject. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 1, 2, 3, 6, 7

    2024

  18. [26]

    Focal loss for dense object detection

    Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence , 42(2):318–327, 2020. 5

    2020

  19. [27]

    Grounding 3d scene affordance from egocentric interactions

    Cuiyu Liu, Wei Zhai, Yuhang Yang, Hongchen Luo, Sen Liang, Yang Cao, and Zheng-Jun Zha. Grounding 3d scene affordance from egocentric interactions. arXiv preprint arXiv:2409.19650, 2024. 1

    2024 arXiv

  20. [28]

    Roberta: A robustly optimized bert pretraining approach, 2019

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettle- moyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach, 2019. 3, 4

    2019

  21. [29]

    Lobo, Alberto Jiménez-Valverde, and Raimundo Real

    Jorge M. Lobo, Alberto Jiménez-Valverde, and Raimundo Real. Auc: a misleading measure of the performance of pre- dictive distribution models. Global Ecology and Biogeogra- phy, 17:145–151, 2008. 6

    2008

  22. [30]

    Phrase-based affordance detection via cyclic bilateral inter- action

    Liangsheng Lu, Wei Zhai, Hongchen Luo, and Yang Cao. Phrase-based affordance detection via cyclic bilateral inter- action. 2022. 3

    2022

  23. [31]

    One-shot affordance detection

    Hongchen Luo, Wei Zhai, Jing Zhang, Yang Cao, and Dacheng Tao. One-shot affordance detection. In IJCAI,

  24. [32]

    Learning affordance grounding from exocen- tric images

    Hongchen Luo, Wei Zhai, Jing Zhang, Yang Cao, and Dacheng Tao. Learning affordance grounding from exocen- tric images. In CVPR, 2022. 5

    2022

  25. [33]

    Leverage interactive affinity for affordance learning

    Hongchen Luo, Wei Zhai, Jing Zhang, Yang Cao, and Dacheng Tao. Leverage interactive affinity for affordance learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 6809–6819, 2023. 3

    2023

  26. [34]

    Learning visual affordance grounding from demonstration videos

    Hongchen Luo, Wei Zhai, Jing Zhang, Yang Cao, and Dacheng Tao. Learning visual affordance grounding from demonstration videos. IEEE Transactions on Neural Net- works and Learning Systems , 35(11):16857–16871, 2024. 3

    2024

  27. [35]

    Madiff: Motion-aware mamba diffusion models for hand trajectory prediction on egocentric videos, 2024

    Junyi Ma, Xieyuanli Chen, Wentao Bao, Jingyi Xu, and Hes- heng Wang. Madiff: Motion-aware mamba diffusion models for hand trajectory prediction on egocentric videos, 2024. 1

    2024

  28. [36]

    Gpt-driver: Learning to drive with gpt, 2023

    Jiageng Mao, Yuxi Qian, Junjie Ye, Hang Zhao, and Yue Wang. Gpt-driver: Learning to drive with gpt, 2023. 1

    2023

  29. [37]

    V-net: Fully convolutional neural networks for volumetric medical image segmentation

    Fausto Milletari, Nassir Navab, and Seyed-Ahmad Ahmadi. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In 2016 F ourth International Conference on 3D Vision (3DV), pages 565–571, 2016. 5

    2016

  30. [38]

    Chang, Li Yi, Subarna Tripathi, Leonidas J

    Kaichun Mo, Shilin Zhu, Angel X. Chang, Li Yi, Subarna Tripathi, Leonidas J. Guibas, and Hao Su. PartNet: A large- scale benchmark for fine-grained and hierarchical part-level 3D object understanding. In The IEEE Conference on Com- puter Vision and Pattern Recognition (CVPR), 2019. 3

    2019

  31. [39]

    O2O-Afford: Annotation-free large-scale object-object affordance learning

    Kaichun Mo, Yuzhe Qin, Fanbo Xiang, Hao Su, and Leonidas Guibas. O2O-Afford: Annotation-free large-scale object-object affordance learning. In Conference on Robot Learning (CoRL), 2021. 3

    2021

  32. [40]

    Learning relational affor- dance models for robots in multi-object manipulation tasks

    Bogdan Moldovan, Plinio Moreno, Martijn van Otterlo, José Santos-Victor, and Luc De Raedt. Learning relational affor- dance models for robots in multi-object manipulation tasks. In 2012 IEEE International Conference on Robotics and Au- tomation, pages 4373–4378, 2012. 1

    2012

  33. [41]

    Open-vocabulary af- fordance detection in 3d point clouds

    Toan Nguyen, Minh Nhat Vu, An Vuong, Dzung Nguyen, Thieu V o, Ngan Le, and Anh Nguyen. Open-vocabulary af- fordance detection in 3d point clouds. 2023. 1, 2, 3

    2023

  34. [42]

    Global-local collab- orative inference with llm for lidar-based open-vocabulary detection, 2024

    Xingyu Peng, Yan Bai, Chen Gao, Lirong Yang, Fei Xia, Beipeng Mu, Xiaofei Wang, and Si Liu. Global-local collab- orative inference with llm for lidar-based open-vocabulary detection, 2024. 1, 3

    2024

  35. [43]

    Point- net++: Deep hierarchical feature learning on point sets in a metric space

    Charles R Qi, Li Yi, Hao Su, and Leonidas J Guibas. Point- net++: Deep hierarchical feature learning on point sets in a metric space. arXiv preprint arXiv:1706.02413, 2017. 3, 4, 5, 6

    2017 arXiv

  36. [44]

    Affordancellm: Grounding affordance from vision language models

    Shengyi Qian, Weifeng Chen, Min Bai, Xiong Zhou, Zhuowen Tu, and Li Erran Li. Affordancellm: Grounding affordance from vision language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7587–7597, 2024. 3

    2024

  37. [45]

    Optimizing intersection-over-union in deep neural networks for image segmentation

    Md.Atiqur Rahman and Yang Wang. Optimizing intersection-over-union in deep neural networks for image segmentation. In International Symposium on Visual Com- puting, 2016. 6

    2016

  38. [46]

    Morley Mao

    Jiachen Sun, Qingzhao Zhang, Bhavya Kailkhura, Zhiding Yu, Chaowei Xiao, and Z. Morley Mao. Benchmarking ro- bustness of 3d point cloud recognition against common cor- ruptions. arXiv preprint arXiv:2201.12296, 2022. 3

    2022 arXiv

  39. [47]

    Swain and Dana H

    Michael J. Swain and Dana H. Ballard. Color indexing. In- ternational Journal of Computer Vision, 7:11–32, 1991. 6

    1991

  40. [48]

    Cot- det: Affordance knowledge prompting for task driven object detection

    Jiajin Tang, Ge Zheng, Jingyi Yu, and Sibei Yang. Cot- det: Affordance knowledge prompting for task driven object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pages 3068–3078,

  41. [49]

    Llama: Open and efficient foundation lan- guage models, 2023

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aure- lien Rodriguez, Armand Joulin, Edouard Grave, and Guil- laume Lample. Llama: Open and efficient foundation lan-...

    2023

  42. [50]

    Open-vocabulary affordance detection using knowledge distillation and text- point correlation

    Tuan Van V o, Minh Nhat Vu, Baoru Huang, Toan Nguyen, Ngan Le, Thieu V o, and Anh Nguyen. Open-vocabulary affordance detection using knowledge distillation and text- point correlation. 2024. 1, 2, 3

    2024

  43. [51]

    Chi, Quoc Le, and Denny Zhou

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed H. Chi, Quoc Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language mod- els. CoRR, abs/2201.11903, 2022. 3

    2022 arXiv

  44. [52]

    Willmott and Kenji Matsuura

    Cort J. Willmott and Kenji Matsuura. Advantages of the mean absolute error (mae) over the root mean square error (rmse) in assessing average model performance.Climate Re- search, 30:79–82, 2005. 6

    2005

  45. [53]

    Det- toolchain: A new prompting paradigm to unleash detection ability of mllm, 2024

    Yixuan Wu, Yizhou Wang, Shixiang Tang, Wenhao Wu, Tong He, Wanli Ouyang, Philip Torr, and Jian Wu. Det- toolchain: A new prompting paradigm to unleash detection ability of mllm, 2024. 3

    2024

  46. [54]

    Fusionrcnn: Lidar- camera fusion for two-stage 3d object detection

    Xinli Xu, Shaocong Dong, Tingfa Xu, Lihe Ding, Jie Wang, Peng Jiang, Liqiang Song, and Jianan Li. Fusionrcnn: Lidar- camera fusion for two-stage 3d object detection. arXiv preprint arXiv:2209.10733, 2022. 6

    2022 arXiv

  47. [55]

    Kptllm: Unveiling the power of large language model for keypoint comprehension,

    Jie Yang, Wang Zeng, Sheng Jin, Lumin Xu, Wentao Liu, Chen Qian, and Ruimao Zhang. Kptllm: Unveiling the power of large language model for keypoint comprehension,

  48. [56]

    Grounding 3d object affordance from 2d interactions in images

    Yuhang Yang, Wei Zhai, Hongchen Luo, Yang Cao, Jiebo Luo, and Zheng-Jun Zha. Grounding 3d object affordance from 2d interactions in images. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 10905–10915, 2023. 1, 2, 3, 6, 7

    2023

  49. [57]

    Lemon: Learning 3d human-object in- teraction relation from 2d images

    Yuhang Yang, Wei Zhai, Hongchen Luo, Yang Cao, and Zheng-Jun Zha. Lemon: Learning 3d human-object in- teraction relation from 2d images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16284–16295, 2024. 5

    2024

  50. [58]

    Egochoir: Capturing 3d human-object interaction regions from egocentric views

    Yuhang Yang, Wei Zhai, Chengfeng Wang, Chengjun Yu, Yang Cao, and Zheng-Jun Zha. Egochoir: Capturing 3d human-object interaction regions from egocentric views. arXiv preprint arXiv:2405.13659, 2024. 1

    2024 arXiv

  51. [59]

    Griffiths, Yuan Cao, and Karthik Narasimhan

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large lan- guage models, 2023. 3

    2023

  52. [60]

    Beyond chain-of- thought, effective graph-of-thought reasoning in language models, 2024

    Yao Yao, Zuchao Li, and Hai Zhao. Beyond chain-of- thought, effective graph-of-thought reasoning in language models, 2024. 3

    2024

  53. [61]

    One-shot object affordance detection in the wild

    Wei Zhai, Hongchen Luo, Jing Zhang, Yang Cao, and Dacheng Tao. One-shot object affordance detection in the wild. arXiv preprint arXiv:2108.03658, 2021. 3

    2021 arXiv

  54. [62]

    Ex- ploring figure-ground assignment mechanism in perceptual organization

    Wei Zhai, Yang Cao, Jing Zhang, and Zheng-Jun Zha. Ex- ploring figure-ground assignment mechanism in perceptual organization. Advances in Neural Information Processing Systems, 35:17030–17042, 2022. 1

    2022

  55. [63]

    Background activation suppression for weakly supervised object localization and semantic segmen- tation

    Wei Zhai, Pingyu Wu, Kai Zhu, Yang Cao, Feng Wu, and Zheng-Jun Zha. Background activation suppression for weakly supervised object localization and semantic segmen- tation. International Journal of Computer Vision , pages 1– 26, 2023. 3

    2023

  56. [64]

    On exploring multiplicity of prim- itives and attributes for texture recognition in the wild

    Wei Zhai, Yang Cao, Jing Zhang, Haiyong Xie, Dacheng Tao, and Zheng-Jun Zha. On exploring multiplicity of prim- itives and attributes for texture recognition in the wild. IEEE Transactions on Pattern Analysis and Machine Intelligence , 46(1):403–420, 2024. 1

    2024

  57. [65]

    Automatic chain of thought prompting in large language models, 2022

    Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. Automatic chain of thought prompting in large language models, 2022. 3

    2022

  58. [66]

    Bidirectional progressive transformer for interac- tion intention anticipation

    Zichen Zhang, Hongchen Luo, Wei Zhai, Yang Cao, and Yu Kang. Bidirectional progressive transformer for interac- tion intention anticipation. arXiv preprint arXiv:2405.05552,

    arXiv

  59. [67]

    Pear: Phrase-based hand-object interaction anticipa- tion

    Zichen Zhang, Hongchen Luo, Wei Zhai, Yang Cao, and Yu Kang. Pear: Phrase-based hand-object interaction anticipa- tion. arXiv preprint arXiv:2407.21510, 2024. 1

    2024 arXiv

  60. [68]

    Minigpt-4: Enhancing vision-language understanding with advanced large language models

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mo- hamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592, 2023. 3

    2023 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.