Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

This paper argues that a robot can learn where and how two objects should interact—per-point on both 3D point clouds—from a single annotated example of each interaction type, and that these affordance maps improve LLM-generated manipulation

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A one-shot training regime with DINOv2-enriched point clouds and joint cross-attention predicts 3D object-to-object affordance maps that guide optimization-based robotic manipulation.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection A solid one-shot object-to-object affordance system with real robot demos, but the quantitative grounding results rest on self-annotated labels that need independent validation. the 4 major comments →

arxiv 2509.06233 v1 pith:TY2N3TY5 submitted 2025-09-07 cs.RO cs.CV

O$^3$Afford: One-Shot 3D Object-to-Object Affordance Grounding for Generalizable Robotic Manipulation

Tongxuan Tian , Xuhui Kang , Yen-Ling Kuo This is my paper
classification cs.RO cs.CV
keywords affordance groundingone-shot learning3D point cloudsobject-to-object interactionvision foundation modelsrobotic manipulationLLM constraint generationsemantic features
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a robot can learn to predict where two objects should interact—which point on a teapot should meet which region of a bowl, which edge of a knife should contact which part of an apple—from a single annotated example of each interaction type. This matters because daily manipulation tasks are almost always relational, yet existing affordance systems mostly label one object in isolation or only handle simple contact relations learned from large simulation data. The proposed system fuses semantic features from a visual foundation model with 3D point-cloud geometry for both objects, then uses a bidirectional joint-attention decoder to emit per-point interaction scores on both objects at once. In the paper's experiments, these maps generalize to unseen object instances and categories, survive occlusion, and, when turned into constraints for an LLM-generated pose optimizer, improve success rates on five two-object manipulation tasks in simulation and on a real robot. If the claim holds, object-to-object affordance becomes a data-cheap intermediate representation for robotic manipulation.

Core claim

The paper's central claim is that object-to-object affordance can be learned in 3D from one annotated example per interaction verb. The system takes two point clouds—a source object and a target object—with per-point semantic features distilled from a self-supervised visual foundation model, and a joint-attention transformer decoder predicts a per-point interaction-probability map on both objects simultaneously. The affordance category is defined by the verb (pour, cut, press, hang, insert), not by the object pair, so at test time the same decoder is applied zero-shot to entirely unseen object pairs and even unseen categories with similar functional parts. The paper further claims that feedi

What carries the argument

The load-bearing object is the 3D semantic point cloud: DINOv2 features extracted from multiple RGB-D views are projected and fused onto each object's point cloud, giving every point a rich semantic descriptor alongside its coordinates. On top of this, a joint-attention transformer decoder tokenizes each point cloud into local patches with a PointNet encoder, then applies cross-attention in both directions—source attending to target and target attending to source—so that the predicted affordance on each object depends on the other object's geometry and semantics. One-hot source/target embeddings keep the roles distinct. The final per-point maps are trained with binary cross-entropy against d

Load-bearing premise

The results assume that a few user-picked contact points, spread to nearby points by distance, correctly mark the functional interaction region on each object, and that image-derived semantic features projected onto the point cloud carry enough signal for one example per interaction verb to generalize to unseen objects and categories.

What would settle it

Retrain the model on the same object pairs using independently annotated contact points from a second annotator, or contact labels recovered from physical interaction, instead of distance-propagated seed labels, then compare held-out IOU and real-robot success; a large drop would show the model learned label propagation artifacts rather than object-interaction semantics.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A robot can be taught a new interaction verb from a single example pair and generalize it to unseen object instances and categories.
  • Object-to-object affordance in 3D is more robust to viewpoint and occlusion changes than 2D affordance grounding, because semantic and geometric cues are fused in point-cloud space.
  • Affordance maps can act as effective intermediate constraints for LLM-based optimization planners, raising manipulation success in two-object tasks.
  • The one-shot formulation avoids large annotation efforts for pairwise affordance data, making the approach scalable to many everyday tasks.
  • The model degrades gracefully under partial observations, such as two camera views instead of four, supporting deployment with limited sensing.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A direct implication the paper leaves implicit: because the affordance category is a verb rather than a fixed object pair, the same decoder could be trained on one example per verb from human demonstrations, replacing seed-point annotations with contact traces recovered from manipulation videos.
  • The failure analysis suggests that once affordance grounding is adequate, sensor depth quality and inverse-kinematics feasibility become the dominant bottlenecks; combining this method with sparse-view 3D reconstruction could raise success more than further affordance-model tuning.
  • The one-shot formulation could be tested as an open-vocabulary system: substitute a language embedding for the one-hot affordance channel and ask the model to ground any verb, letting a user specify novel interactions without retraining.
  • Cross-category generalization to objects that share no part geometry (e.g., pouring from a bag versus a teapot) would stress whether the method relies on semantic features or on local geometry clusters; the paper does not test that limit.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes O3Afford, a one-shot 3D object-to-object affordance grounding method. It constructs semantic point clouds by projecting DINOv2 features from multi-view RGB-D frames, encodes source and target point clouds with a PointNet patch encoder, and uses a joint-attention transformer decoder to predict per-point affordance maps for both objects. The affordance maps are then integrated with LLM-generated geometric constraint functions for optimization-based robotic manipulation. Training uses a single annotated object pair per affordance category. Experiments report affordance grounding metrics (Table 1), ablations (Table 4), and success rates on five simulated and real-robot manipulation tasks (Tables 2-3), with qualitative generalization to unseen categories and occlusion conditions.

Significance. If the results hold, the paper makes a useful contribution: it formulates object-to-object affordance grounding as a one-shot 3D problem, combines DINOv2 feature distillation with a joint-attention interaction module, and demonstrates a plausible integration of affordance maps with LLM-generated constraints for manipulation. The simulation and real-robot evaluation scope is broader than in many affordance papers, and the method is relatively simple to reproduce from the description. The strongest parts are the clean problem formulation, the use of pretrained semantic features to reduce supervision, and the concrete downstream manipulation formulation. However, the central claims rest on a self-annotated label-generation procedure and on success counts without statistical support; these need independent validation before the reported margins can be taken as evidence of genuine object-to-object affordance understanding.

major comments (4)
  1. [Sec. 5.1, Table 1, Sec. 4.3] The ground-truth affordance labels are created by the authors via user-assigned contact points followed by distance-based propagation (Sec. 5.1, following [26]). The same annotation procedure is used for both training (BCE loss in Sec. 4.3) and evaluation (Table 1). There is no independent human annotation, no inter-annotator agreement, and no sensitivity analysis of the propagation radius/kernel. Consequently, the large margins in Table 1 (aIOU 26.19 vs 16.89 for the best baseline) may reflect learning the distance-propagation label bias rather than functional object-to-object semantics. The concern is reinforced by Table 4: removing DINOv2 features drops aIOU from 26.19 to 6.59, suggesting that the pretrained semantic features dominate the result. I recommend adding an independent human-annotated evaluation set (or a leave-one-out cross-check against human labels), reporting per-catego
  2. [Tables 2 and 3, Sec. 5.3] Success rates are reported as success counts over 10 trials per task with no confidence intervals, no significance tests, and no description of trial randomization or initial-state variation. The text repeatedly states that the method 'significantly outperforms' baselines (e.g., Sec. 5.3). With 10 trials, a 3/10 vs 8/10 difference is suggestive but not statistically secured, and smaller differences (e.g., Hang 3/10 vs 1/10 under 2-view) are within binomial noise. Please provide binomial confidence intervals, per-trial variability, and, where feasible, repeated seeds or independent trials. Also specify how ReKep is configured (keypoint proposal, optimizer, and whether the same LLM is used) so the comparison is reproducible.
  3. [Sec. 5.2.2 and Fig. 4] Category-level generalization is shown only qualitatively on unseen categories (scissor, coat rack, spray bottle). Since the title and abstract claim 'generalizable' and 'strong semantic and geometric generalization,' a quantitative evaluation on held-out categories is needed. As written, the claim rests on three qualitative examples. An extension would be to add a held-out category split to Tables 1/4 or report per-category aIOU/AUC for unseen categories. This is load-bearing for the generalization claim.
  4. [Table 1 and Appendix B.1] Baseline comparability is questionable: O2O-Afford is designed for annotation-free contact-based affordances (place/fit), and IAGNet is adapted from a single-image 2D setting by providing both source and target bounding boxes. The paper should justify that these are meaningful comparators for the one-shot 3D O2O setting, or add a baseline that uses the same input representation and supervision level. Otherwise the 'outperforms baselines' claim is difficult to interpret.
minor comments (5)
  1. [Abstract and Sec. 5.1] Typo: 'contraints' in the abstract. In Sec. 5.1, the phrase 'several user-assigned contact points' is vague; please specify how many contact points are used per object and the exact distance-propagation rule.
  2. [Table 1] The header arrows are inconsistent: IOU and SIM are marked with up arrows while MAE is marked with a down arrow, but the text later refers to '↑ IOU ↑ SIM ↓ MAE ↑ AUC' without explaining the direction. Clarify which direction is better for each metric.
  3. [Sec. 4.4] Equation (1) is referenced as 'Eq. 4.4' in the text, which is a formatting artifact. Also, the objective defines a weighted sum of score terms but does not explicitly list the constraints mentioned in the introductory sentence.
  4. [References] References [11] and [13] are duplicates (CLIP), and [12] and [34] are duplicates (latent diffusion). Please consolidate.
  5. [Sec. 3 and Sec. 4.3] The symbol K is used both for the number of affordance categories in the problem formulation and for the output-channel dimension in A ∈ [0,1]^{2B×K×N} in Sec. 4.3. Use distinct symbols to avoid confusion.

Circularity Check

0 steps flagged

No significant circularity: the core training/evaluation loop is supervised against human-assigned labels and external benchmarks, with no fitted input renamed as a prediction.

full rationale

No load-bearing circular step is present in the paper. The affordance model is trained with a standard binary cross-entropy loss (Sec. 4.3) against ground-truth affordance maps produced by user-assigned contact points propagated by distance (Sec. 5.1). This label-generation procedure is borrowed from prior work (3D AffordanceNet [26]) and is external to the authors' own method; the test-time affordance metrics (aIOU, SIM, MAE, AUC) are computed against held-out labels of the same type. The model does not fit a parameter to the test set and then rename it a prediction; it is trained on one example per category and evaluated on unseen object pairs, which is a standard supervised generalization setup. The manipulation experiments (Tables 2-3) measure physical task success in simulation and on a real Franka robot, which is an external, task-level outcome rather than a quantity derived from the model's own outputs. The only pretrained component, DINOv2, is used as a frozen external feature extractor, and the multi-view feature projection method is adapted from D3Field [14], which is not authored by the present authors. There are no self-citations in the reference list that carry the argument, and the paper does not invoke any uniqueness theorem or ansatz that is smuggled in via the authors' prior work. Concerns about distance-propagated label quality, the small number of trials, or the choice of LLM constraint weights in Table 7 are validity and robustness issues, not circularity: they do not make any stated prediction equivalent to its own input by construction.

Axiom & Free-Parameter Ledger

2 free parameters · 4 axioms · 0 invented entities

The model itself has no exotic free constants, but the manipulation pipeline uses hand-set LLM constraint weights per task, and the entire evaluation rests on self-annotated affordance labels and the transferability of DINOv2 semantic features.

free parameters (2)
  • LLM constraint weights lambda_i per task = e.g., Pour: position 0.3, orientation 0.2, clearance 0.3, affordance alignment 0.1 (Table 7)
    These weights in Eq. 4.4 are selected by the authors/LLM for each task and directly affect the optimized pose; they are not learned from data.
  • Number of training examples per category = 1 (one-shot)
    The one-shot assumption is the core experimental setting; results may not transfer to other values of k.
axioms (4)
  • domain assumption DINOv2 features projected onto 3D point clouds preserve semantic part correspondence useful for affordance grounding.
    The entire one-shot generalization depends on the transferability of these 2D features to novel 3D shapes (Sec. 4.2, Sec. 5.2.2).
  • domain assumption PointNet patch encoder captures local geometric primitives that transfer across object categories.
    The model relies on PointNet embeddings to generalize under occlusion and to new categories (Sec. 4.3, Appendix A.5).
  • domain assumption LLM-generated constraint functions are physically meaningful and correctly weighted for the specified tasks.
    Manipulation success depends on the LLM's commonsense translation of task names into geometric costs and on the hand-set weights in Table 7 (Sec. 4.4).
  • domain assumption The affordance labels produced by user-assigned contact points and distance propagation are a valid ground truth for object-to-object interaction.
    Both training and evaluation use these labels, so the metric values inherit their quality (Sec. 5.1).

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of O$^3$Afford: One-Shot 3D Object-to-Object Affordance Grounding for Generalizable Robotic Manipulation." pith.science (2026). https://pith.science/paper/TY2N3TY5

@misc{pith2026250906233,
  author       = {Pith},
  title        = {Pith review of: O$^3$Afford: One-Shot 3D Object-to-Object Affordance Grounding for Generalizable Robotic Manipulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TY2N3TY5}},
  note         = {Machine review of arXiv:2509.06233}
}
Share X Bluesky LinkedIn Reddit HN
abstract

Grounding object affordance is fundamental to robotic manipulation as it establishes the critical link between perception and action among interacting objects. However, prior works predominantly focus on predicting single-object affordance, overlooking the fact that most real-world interactions involve relationships between pairs of objects. In this work, we address the challenge of object-to-object affordance grounding under limited data contraints. Inspired by recent advances in few-shot learning with 2D vision foundation models, we propose a novel one-shot 3D object-to-object affordance learning approach for robotic manipulation. Semantic features from vision foundation models combined with point cloud representation for geometric understanding enable our one-shot learning pipeline to generalize effectively to novel objects and categories. We further integrate our 3D affordance representation with large language models (LLMs) for robotics manipulation, significantly enhancing LLMs' capability to comprehend and reason about object interactions when generating task-specific constraint functions. Our experiments on 3D object-to-object affordance grounding and robotic manipulation demonstrate that our O$^3$Afford significantly outperforms existing baselines in terms of both accuracy and generalization capability.

Figures

Figures reproduced from arXiv: 2509.06233 by Tongxuan Tian, Xuhui Kang, Yen-Ling Kuo.

Figure 1
Figure 1. Figure 1: O3Afford framework. O 3Afford first (a) constructs semantic point clouds; (b) employs joint cross-attention between the source and target objects for decoding; lastly, (c) leverages LLMs for task-relevant constraint function generation and optimizes the target pose for robot execution. fordance is commonly used either as an additional conditional input [45, 46] or as an auxiliary learn￾ing objective [47, 4… view at source ↗
Figure 2
Figure 2. Figure 2: Qualitative examples of the predicted object-to-object affordance in O3Afford. Bolt cameras to get RGB-D observations. We employ the LLM GPT-4o [54] from OpenAI for constraint function generation in planning. We design five tasks that require two objects to interact meaningfully with each other: pouring from teapot into bowl, inserting toast into toaster, pressing the button with hammer, hanging mug onto m… view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative Results of Unseen Object Category. 5.2.2 Generalization Analysis Beyond the robust intra-class generalization under extreme data constraints demonstrated in Sec. 5.2.1, we also analyze the generalization capability of O3Afford in this section. We study two forms of generalization: generalization to different levels of occlusion and cross-category gen￾eralization, both of which are prevalent and… view at source ↗
Figure 3
Figure 3. Figure 3: Quantitative results under different occlusion levels. Occlusion level (x-axis) ranges from 10% to 50% occluded point clouds. Occlusion Generalization. Rather than sim￾ulating occlusion by directly dropping points from the object point cloud, we evaluate gen￾eralization in a more realistic setting where ob￾jects are occluded by other objects (e.g., cubes). We exclude RoboPoint in the comparison, as it is n… view at source ↗
Figure 5
Figure 5. Figure 5: Example execution of real-world manipulation tasks. (Top) inserting, hanging, and cutting; (Bottom) pouring and pressing. The colors on the point clouds indicate the predicted affor￾dance values as mapped to the colorbar on the right. Results in simulation. Tab. 2 shows the success rate of each method. We evaluate in settings with different numbers of camera views, ranging from four views, representing ful… view at source ↗
Figure 6
Figure 6. Figure 6: presents the qualitative examples of our occlusion experiment in the simulated environment. We show that our method, leveraging the generalizable semantic features of DINOv2, demonstrates robustness under varying levels of occlusion. This includes extreme case with up to 50% point cloud occluded which is considered the most difficult scenario in real-world applications. 50% 40% 30% 20% 10% GT Pred [PITH_F… view at source ↗
Figure 7
Figure 7. Figure 7: Qualitative examples of RoboPoint predictions. in [PITH_FULL_IMAGE:figures/full_fig_p015_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Qualitative examples of OOAL predictions for four different views. Each row renders the affordance heatmaps of the same source/target object across views. A.5 Visualization of Latent Patch Embedding For the PointNet-encoded patch embeddings, we observe that they encode the reusable geometric primitives across different objects as shown in our t-SNE plot ( [PITH_FULL_IMAGE:figures/full_fig_p015_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: t-SNE visualization of latent patch embeddings. Pour Insert Cut Hang Press [PITH_FULL_IMAGE:figures/full_fig_p016_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Qualitative results in simulation environment. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Example of the generated function for object alignment. [PITH_FULL_IMAGE:figures/full_fig_p018_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Example of incorrect point cloud. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Example of IK failure. D Environment Setup Franka Research 3 Femto Bolt [PITH_FULL_IMAGE:figures/full_fig_p020_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Real robot experiment setup. Our task uses a Franka Research 3 robot, a 7-DOF manipulator. To capture comprehensive visual information and mitigate occlusions, we employ two Femto Bolt RGB-D cameras. These Time￾20 [PITH_FULL_IMAGE:figures/full_fig_p020_14.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. SemAnCorr: Semantic Anchored Correspondence for Zero-Shot Manipulation Skill Transfer

    cs.RO 2026-07 conditional novelty 5.0

    Training-free semantic anchors plus functional maps yield dense correspondences that are both part-accurate and geometrically coherent, improving zero-shot robot skill transfer over nearest-neighbor 3D feature matching.

  2. Beyond Point-Attached Semantics: Object-Centric Semantic Fields for Generalizable Manipulation

    cs.RO 2026-07 conditional novelty 5.0

    An object-conditioned continuous semantic field queried at explicit 3D locations yields more stable part cues and higher manipulation success than point-attached 2D/3D features.

Reference graph

Works this paper leans on

61 extracted references · 45 canonical work pages · cited by 2 Pith papers · 1 internal anchor

  1. [1]

    Kaptelinin and B

    V . Kaptelinin and B. Nardi. Affordances in hci: toward a mediated action perspective. In Proceedings of the SIGCHI conference on human factors in computing systems , pages 967– 976, 2012

    work page 2012

  2. [2]

    Delitzas, A

    A. Delitzas, A. Takmaz, F. Tombari, R. Sumner, M. Pollefeys, and F. Engelmann. Scene- fun3d: fine-grained functionality and affordance understanding in 3d scenes. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14531–14542, 2024

    work page 2024

  3. [3]

    Kuang, J

    Y . Kuang, J. Ye, H. Geng, J. Mao, C. Deng, L. Guibas, H. Wang, and Y . Wang. Ram: Retrieval- based affordance transfer for generalizable zero-shot robotic manipulation. arXiv preprint arXiv:2407.04689, 2024

    Pith/arXiv arXiv 2024

  4. [4]

    G. Li, D. Sun, L. Sevilla-Lara, and V . Jampani. One-shot open affordance learning with foun- dation models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3086–3096, 2024

    work page 2024

  5. [5]

    H. Luo, W. Zhai, J. Zhang, Y . Cao, and D. Tao. Learning affordance grounding from exo- centric images. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2252–2261, 2022

    work page 2022

  6. [6]

    H. Luo, W. Zhai, J. Zhang, Y . Cao, and D. Tao. Leverage interactive affinity for affordance learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, pages 6809–6819, 2023

    work page 2023

  7. [7]

    W. Zhai, H. Luo, J. Zhang, Y . Cao, and D. Tao. One-shot object affordance detection in the wild. International Journal of Computer Vision, 130(10):2472–2500, 2022

    work page 2022

  8. [8]

    X. Zhao, Y . Cao, and Y . Kang. Object affordance detection with relationship-aware network. Neural Computing and Applications, 32(18):14321–14333, 2020. 9

    work page 2020

  9. [9]

    K. Mo, Y . Qin, F. Xiang, H. Su, and L. Guibas. O2o-afford: Annotation-free large-scale object-object affordance learning. In Conference on robot learning, pages 1666–1677. PMLR, 2022

    work page 2022

  10. [10]

    Oquab, T

    M. Oquab, T. Darcet, T. Moutakanni, H. V o, M. Szafraniec, V . Khalidov, P. Fernandez, D. Haz- iza, F. Massa, A. El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023

    Pith/arXiv arXiv 2023

  11. [11]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever. Learning transferable visual models from natural language supervision. In M. Meila and T. Zhang, editors, Proceedings of the 38th In- ternational Conference on Machine Learning, volume 139 of Proceedings of Machine Le...

    work page 2021

  12. [12]

    Rombach, A

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image syn- thesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    work page 2022

  13. [13]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, pages 8748–8763. PmLR, 2021

    work page 2021

  14. [14]

    Y . Wang, M. Zhang, Z. Li, T. Kelestemur, K. Driggs-Campbell, J. Wu, L. Fei-Fei, and Y . Li. D3fields: Dynamic 3d descriptor fields for zero-shot generalizable rearrangement. In 8th An- nual Conference on Robot Learning, 2024

    work page 2024

  15. [15]

    Z. Zhou, Y . Lei, B. Zhang, L. Liu, and Y . Liu. Zegclip: Towards adapting clip for zero-shot semantic segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023

    work page 2023

  16. [16]

    Han and S.-N

    G. Han and S.-N. Lim. Few-shot object detection with foundation models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 28608–28618, June 2024

    work page 2024

  17. [17]

    Zhang, C

    J. Zhang, C. Herrmann, J. Hur, L. Polania Cabrera, V . Jampani, D. Sun, and M.-H. Yang. A tale of two features: Stable diffusion complements dino for zero-shot semantic correspondence. Advances in Neural Information Processing Systems, 36:45533–45547, 2023

    work page 2023

  18. [18]

    Myers, C

    A. Myers, C. L. Teo, C. Fermüller, and Y . Aloimonos. Affordance detection of tool parts from geometric features. In 2015 IEEE international conference on robotics and automation (ICRA), pages 1374–1381. IEEE, 2015

    work page 2015

  19. [19]

    Sawatzky, A

    J. Sawatzky, A. Srikantha, and J. Gall. Weakly supervised affordance detection. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2795–2804, 2017

    work page 2017

  20. [20]

    J. H. Jang, H. Seo, and S. Y . Chun. Intra: Interaction relationship-aware weakly supervised affordance grounding. In European Conference on Computer Vision, pages 18–34. Springer, 2024

    work page 2024

  21. [21]

    G. Li, V . Jampani, D. Sun, and L. Sevilla-Lara. Locate: Localize and transfer object parts for weakly supervised affordance grounding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10922–10931, 2023

    work page 2023

  22. [22]

    S. Qian, W. Chen, M. Bai, X. Zhou, Z. Tu, and L. E. Li. Affordancellm: Grounding affordance from vision language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7587–7597, 2024. 10

    work page 2024

  23. [23]

    C. Chen, Y . Cong, and Z. Kan. Worldafford: Affordance grounding based on natural language instructions. In 2024 IEEE 36th International Conference on Tools with Artificial Intelligence (ICTAI), pages 822–828. IEEE, 2024

    work page 2024

  24. [24]

    Y . Ju, K. Hu, G. Zhang, G. Zhang, M. Jiang, and H. Xu. Robo-abc: Affordance general- ization beyond categories via semantic correspondence for robot manipulation. In European Conference on Computer Vision, pages 222–239. Springer, 2024

    work page 2024

  25. [25]

    E. Tong, A. Opipari, S. Lewis, Z. Zeng, and O. C. Jenkins. Oval-prompt: Open-vocabulary af- fordance localization for robot manipulation through llm affordance-grounding.arXiv preprint arXiv:2404.11000, 2024

    Pith/arXiv arXiv 2024

  26. [26]

    S. Deng, X. Xu, C. Wu, K. Chen, and K. Jia. 3d affordancenet: A benchmark for visual object affordance understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1778–1787, June 2021

    work page 2021

  27. [27]

    Y . Yang, W. Zhai, H. Luo, Y . Cao, J. Luo, and Z.-J. Zha. Grounding 3d object affordance from 2d interactions in images. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 10905–10915, October 2023

    work page 2023

  28. [28]

    Y . Yang, W. Zhai, H. Luo, Y . Cao, and Z.-J. Zha. Lemon: Learning 3d human-object interaction relation from 2d images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16284–16295, 2024

    work page 2024

  29. [29]

    H. Chu, X. Deng, Q. Lv, X. Chen, Y . Li, J. Hao, and L. Nie. 3d-affordancellm: Harnessing large language models for open-vocabulary affordance detection in 3d worlds. arXiv preprint arXiv:2502.20041, 2025

    Pith/arXiv arXiv 2025

  30. [30]

    Y . Li, N. Zhao, J. Xiao, C. Feng, X. Wang, and T.-s. Chua. Laso: Language-guided affordance segmentation on 3d object. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14251–14260, June 2024

    work page 2024

  31. [31]

    Brown, B

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Nee- lakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-V oss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Rad- ford, I. Sutskever, and D. Am...

    work page 1901

  32. [32]

    H. Song, L. Dong, W.-N. Zhang, T. Liu, and F. Wei. Clip models are few-shot learners: Empirical studies on vqa and visual entailment. arXiv preprint arXiv:2203.07190, 2022

    Pith/arXiv arXiv 2022

  33. [33]

    Touvron, L

    H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023

    Pith/arXiv arXiv 2023

  34. [34]

    Rombach, A

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image syn- thesis with latent diffusion models, 2021

    work page 2021

  35. [35]

    Qiu, Y .-X

    R.-Z. Qiu, Y .-X. Wang, and K. Hauser. Aligndiff: aligning diffusion models for general few- shot segmentation. In European Conference on Computer Vision , pages 384–400. Springer, 2024

    work page 2024

  36. [36]

    M. Zhu, Y . Liu, Z. Luo, C. Jing, H. Chen, G. Xu, X. Wang, and C. Shen. Unleash- ing the potential of the diffusion model in few-shot semantic segmentation. arXiv preprint arXiv:2410.02369, 2024. 11

    Pith/arXiv arXiv 2024

  37. [37]

    W. Tan, S. Chen, and B. Yan. Diffss: Diffusion model for few-shot semantic segmentation. arXiv preprint arXiv:2307.00773, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  38. [38]

    J. Zhou, J. Wang, B. Ma, Y .-S. Liu, T. Huang, and X. Wang. Uni3d: Exploring unified 3d rep- resentation at scale. In International Conference on Learning Representations (ICLR), 2024

    work page 2024

  39. [39]

    Zhang, Z

    R. Zhang, Z. Guo, W. Zhang, K. Li, X. Miao, B. Cui, Y . Qiao, P. Gao, and H. Li. Pointclip: Point cloud understanding by clip. arXiv preprint arXiv:2112.02413, 2021

    Pith/arXiv arXiv 2021

  40. [40]

    Borja-Diaz, O

    J. Borja-Diaz, O. Mees, G. Kalweit, L. Hermann, J. Boedecker, and W. Burgard. Affordance learning from play for sample-efficient policy learning. In 2022 International Conference on Robotics and Automation (ICRA), pages 6372–6378. IEEE, 2022

    work page 2022

  41. [41]

    O. Y . Lee, A. Xie, K. Fang, K. Pertsch, and C. Finn. Affordance-guided reinforcement learning via visual prompting. arXiv preprint arXiv:2407.10341, 2024

    Pith/arXiv arXiv 2024

  42. [42]

    S. Bahl, R. Mendonca, L. Chen, U. Jain, and D. Pathak. Affordances from human videos as a versatile representation for robotics. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13778–13790, 2023

    work page 2023

  43. [43]

    Y . Geng, B. An, H. Geng, Y . Chen, Y . Yang, and H. Dong. Rlafford: End-to-end affordance learning for robotic manipulation. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 5880–5886. IEEE, 2023

    work page 2023

  44. [44]

    K. Fang, P. Yin, A. Nair, H. R. Walke, G. Yan, and S. Levine. Generalization with lossy affordances: Leveraging broad offline data for learning visuomotor tasks. In Conference on Robot Learning, pages 106–117. PMLR, 2023

    work page 2023

  45. [45]

    Nasiriany, S

    S. Nasiriany, S. Kirmani, T. Ding, L. Smith, Y . Zhu, D. Driess, D. Sadigh, and T. Xiao. Rt-affordance: Affordances are versatile intermediate representations for robot manipulation. arXiv preprint arXiv:2411.02704, 2024

    Pith/arXiv arXiv 2024

  46. [46]

    S. Wu, Y . Zhu, Y . Huang, K. Zhu, J. Gu, J. Yu, Y . Shi, and J. Wang. Afforddp: Generalizable diffusion policy with transferable affordance. arXiv preprint arXiv:2412.03142, 2024

    Pith/arXiv arXiv 2024

  47. [47]

    Y . Zha, S. Bhambri, and L. Guan. Contrastively learning visual attention as affordance cues from demonstrations for robotic grasping. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 7835–7842. IEEE, 2021

    work page 2021

  48. [48]

    O. Mees, J. Borja-Diaz, and W. Burgard. Grounding language with visual affordances over unstructured data. In Proceedings of the IEEE International Conference on Robotics and Au- tomation (ICRA), London, UK, 2023

    work page 2023

  49. [49]

    Huang, C

    W. Huang, C. Wang, R. Zhang, Y . Li, J. Wu, and L. Fei-Fei. V oxposer: Composable 3d value maps for robotic manipulation with language models. In J. Tan, M. Toussaint, and K. Darvish, editors, Proceedings of The 7th Conference on Robot Learning , volume 229 of Proceedings of Machine Learning Research, pages 540–562. PMLR, 06–09 Nov 2023. URL https://proce...

    work page 2023

  50. [50]

    Huang, C

    W. Huang, C. Wang, Y . Li, R. Zhang, and L. Fei-Fei. Rekep: Spatio-temporal reasoning of relational keypoint constraints for robotic manipulation. arXiv preprint arXiv:2409.01652 , 2024

    Pith/arXiv arXiv 2024

  51. [51]

    W. Yuan, J. Duan, V . Blukis, W. Pumacay, R. Krishna, A. Murali, A. Mousavian, and D. Fox. Robopoint: A vision-language model for spatial affordance prediction for robotics. arXiv preprint arXiv:2406.10721, 2024

    Pith/arXiv arXiv 2024

  52. [52]

    Xiang, Y

    F. Xiang, Y . Qin, K. Mo, Y . Xia, H. Zhu, F. Liu, M. Liu, H. Jiang, Y . Yuan, H. Wang, L. Yi, A. X. Chang, L. J. Guibas, and H. Su. SAPIEN: A simulated part-based interactive environ- ment. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2020. 12

    work page 2020

  53. [53]

    I. Liu, E. Yang, J. Tao, R. Chen, X. Zhang, Q. Ran, Z. Liu, and H. Su. Activezero: Mixed domain learning for active stereovision with zero annotation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13033–13042, 2022

    work page 2022

  54. [54]

    Achiam, S

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    Pith/arXiv arXiv 2023

  55. [55]

    M. A. Rahman and Y . Wang. Optimizing intersection-over-union in deep neural networks for image segmentation. In International symposium on visual computing , pages 234–244. Springer, 2016

    work page 2016

  56. [56]

    M. J. Swain and D. H. Ballard. Color indexing. International journal of computer vision, 7(1): 11–32, 1991

    work page 1991

  57. [57]

    C. J. Willmott and K. Matsuura. Advantages of the mean absolute error (mae) over the root mean square error (rmse) in assessing average model performance. Climate research, 30(1): 79–82, 2005

    work page 2005

  58. [58]

    J. M. Lobo, A. Jiménez-Valverde, and R. Real. Auc: a misleading measure of the performance of predictive distribution models. Global ecology and Biogeography, 17(2):145–151, 2008

    work page 2008

  59. [59]

    Y . Tang, W. Huang, Y . Wang, C. Li, R. Yuan, R. Zhang, J. Wu, and L. Fei-Fei. Uad: Un- supervised affordance distillation for generalization in robotic manipulation. arXiv preprint arXiv:2506.09284, 2025

    Pith/arXiv arXiv 2025

  60. [60]

    Xiang, Z

    J. Xiang, Z. Lv, S. Xu, Y . Deng, R. Wang, B. Zhang, D. Chen, X. Tong, and J. Yang. Structured 3d latents for scalable and versatile 3d generation. arXiv preprint arXiv:2412.01506, 2024. 13 Appendices A Additional Results A.1 Ablation We present ablation results on the model architecture to clarify the specific contributions of each module within our affo...

    Pith/arXiv arXiv 2024

  61. [2020]

    URL https://proceedings.neurips.cc/paper_files/paper/2020/ file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf

    work page 2020

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.