Pith. sign in

REVIEW 3 major objections 5 minor 105 references

Dense functional correspondence across object categories can be learned with weak supervision.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A weakly-supervised pipeline that distills VLM functional part knowledge and multi-view spatial structure into a model for dense cross-category functional correspondence, outperforming baselines on new synthetic and real benchmarks.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection Defines a new task with useful benchmarks and a solid weakly-supervised method, but the synthetic evaluation may have a train/eval asset overlap that needs checking. the 3 major comments →

arxiv 2509.03893 v1 pith:6Z624KOW submitted 2025-09-04 cs.CV

Weakly-Supervised Learning of Dense Functional Correspondences

Stefan Stojanov , Linan Zhao , Yunzhi Zhang , Daniel L. K. Yamins , Jiajun Wu This is my paper
classification cs.CV
keywords dense correspondencefunctional correspondenceweakly supervised learningcontrastive learningvision-language pseudo-labelscross-category matching3D object alignmentaffordance grounding
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that dense pixel-level correspondence between objects of different categories—e.g., matching a kettle's spout to a bottle's neck for pouring—can be defined and learned without manual dense labeling. The authors derive the task from 3D alignment of functionally equivalent parts, then train a model by combining pseudo-labeled functional parts from a vision-language model with multi-view contrastive learning that preserves spatial structure. They build synthetic and real benchmarks for evaluation and report that the trained model outperforms off-the-shelf self-supervised features and vision-language grounding baselines on both. If correct, the result matters for applications like transferring a demonstrated manipulation from one object to another.

Core claim

The central claim is that object function—the effect an object can cause on another object or substance—provides a reliable guide for dense correspondence across categories, because parts that carry out the same function tend to be similar in shape and appearance even when whole objects are visually different. The paper formalizes dense functional correspondence as a pixel-level mapping induced by aligning the functional parts of two 3D objects in space, and proposes a weakly supervised training scheme: use an off-the-shelf grounded vision-language model to propose functional-part bounding boxes on multi-view renders, aggregate those proposals into 3D pseudo-labels, then train a function-con

What carries the argument

The load-bearing mechanism is a function-conditioned dense feature extractor: frozen self-supervised image features are combined at each pixel with a text embedding of the function, then passed through a small MLP. Two contrastive objectives shape the feature space. The functional-part contrastive loss distills semantic knowledge from the vision-language pseudo-labels by pulling pixels on matching functional parts together and pushing other pixels apart. The spatial multi-view contrastive loss prevents mode collapse by requiring a pixel to match its projection in a second view of the same object and to differ from all other pixels, preserving the internal geometry of a part such as the tip v

Load-bearing premise

The whole evaluation rests on the assumption that one human 3D alignment of functional parts, such as a kettle spout against a bottle neck, gives the unambiguous ground truth for dense pixel correspondence; the paper notes this can be ambiguous for multi-functional parts or radially symmetric objects.

What would settle it

Run an inter-annotator study in which several annotators independently align each object pair; if their dense correspondences disagree on a substantial fraction of pairs, then the ground-truth benchmark and reported gains would be tied to one particular alignment convention.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A model can transfer a functional-part label from one object category to another without ever seeing human-annotated correspondence pairs.
  • Because training uses rendered synthetic images, the resulting representation generalizes to real photographs of unseen objects.
  • The pseudo-labeling pipeline converts noisy vision-language bounding-box proposals into spatially consistent pixel-level part masks by aggregating over many views in 3D.
  • Correspondence discovery—finding which pixels correspond without being given either part mask—becomes feasible, a capability needed for aligning a robot's actions to a new object.
  • The method's performance improves with larger backbones and higher spatial resolution, suggesting headroom from scaling.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural extension would be to condition on finer descriptions than a single function, such as 'pour-with while keeping the container upright,' to resolve the ambiguities the paper acknowledges.
  • The same vision-language pseudo-label plus 3D aggregation recipe could be applied to functions that are not object-part based, such as those involving object-object or object-substance interactions, as long as prompts can be written.
  • The learned feature space could be used as a reward or cost signal for motion planning, not just for matching, because it encodes functionally equivalent alignment.
  • The benchmark construction implies that inter-annotator agreement on functional alignment should be measured; if agreement is low, a probabilistic correspondence model would be needed.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces the task of dense functional correspondence: establishing dense pixel-level matches between images of objects from different categories whose parts support the same function (e.g., kettle spout to bottle neck). The authors propose a weakly-supervised training pipeline that (i) uses GPT-4 to build a function/object taxonomy, (ii) pseudo-labels functional parts on multi-view renders of Objaverse assets with CogVLM bounding boxes aggregated on 3D point clouds, and (iii) trains a function-conditioned MLP on top of frozen DINOv2/CLIP features with two contrastive losses—a functional part loss and a multi-view spatial loss—plus an optional mask loss. For evaluation, they define ground truth by aligning 3D object pairs in functionally equivalent poses and derive dense 2D correspondences; they curate 950 synthetic Objaverse pairs and 190 real HANDAL pairs. Experiments compare against DINOv2, Stable Diffusion, and VLM-grounded baselines (CogVLM+DINO, ManipVQA+DINO), reporting consistent improvements on label transfer and correspondence discovery metrics.

Significance. If the results hold, the paper makes a useful contribution: it gives a clean, actionable definition of dense functional correspondence, a scalable pseudo-labeling recipe that combines VLM semantic grounding with self-supervised spatial features, and two new benchmarks. The method is clearly specified, the grounding of pseudo-labels in an external VLM avoids the most obvious circularity in training, and the ablations (functional-only, spatial-only, mask loss, backbone scaling, loss-weight sensitivity) are extensive. The main uncertainty is not the method itself but the validity of the synthetic benchmark and the benchmark curation process, which the authors partly acknowledge. The real HANDAL evaluation provides some external anchor, but the headline synthetic numbers are the strongest quantitative claim.

major comments (3)
  1. [§3.2 and §4.1] The synthetic evaluation set may overlap with the training set, and this is load-bearing for the main claim. Section 4.1 states that 8,285 curated assets are used, with 80% for training, but Section 3.2 says the 950 evaluation pairs are 'assets from Objaverse selected for quality and diversity' and refers to Section 4.1 for selection without stating that the evaluation assets are held out from the training split. Since the spatial contrastive loss (Eq. 2) is applied across rendered views of the same training assets, the model could memorize object-specific geometry, inflating both label-transfer and discovery metrics on the synthetic set. Please state explicitly whether the 950 pairs are drawn exclusively from the held-out 20% (or otherwise exclude all training assets), and if not, report the evaluation on a disjoint held-out split. This is a concrete, checkable methodological requiremen
  2. [§9.3, Table 4] Hyperparameters λspatial and λmask appear to be selected on the same benchmark used for final reporting. Table 4 sweeps λspatial ∈ {1,5,10} and λmask ∈ {1,5,10} and reports test metrics on the synthetic and real evaluation sets, with λspatial=10 chosen as 'best result.' No validation split is described. If these choices were made by looking at the evaluation set, the reported numbers are optimistic. Please describe the model-selection protocol (e.g., a separate validation subset, or a fixed choice made before evaluation).
  3. [§8.1 and §11] The ground-truth annotation procedure involves manual disambiguation and filtering. The appendix says ambiguity is resolved 'based on the objects' orientation' and that annotations are 'manually filter[ed] ... to disambiguate and ensure high quality,' and the Limitations section concedes that radial symmetry and multiple functional regions make the task ambiguous. This is reasonable for a first benchmark, but the evaluation is partly defined by the annotators' judgment. Please report inter-annotator agreement or at least quantify how many pairs were filtered and whether the reported metrics are stable under alternative disambiguation choices. Without this, it is hard to know how much of the measured performance reflects the model versus the annotation protocol.
minor comments (5)
  1. [Table 1] The row labels 'DINO' and 'DINOv2' are used inconsistently; the text cites DINOv2 but the table says DINO. Please unify.
  2. [§5.1] The definition of PCK@k is missing an indicator function: it should read 1[||p2^{j(i)} - p2^i|| < k], not the expression as written.
  3. [Eq. (2)] The negative set for the spatial contrastive loss is described only as 'any other pixel on I'. Please clarify whether negatives are sampled uniformly, how many are used, and whether hard-negative mining is applied; this affects reproducibility.
  4. [§4.2 and Table 1] The ablation labeled 'Ours (functional only)' underperforms DINOv2 on several metrics. The text says the full model 'consistently outperforms,' but the functional-only variant does not; please qualify the claim to refer to the full model.
  5. [§5.2] ManipVQA is described as a 7B model, while CogVLM is described as 17B. It would be useful to note the parameter-count gap when interpreting the comparison, since the baselines have very different capacities.

Circularity Check

0 steps flagged

No significant circularity: human-annotated evaluation is independent of the VLM pseudo-label training loop, with only a minor non-load-bearing self-citation.

full rationale

The paper's derivation chain is not circular. The task definition (Sec. 3.1) is a formal definition based on 3D functional alignment, and the evaluation benchmarks (Sec. 3.2) apply that definition through human 3D alignments to obtain ground truth. Training uses external VLM pseudo-labels (GPT-4, CogVLM) and multi-view geometry, while evaluation uses human-annotated correspondences, so the reported predictions are not equal to the training inputs by construction. The only self-citation is [68] for general multi-view correspondence, and it is not load-bearing: the spatial contrastive loss (Eq. 2) relies on known render geometry to identify corresponding pixels, not on learned descriptors from [68]. Two experimental-validity concerns should be flagged but do not constitute circularity: (i) the synthetic evaluation assets are selected via the same curation pipeline as training assets (Sec. 3.2 vs. Sec. 4.1), and the paper never states that the evaluation pairs are disjoint from the 80% training split, so synthetic generalization may be optimistically biased; (ii) loss weights are selected using the same evaluation sets reported in Table 1 (Sec. 9.3), introducing test-set tuning bias. Neither concern makes a prediction reduce to an input by construction, and the real HANDAL benchmark is externally sourced and partially mitigates these issues.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The method is empirical; its success depends on the quality of external VLM pseudo-labels, the suitability of DINOv2 features, and the validity of the human-annotated 3D alignments. These are domain assumptions. The loss weights and sampling choices are free parameters. No invented entities are proposed.

free parameters (5)
  • lambda_spatial = 10
    Weight for the multi-view contrastive loss; chosen via sensitivity analysis and reported as best on the evaluation sets (Section 9.3).
  • lambda_mask = 1
    Weight for the binary functional part mask loss; stable across values (Section 9.3).
  • temperature_tau = not reported
    Temperature in the contrastive losses (Equations 1 and 2) is not specified; affects feature sharpness.
  • num_sampled_points = 128
    Number of positive and negative points sampled per image for contrastive learning (Section 4.3).
  • num_views_and_trials = 19 views x 4 trials
    Heuristics for pseudo-labeling with CogVLM; more views/trials improve consistency but cost compute (Section 7.4).
axioms (5)
  • domain assumption Form follows function: parts enabling the same function share shape/appearance consistency across objects.
    Central premise behind the definition of dense functional correspondence (Section 3).
  • domain assumption CogVLM can produce sufficiently accurate functional part bounding boxes for pseudo-labeling.
    The training pipeline relies on these labels; some validation via segmentation IoU in Table 5 (Section 4.1).
  • domain assumption Human 3D alignment of functional parts yields valid ground-truth correspondences despite acknowledged ambiguity.
    Evaluation benchmarks depend on this; limitations are discussed in Section 11.
  • domain assumption DINOv2 features provide a spatial prior sufficient for cross-category structural generalization after training.
    The method builds on frozen DINOv2 features and a small MLP (Section 4.2).
  • domain assumption Multi-view contrastive learning prevents modal collapse and preserves structural detail within functional parts.
    Motivates Lspatial in Section 4.2; supported by ablations.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Weakly-Supervised Learning of Dense Functional Correspondences." pith.science (2026). https://pith.science/paper/6Z624KOW

@misc{pith2026250903893,
  author       = {Pith},
  title        = {Pith review of: Weakly-Supervised Learning of Dense Functional Correspondences},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6Z624KOW}},
  note         = {Machine review of arXiv:2509.03893}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Establishing dense correspondences across image pairs is essential for tasks such as shape reconstruction and robot manipulation. In the challenging setting of matching across different categories, the function of an object, i.e., the effect that an object can cause on other objects, can guide how correspondences should be established. This is because object parts that enable specific functions often share similarities in shape and appearance. We derive the definition of dense functional correspondence based on this observation and propose a weakly-supervised learning paradigm to tackle the prediction task. The main insight behind our approach is that we can leverage vision-language models to pseudo-label multi-view images to obtain functional parts. We then integrate this with dense contrastive learning from pixel correspondences to distill both functional and spatial knowledge into a new model that can establish dense functional correspondence. Further, we curate synthetic and real evaluation datasets as task benchmarks. Our results demonstrate the advantages of our approach over baseline solutions consisting of off-the-shelf self-supervised image representations and grounded vision language models.

Figures

Figures reproduced from arXiv: 2509.03893 by Daniel L. K. Yamins, Jiajun Wu, Linan Zhao, Stefan Stojanov, Yunzhi Zhang.

Figure 1
Figure 1. Figure 1: Dense Functional Correspondence refers to establishing dense correspondences across object instances based on function sim￾ilarity (e.g., “pour-with”). This task is especially challenging when objects have visually different but functionally similar parts, requiring both semantic understanding, i.e., identifying which parts can perform the same function, and structural understanding, i.e., establishing den… view at source ↗
Figure 2
Figure 2. Figure 2: Annotation Pipeline (Evaluation Only). Given a 3D object pair (left) and a function (“pour-with”), we annotate the functional alignment of two objects by aligning the functional parts in 3D (middle). Afterward, we derive dense 2D correspon￾dences (right) based on 3D distances of corresponding object sur￾face points, with matching pixels shown in the same color. ally equivalent parts, which is central to ro… view at source ↗
Figure 3
Figure 3. Figure 3: Training Data Curation via VLM Pseudo Labeling. Given a large unstructured dataset like Objaverse [7], we leverage off￾the-shelf VLMs to curate and label the functional parts. Specifically, GPT-4 [55] generates category-specific functional part prompts, and CogVLM [82] produces bounding box proposals for multi-view image renderings, which are aggregated onto a 3D point cloud. The point cloud is post-proces… view at source ↗
Figure 4
Figure 4. Figure 4: Local Functional Feature Extraction. To obtain dense functionally conditioned features, we apply an MLP on top of a function text embedding and the spatial DINO features. The MLP is trained with both functional and spatial contrastive losses. tion “pour-with,” the features for the neck of the bottle and the spout of the kettle should be similar. Moreover, the mouth of the bottle and the tip of the kettle s… view at source ↗
Figure 5
Figure 5. Figure 5: Training Objectives. To ensure functional part simi￾larity in the learned feature space, we use a part-level contrastive objective to distill functional part semantics from VLMs (left). The spatial contrastive loss (right) serves a complementary role and prevents the model from collapsing predictions for different regions of a part, e.g., the top and bottom of a kettle spout. default hyperparameters, a bat… view at source ↗
Figure 6
Figure 6. Figure 6: Correspondence Discovery Comparisons. We observe that our approach more reliably retrieves the functionally relevant correspondences than off-the-shelf DINOv2. The top 10 highest-ranked matches are shown. • ManipVQA [25], an affordance-grounding model that outputs bounding boxes conditioned on actions. We use the 7B model in our experiments. We also prompt ManipVQA in two ways, one with the functional part… view at source ↗
Figure 7
Figure 7. Figure 7: Examples of pseudo-labeled functional parts in point clouds and images using CogVLM [82]. Using the procedure outlined in Section 7.4, we pseudo-label images with masks for the object functional parts. Notably, this pipeline has the ability to generate part labels for non-convex object parts, such as a mug’s rim, and for parts that lack clear edge boundaries, such as a teapot’s spout. Point clouds are show… view at source ↗
Figure 8
Figure 8. Figure 8: Additional Label Transfer Dense Visualizations. For each target image (right), our model predicts the functional part mask. To generate the transferred color map, each pixel in the predicted mask is matched to its best corresponding pixel within the ground-truth mask of the source image (left) in terms of feature similarity [PITH_FULL_IMAGE:figures/full_fig_p018_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Qualitative examples for the impact of mask loss. Functional part predictions can help avoid incorrect matches out￾side the functional parts in correspondence discovery [PITH_FULL_IMAGE:figures/full_fig_p018_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Comparison with [38] A visual comparison of dense correspondence between [38] (left) and our method (right). for dense correspondence, the method is not designed for this and it qualitatively appears to be relatively coarse. A visual comparison is provided in [PITH_FULL_IMAGE:figures/full_fig_p019_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Additional Correspondence Discovery Results on Objaverse Evaluation Dataset. We show more qualitative examples of correspondence discovery on the synthetic Objaverse evaluation dataset, comparing our model against baselines [PITH_FULL_IMAGE:figures/full_fig_p020_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Additional Correspondence Discovery Results on HANDAL Evaluation Dataset [PITH_FULL_IMAGE:figures/full_fig_p020_12.png] view at source ↗
Figure 14
Figure 14. Figure 14: Overview of our strategy for prompting GPT-4 [ [PITH_FULL_IMAGE:figures/full_fig_p021_14.png] view at source ↗
Figure 13
Figure 13. Figure 13: Our GPT-4 [55] prompting procedure for creating ob￾ject categories given a list of functions. System Prompt: Here are your instructions for the rest of the chat: Respond as if you are a human expert giving simplifying instructions to a robot learning to interact with the world by identifying object parts that correspond to verbs. We want to know what area of the object can be used to perform this action. … view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

105 extracted references · 63 canonical work pages · 4 internal anchors

  1. [1]

    Deep vit features as dense visual descriptors

    Shir Amir, Yossi Gandelsman, Shai Bagon, and Tali Dekel. Deep vit features as dense visual descriptors. arXiv preprint arXiv:2112.05814, 2021. 3

    Pith/arXiv arXiv 2021

  2. [2]

    Contactdb: Analyzing and predicting grasp contact via thermal imaging

    Samarth Brahmbhatt, Cusuh Ham, Charles C Kemp, and James Hays. Contactdb: Analyzing and predicting grasp contact via thermal imaging. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8709–8719, 2019. 2

    2019

  3. [3]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In Pro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 9650–9660, 2021. 3

    2021

  4. [4]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Ge- offrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on ma- chine learning, pages 1597–1607. PMLR, 2020. 5

    2020

  5. [5]

    Learning to act properly: Predicting and explaining affordances from images

    Ching-Yao Chuang, Jiaman Li, Antonio Torralba, and Sanja Fidler. Learning to act properly: Predicting and explaining affordances from images. In Proceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition, pages 975–983, 2018. 2

    2018

  6. [6]

    Blender - a 3D modelling and rendering package

    Blender Online Community. Blender - a 3D modelling and rendering package. Blender Foundation, Stichting Blender Foundation, Amsterdam, 2018. 4

    2018

  7. [7]

    Objaverse: A universe of annotated 3d objects

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13142–13153, 2023. 2, 4, 13, 16

    2023

  8. [8]

    Molmo and pixmo: Open weights and open data for state-of-the-art multimodal models

    Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tri- pathi, Yue Yang, Jae Sung Park, Mohammadreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, et al. Molmo and pixmo: Open weights and open data for state-of-the-art multimodal models. arXiv preprint arXiv:2409.17146, 2024. 5

    Pith/arXiv arXiv 2024

  9. [9]

    Objaverse-xl: A universe of 10m+ 3d objects

    Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram V oleti, Samir Yitzhak Gadre, et al. Objaverse-xl: A universe of 10m+ 3d objects. Advances in Neural Informa- tion Processing Systems, 36, 2024. 5

    2024

  10. [10]

    3d affordancenet: A benchmark for visual object af- fordance understanding

    Shengheng Deng, Xun Xu, Chaozheng Wu, Ke Chen, and Kui Jia. 3d affordancenet: A benchmark for visual object af- fordance understanding. In proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 1778–1787, 2021. 2

    2021

  11. [11]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. 3

    Pith/arXiv arXiv 2018

  12. [12]

    On the effectiveness of retrieval, alignment, and replay in manipulation

    Norman Di Palo and Edward Johns. On the effectiveness of retrieval, alignment, and replay in manipulation. IEEE Robotics and Automation Letters, 2024. 3

    2024

  13. [13]

    PreAfford: Universal Affordance-Based Pre-Grasping for Diverse Objects and Environments

    Kairui Ding, Boyuan Chen, Ruihai Wu, Yuyang Li, Zongzheng Zhang, Huan-ang Gao, Siqi Li, Guyue Zhou, Yixin Zhu, Hao Dong, et al. Preafford: Universal affordance- based pre-grasping for diverse objects and environments. arXiv preprint arXiv:2404.03634, 2024. 3

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  14. [14]

    Affordancenet: An end-to-end deep learning approach for object affordance detection

    Thanh-Toan Do, Anh Nguyen, and Ian Reid. Affordancenet: An end-to-end deep learning approach for object affordance detection. In 2018 IEEE international conference on robotics and automation (ICRA), pages 5882–5889. IEEE, 2018. 2

    2018

  15. [15]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Ab- hishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 ,

    Pith/arXiv arXiv

  16. [16]

    Graspnet-1billion: A large-scale benchmark for general ob- ject grasping

    Hao-Shu Fang, Chenxi Wang, Minghao Gou, and Cewu Lu. Graspnet-1billion: A large-scale benchmark for general ob- ject grasping. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 11444– 11453, 2020. 2

    2020

  17. [17]

    Dense object nets: Learning dense visual object descriptors by and for robotic manipulation

    Peter R Florence, Lucas Manuelli, and Russ Tedrake. Dense object nets: Learning dense visual object descriptors by and for robotic manipulation. In Conference on Robot Learning, pages 373–385. PMLR, 2018. 1, 2, 6

    2018

  18. [18]

    The ecological approach to visual percep- tion: Classic edition

    James J Gibson. The ecological approach to visual percep- tion: Classic edition. Taylor & Francis, 1979. 2

    1979

  19. [19]

    Videoswap: Customized video subject swapping with interactive semantic point cor- respondence

    Yuchao Gu, Yipin Zhou, Bichen Wu, Licheng Yu, Jia-Wei Liu, Rui Zhao, Jay Zhangjie Wu, David Junhao Zhang, Mike Zheng Shou, and Kevin Tang. Videoswap: Customized video subject swapping with interactive semantic point cor- respondence. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 7621– 7630, 2024. 1

    2024

  20. [20]

    Handal: A dataset of real-world manipulable object categories with pose annotations, affordances, and reconstructions

    Andrew Guo, Bowen Wen, Jianhe Yuan, Jonathan Tremblay, Stephen Tyree, Jeffrey Smith, and Stan Birchfield. Handal: A dataset of real-world manipulable object categories with pose annotations, affordances, and reconstructions. In 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 11428–11435. IEEE, 2023. 2, 4, 16

    2023

  21. [21]

    Hdri haven

    HDRI Haven. Hdri haven. https://hdri-haven.com,

  22. [22]

    Momentum contrast for unsupervised visual rep- resentation learning

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual rep- resentation learning. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 9729–9738, 2020. 5

    2020

  23. [23]

    Ditto: Demonstration imitation by trajectory transformation

    Nick Heppert, Max Argus, Tim Welschehold, Thomas Brox, and Abhinav Valada. Ditto: Demonstration imitation by trajectory transformation. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE,

    2024

  24. [24]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models, 2021. 14

    2021

  25. [25]

    Manipvqa: Injecting robotic affordance and physi- cally grounded information into multi-modal large language models

    Siyuan Huang, Iaroslav Ponomarenko, Zhengkai Jiang, Xi- aoqi Li, Xiaobin Hu, Peng Gao, Hongsheng Li, and Hao Dong. Manipvqa: Injecting robotic affordance and physi- cally grounded information into multi-modal large language models. arXiv preprint arXiv:2403.11289, 2024. 1, 3, 7, 8, 15, 16, 17

    Pith/arXiv arXiv 2024

  26. [26]

    Flowformer: A transformer architecture for optical flow

    Zhaoyang Huang, Xiaoyu Shi, Chao Zhang, Qiang Wang, Ka Chun Cheung, Hongwei Qin, Jifeng Dai, and Hongsheng Li. Flowformer: A transformer architecture for optical flow. In European conference on computer vision, pages 668–685. Springer, 2022. 2

    2022

  27. [27]

    Flownet 2.0: Evolu- tion of optical flow estimation with deep networks

    Eddy Ilg, Nikolaus Mayer, Tonmoy Saikia, Margret Keuper, Alexey Dosovitskiy, and Thomas Brox. Flownet 2.0: Evolu- tion of optical flow estimation with deep networks. In Pro- ceedings of the IEEE conference on computer vision and pat- tern recognition, pages 2462–2470, 2017. 2

    2017

  28. [28]

    Cotr: Correspondence transformer for matching across images

    Wei Jiang, Eduard Trulls, Jan Hosang, Andrea Tagliasacchi, and Kwang Moo Yi. Cotr: Correspondence transformer for matching across images. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6207– 6217, 2021. 2

    2021

  29. [29]

    Doduo: Learning dense visual correspondence from unsupervised semantic-aware flow

    Zhenyu Jiang, Hanwen Jiang, and Yuke Zhu. Doduo: Learning dense visual correspondence from unsupervised semantic-aware flow. In 2024 IEEE International Confer- ence on Robotics and Automation (ICRA) , pages 12420– 12427. IEEE, 2024. 1

    2024

  30. [30]

    Robo-abc: Affordance gener- alization beyond categories via semantic correspondence for robot manipulation

    Yuanchen Ju, Kaizhe Hu, Guowei Zhang, Gu Zhang, Min- grun Jiang, and Huazhe Xu. Robo-abc: Affordance gener- alization beyond categories via semantic correspondence for robot manipulation. arXiv preprint arXiv:2401.07487, 2024. 1, 3

    Pith/arXiv arXiv 2024

  31. [31]

    Segment anything in high qual- ity

    Lei Ke, Mingqiao Ye, Martin Danelljan, Yu-Wing Tai, Chi- Keung Tang, Fisher Yu, et al. Segment anything in high qual- ity. Advances in Neural Information Processing Systems, 36,

  32. [32]

    Two-year-olds will name artifacts by their func- tions

    Deborah G Kemler Nelson, Rachel Russell, Nell Duke, and Kate Jones. Two-year-olds will name artifacts by their func- tions. Child development, 71(5):1271–1288, 2000. 3

    2000

  33. [33]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2017. 6

    2017

  34. [34]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 4015–4026, 2023. 5

    work page 2023

  35. [35]

    OmniNOCS: A unified NOCS dataset and model for 3D lifting of 2D objects

    Akshay Krishnan, Abhijit Kundu, Kevis-Kokitsi Maninis, James Hays, and Matthew Brown. Omninocs: A unified nocs dataset and model for 3d lifting of 2d objects. arXiv preprint arXiv:2407.08711, 2024. 1, 2

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  36. [36]

    Ram: Retrieval-based affordance transfer for gen- eralizable zero-shot robotic manipulation

    Yuxuan Kuang, Junjie Ye, Haoran Geng, Jiageng Mao, Congyue Deng, Leonidas Guibas, He Wang, and Yue Wang. Ram: Retrieval-based affordance transfer for gen- eralizable zero-shot robotic manipulation. arXiv preprint arXiv:2407.04689, 2024. 3

    Pith/arXiv arXiv 2024

  37. [37]

    Ge- ometry and context for semantic correspondences and func- tionality recognition in man-made 3d shapes

    Hamid Laga, Michela Mortara, and Michela Spagnuolo. Ge- ometry and context for semantic correspondences and func- tionality recognition in man-made 3d shapes. ACM Transac- tions on Graphics (TOG), 32(5):1–16, 2013. 3

    work page 2013

  38. [38]

    The functional correspondence problem

    Zihang Lai, Senthil Purushwalkam, and Abhinav Gupta. The functional correspondence problem. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 15772–15781, 2021. 1, 2, 3, 4, 13, 18, 19

    work page 2021

  39. [39]

    Object shape, object function, and object name

    Barbara Landau, Linda Smith, and Susan Jones. Object shape, object function, and object name. Journal of mem- ory and language, 38(1):1–27, 1998. 3

    work page 1998

  40. [40]

    Corresnerf: Image correspondence priors for neural radiance fields

    Yixing Lao, Xiaogang Xu, Xihui Liu, Hengshuang Zhao, et al. Corresnerf: Image correspondence priors for neural radiance fields. Advances in Neural Information Processing Systems, 36:40504–40520, 2023. 1

    work page 2023

  41. [41]

    Locate: Localize and transfer object parts for weakly supervised affordance grounding

    Gen Li, Varun Jampani, Deqing Sun, and Laura Sevilla- Lara. Locate: Localize and transfer object parts for weakly supervised affordance grounding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10922–10931, 2023. 2

    work page 2023

  42. [42]

    Learning precise affordances from egocentric videos for robotic manipulation

    Gen Li, Nikolaos Tsagkas, Jifei Song, Ruaridh Mon- Williams, Sethu Vijayakumar, Kun Shao, and Laura Sevilla- Lara. Learning precise affordances from egocentric videos for robotic manipulation. arXiv preprint arXiv:2408.10123,

    arXiv

  43. [43]

    Surface and edge detection for primitive fitting of point clouds

    Yuanqi Li, Shun Liu, Xinran Yang, Jianwei Guo, Jie Guo, and Yanwen Guo. Surface and edge detection for primitive fitting of point clouds. In ACM SIGGRAPH 2023 conference proceedings, pages 1–10, 2023. 14

    work page 2023

  44. [44]

    Sphinx: The joint mixing of weights, tasks, and visual embeddings for multi-modal large language models

    Ziyi Lin, Chris Liu, Renrui Zhang, Peng Gao, Longtian Qiu, Han Xiao, Han Qiu, Chen Lin, Wenqi Shao, Keqin Chen, et al. Sphinx: The joint mixing of weights, tasks, and visual embeddings for multi-modal large language models. arXiv preprint arXiv:2311.07575, 2023. 3, 5

    Pith/arXiv arXiv 2023

  45. [45]

    Pixel-perfect structure-from- motion with featuremetric refinement

    Philipp Lindenberger, Paul-Edouard Sarlin, Viktor Lars- son, and Marc Pollefeys. Pixel-perfect structure-from- motion with featuremetric refinement. In Proceedings of the IEEE/CVF international conference on computer vision, pages 5987–5997, 2021. 1

    work page 2021

  46. [46]

    Learning affordance grounding from exocen- tric images

    Hongchen Luo, Wei Zhai, Jing Zhang, Yang Cao, and Dacheng Tao. Learning affordance grounding from exocen- tric images. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2252–2261,

    work page

  47. [47]

    Scalable 3d captioning with pretrained models

    Tiange Luo, Chris Rockwell, Honglak Lee, and Justin John- son. Scalable 3d captioning with pretrained models. Ad- vances in Neural Information Processing Systems, 36, 2024. 5, 13, 21

    work page 2024

  48. [48]

    TrackNeRF: Bundle Adjusting NeRF from Sparse and Noisy Views via Feature Tracks

    Jinjie Mai, Wenxuan Zhu, Sara Rojas, Jesus Zarzar, Abdul- lah Hamdi, Guocheng Qian, Bing Li, Silvio Giancola, and Bernard Ghanem. Tracknerf: Bundle adjusting nerf from sparse and noisy views via feature tracks. arXiv preprint arXiv:2408.10739, 2024. 1

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  49. [49]

    Spair-71k: A large-scale benchmark for semantic correspon- dence

    Juhong Min, Jongmin Lee, Jean Ponce, and Minsu Cho. Spair-71k: A large-scale benchmark for semantic correspon- dence. arXiv preprint arXiv:1908.10543, 2019. 2, 6

    Pith/arXiv arXiv 1908

  50. [50]

    6-dof graspnet: Variational grasp generation for object manipula- tion

    Arsalan Mousavian, Clemens Eppner, and Dieter Fox. 6-dof graspnet: Variational grasp generation for object manipula- tion. In Proceedings of the IEEE/CVF international confer- ence on computer vision, pages 2901–2910, 2019. 2

    work page 2019

  51. [51]

    Same object, differ- ent grasps: Data and semantic knowledge for task-oriented grasping

    Adithyavairavan Murali, Weiyu Liu, Kenneth Marino, So- nia Chernova, and Abhinav Gupta. Same object, differ- ent grasps: Data and semantic knowledge for task-oriented grasping. In Conference on robot learning , pages 1540–

    work page

  52. [52]

    Affordance detection of tool parts from ge- ometric features

    Austin Myers, Ching L Teo, Cornelia Ferm ¨uller, and Yian- nis Aloimonos. Affordance detection of tool parts from ge- ometric features. In 2015 IEEE International Conference on Robotics and Automation (ICRA) , pages 1374–1381. IEEE,

    work page 2015

  53. [53]

    Grounded human-object interaction hotspots from video

    Tushar Nagarajan, Christoph Feichtenhofer, and Kristen Grauman. Grounded human-object interaction hotspots from video. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 8688–8697, 2019. 2

    work page 2019

  54. [54]

    Caldwell, and Nikos G

    Anh Nguyen, Dimitrios Kanoulas, Darwin G. Caldwell, and Nikos G. Tsagarakis. Object-based affordances detection with convolutional neural networks and dense conditional random fields. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5908–5915,

    work page 2017

  55. [55]

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anad- kat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Bal- com, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, Irwan Bello, Jake Berdine, Gabriel Bernadett- Shapiro, Christopher B...

    work page 2024

  56. [56]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 1, 2, 3, 5, 7, 15, 16, 18

    Pith/arXiv arXiv 2023

  57. [57]

    A threshold selection method from gray-level histograms

    Nobuyuki Otsu et al. A threshold selection method from gray-level histograms. Automatica, 11(285-296):23–27,

    work page

  58. [58]

    Dinobot: Robot ma- nipulation via retrieval and alignment with vision foundation models

    Norman Di Palo and Edward Johns. Dinobot: Robot ma- nipulation via retrieval and alignment with vision foundation models. In IEEE International Conference on Robotics and Automation (ICRA), 2024. 1, 3

    work page 2024

  59. [59]

    Miles: Making imitation learning easy with self-supervision

    Georgios Papagiannis and Edward Johns. Miles: Making imitation learning easy with self-supervision. InProceedings of the Conference on Robot Learning (CoRL), 2024. 3

    work page 2024

  60. [60]

    Film: Visual reasoning with a general conditioning layer, 2017

    Ethan Perez, Florian Strub, Harm de Vries, Vincent Du- moulin, and Aaron Courville. Film: Visual reasoning with a general conditioning layer, 2017. 14

    work page 2017

  61. [61]

    Affordancellm: Grounding affordance from vision language models

    Shengyi Qian, Weifeng Chen, Min Bai, Xiong Zhou, Zhuowen Tu, and Li Erran Li. Affordancellm: Grounding affordance from vision language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7587–7597, 2024. 3

    work page 2024

  62. [62]

    Keto: Learning keypoint representations for tool manipulation

    Zengyi Qin, Kuan Fang, Yuke Zhu, Li Fei-Fei, and Silvio Savarese. Keto: Learning keypoint representations for tool manipulation. In 2020 IEEE International Conference on Robotics and Automation (ICRA) , pages 7278–7285. IEEE,

    work page 2020

  63. [63]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, pages 8748–8763. PMLR, 2021. 3, 5

    2021

  64. [64]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learn- ing Research, 21(140):1–67, 2020. 3

    work page 2020

  65. [65]

    Com- mon objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction

    Jeremy Reizenstein, Roman Shapovalov, Philipp Henzler, Luca Sbordone, Patrick Labatut, and David Novotny. Com- mon objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10901–10911, 2021. 4

    work page 2021

  66. [66]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 1, 3, 7

    work page 2022

  67. [67]

    Superglue: Learning feature matching with graph neural networks

    Paul-Edouard Sarlin, Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. Superglue: Learning feature matching with graph neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4938–4947, 2020. 2

    work page 2020

  68. [68]

    Learning dense object descriptors from multiple views for low-shot category generalization

    Stefan Stojanov, Anh Thai, Zixuan Huang, and James M Rehg. Learning dense object descriptors from multiple views for low-shot category generalization. Advances in Neural In- formation Processing Systems, 35:12566–12580, 2022. 2

    work page 2022

  69. [69]

    Loftr: Detector-free local feature matching with transformers

    Jiaming Sun, Zehong Shen, Yuang Wang, Hujun Bao, and Xiaowei Zhou. Loftr: Detector-free local feature matching with transformers. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 8922–8931, 2021. 2

    work page 2021

  70. [70]

    Misc210k: A large-scale dataset for multi-instance seman- tic correspondence

    Yixuan Sun, Yiwen Huang, Haijing Guo, Yuzhou Zhao, Run- min Wu, Yizhou Yu, Weifeng Ge, and Wenqiang Zhang. Misc210k: A large-scale dataset for multi-instance seman- tic correspondence. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 7121–7130, 2023. 2, 6

    work page 2023

  71. [71]

    Emergent correspondence from image diffusion

    Luming Tang, Menglin Jia, Qianqian Wang, Cheng Perng Phoo, and Bharath Hariharan. Emergent correspondence from image diffusion. Advances in Neural Information Pro- cessing Systems, 36:1363–1389, 2023. 3

    work page 2023

  72. [72]

    Joint re- covery of dense correspondence and cosegmentation in two images

    Tatsunori Taniai, Sudipta N Sinha, and Yoichi Sato. Joint re- covery of dense correspondence and cosegmentation in two images. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4246–4255, 2016. 1

    work page 2016

  73. [73]

    Raft: Recurrent all-pairs field transforms for optical flow

    Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23– 28, 2020, Proceedings, Part II 16, pages 402–419. Springer,

    work page 2020

  74. [74]

    Shape-based transfer of generic skills

    Skye Thompson, Leslie Pack Kaelbling, and Tomas Lozano- Perez. Shape-based transfer of generic skills. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 5996–6002. IEEE, 2021. 3

    work page 2021

  75. [75]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. 3

    Pith/arXiv arXiv 2023

  76. [76]

    Sparf: Neural radiance fields from sparse and noisy poses

    Prune Truong, Marie-Julie Rakotosaona, Fabian Manhardt, and Federico Tombari. Sparf: Neural radiance fields from sparse and noisy poses. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4190–4200, 2023. 1

    work page 2023

  77. [77]

    GIFT: Generalizable Interaction-aware Functional Tool Affordances without Labels

    Dylan Turpin, Liquan Wang, Stavros Tsogkas, Sven Dick- inson, and Animesh Garg. Gift: Generalizable interaction- aware functional tool affordances without labels. arXiv preprint arXiv:2106.14973, 2021. 2

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  78. [78]

    Repre- sentation learning with contrastive predictive coding, 2019

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Repre- sentation learning with contrastive predictive coding, 2019. 6

    work page 2019

  79. [79]

    Knowledge transfer from vision foundation models for efficient training of small task-specific models

    Raviteja Vemulapalli, Hadi Pouransari, Fartash Faghri, Sachin Mehta, Mehrdad Farajtabar, Mohammad Rastegari, and Oncel Tuzel. Knowledge transfer from vision foundation models for efficient training of small task-specific models. In Forty-first International Conference on Machine Learning ,

    work page

  80. [80]

    Socs: Semantically- aware object coordinate space for category-level 6d object pose estimation under large shape variations

    Boyan Wan, Yifei Shi, and Kai Xu. Socs: Semantically- aware object coordinate space for category-level 6d object pose estimation under large shape variations. InProceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 14065–14074, 2023. 2

    work page 2023

Showing first 80 references.

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.