Pith. sign in

REVIEW 4 major objections 5 minor 3 cited by

DenseMatcher: Learning 3D Semantic Correspondence for Category-Level Manipulation from a Single Demo

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Pith's one-line read DenseMatcher computes dense 3D semantic correspondences between everyday objects, using multiview 2D foundation-model features refined by a 3D network and functional map, and reports that this transfers a single human demonstration to new…

desk verdict Sensible pipeline and a useful dataset, but the cross-category headline is not backed by any quantitative cross-category pair in the benchmark. read the letter →

arxiv 2412.05268 v1 pith:VNSQKKPT submitted 2024-12-06 cs.RO cs.CV

classification cs.ROcs.CV
keywords 3Dsemanticcorrespondencefunctionalmapsingle-demoroboticmanipulationDiffusionNetSD-DINOfeaturesdensematchingdatasetcolortransfercross-categorygeneralization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that dense semantic correspondence—matching parts of different objects by what they mean rather than how they look—can be computed accurately for everyday objects and can transfer a manipulation plan from one human demonstration to a new object, including objects from categories the system has never trained on. DenseMatcher lifts frozen 2D foundation-model features onto mesh vertices from multiple rendered views, refines them with a lightweight 3D network, and solves a functional-map optimization to get a dense source-to-target mapping. To support this, the paper releases DenseCorr3D, a benchmark of 589 colored meshes across 23 categories with dense semantic-group annotations, and reports a 43.5% improvement over prior shape-matching baselines on it. If the claims hold, a robot could learn long-horizon tasks such as peeling a banana or arranging flowers from a single demonstration, and 3D assets could borrow colors from other objects with related shapes.

What carries the argument

The load-bearing machinery is a three-stage pipeline: multiview SD-DINO features are projected onto mesh vertices and averaged; a trainable DiffusionNet refines them under a semantic-distance loss and a feature-preservation loss; and functional map—a low-rank representation of a map between surfaces via Laplace-Beltrami eigenfunctions—recovers dense correspondences with two added regularizers that force the point-to-point map to be sparse and to behave like a soft assignment. The paper proves that minimizing the functional-map feature-matching term minimizes total semantic-group distance between matched vertices, so the design makes 'closer in feature space' mean 'closer in semantic meaning'.

What would settle it

Run DenseMatcher on a held-out banana mesh and an eggplant mesh and check whether the dense map sends the banana's stem to the eggplant's stem and the banana's body to the eggplant's body; if the normalized geodesic error to the correct semantic group is much larger than the reported 2.82 held-out average, the claim that pretrained 2D features carry cross-category semantics is contradicted.

Watch

Extended reading notes

Core claim

The central claim is that semantic 3D correspondence across in-the-wild daily objects can be obtained by combining the generalization of pretrained 2D visual features with geometric refinement and a carefully regularized functional map. The paper argues that previous approaches fail because they rely either on category-specific geometry, which does not transfer, or on naively averaged multiview 2D features, which are noisy and globally inconsistent. DenseMatcher instead projects SD-DINO features from several views onto mesh vertices, averages only visible views, concatenates Heat Kernel Signature and positional encoding, and trains a lightweight DiffusionNet to make features respect semantic groups while preserving the 2D model's rich information. Dense correspondences are then recovered by functional map with additional sparsity and soft-assignment constraints. The paper reports AUC 0.845 on its benchmark versus 0.589 for the strongest prior baseline, cross-category generalization on held-out categories, real-robot success of 76.7% averaged over six single-demo long-horizon tasks, and zero-shot color transfer between assets.

Load-bearing premise

The load-bearing premise is that the visual features from a pretrained image model stay semantically aligned across different object categories, so that the stem of a banana and the stem of an eggplant produce similar features; if that alignment fails for unseen pairs, the 3D refinement has no trustworthy signal to amplify.

Editorial extensions

If this is right

  • A robot can transfer contact points and grasp sequences from one human video to a new object instance, including objects from categories not seen in training, with no additional demonstrations.
  • The method works on held-out categories such as celery, cucumber, and eggplant with zero training examples, because the frozen 2D backbone preserves cross-category semantic knowledge.
  • Dense correspondences enable zero-shot color transfer between 3D assets with relatable geometry, which the paper reports as a new capability in 3D generation.
  • The proposed functional-map regularizers produce spatially smooth maps, unlike nearest-neighbor or Hungarian feature matching, which yield speckled mismatches.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A test the paper leaves implicit is to swap SD-DINO for a different frozen vision backbone and check whether held-out accuracy tracks the backbone's semantic alignment rather than the 3D refiner's capacity.
  • Because the full pipeline needs a mesh reconstructed from RGB-D, the single-demo promise in real robotics is bounded by mesh quality; evaluating directly on partial or noisy meshes would set a more honest practical ceiling.
  • Semantic groups are user-defined, so the same machinery could transfer to other grouping criteria—functional parts, affordances, materials—provided the 2D features encode them; this points toward affordance transfer beyond the six demonstrated tasks.
  • The 43.5% margin is measured on the authors' own benchmark; reproducing it on an independent, larger set of categories with stronger topological variation would test whether the margin holds.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The authors present DenseMatcher, a method for dense 3D semantic correspondence between textured objects. The pipeline renders multiview SD-DINO features, projects and averages them onto remeshed objects, refines the per-vertex features with a trainable DiffusionNet, and computes correspondences with a functional map solver augmented by entropy and row/column assignment regularizers. They also introduce DenseCorr3D, a new benchmark of 589 colored meshes across 23 categories with semantic-group annotations, and evaluate DenseMatcher on this benchmark, on ablations, on six real-robot manipulation tasks, and on qualitative color-transfer examples. The paper reports a 43.5% improvement over prior shape-matching baselines and a 76.7% overall robot task success rate.

Significance. The paper addresses an important problem: transferring manipulation-relevant semantic knowledge across object instances and categories from a single demonstration. The technical combination of frozen 2D foundation-model features with a trainable 3D refiner is sensible, and the new textured benchmark fills a real gap in the 3D-correspondence literature. The functional-map refinements are clearly described, and the ablation study is informative. The real-robot evaluation, although small-scale, is a genuine strength compared with simulation-only studies. However, the evidence currently falls short of the paper's headline claims: the quantitative benchmark does not measure cross-category pairs, and the robot pipeline manually supplies trajectory waypoints, so the 'one demo' framing is overstated. The theoretical appendix also rests on an unverified exact-proportionality assumption. If the authors can add quantitative cross-category evaluation, clarify the manual components of the robot pipeline, and either verify or weaken the proof's assumption, the contribution would be solid.

major comments (4)
  1. [§6.1.2, Table 1] The abstract and Section 1 claim 'cross-instance and cross-category generalization,' but the quantitative benchmark only evaluates pairs of instances within the same category. The held-out columns (celery, cucumber, eggplant in Table 4) test generalization to unseen categories, but all correspondence pairs are still within-category; no cross-category pair such as banana→eggplant is scored with AUC/Err. The only quantitative cross-category robot example (panda→dog in 'Point Object Parts with Pen', Table 3) is folded into a task-level success rate and not reported per pair. The 'w/o DiffusionNet' row (held-out AUC 0.662) shows the frozen 2D features carry in-category signal but does not establish cross-category alignment, which is the mechanism claimed in §4.2.1. Please add quantitative cross-category evaluations (e.g., pairs spanning the fruit/vegetable categories and the color-transfer examples) or restrict the claim to cross-instance generalization.
  2. [§6.2.1 and Abstract] The paper repeatedly states that the robot performs tasks 'from observing only one demo,' but the manipulation pipeline does not derive the full behavior from the demo. Contact points are extracted from the human video, yet the text says 'We provide the waypoints of the trajectory after grasping and the final location to move to after completing the grasp.' Thus the trajectory and goal are manually specified, and AnyGrasp supplies the grasp pose. The contribution should be described as transferring contact points from one demo, with the remaining trajectory elements hand-specified; otherwise the single-demo claim is overstated.
  3. [§4.3.1 and §A.4.2] The proof in A.4.2 relies on Eq. (13), which assumes that after training the L2 feature distance is exactly proportional to the semantic distance with a constant s. However, the training loss Lsemantic is a negative cosine similarity over sampled pairs of distance magnitudes, and the features are unit-normalized, so exact proportionality is not guaranteed; the preservation loss Lpreservation (§4.3.2) can also pull features away from pure semantic alignment. The theorem is therefore conditional on a strong assumption that is not verified. Please either state the assumption as an approximation and quantify the residual (e.g., report the correlation between ∥f(vi)−f(vj)∥ and Dsemantic on a validation set), or weaken the claim that functional-map matching provably minimizes semantic distance.
  4. [§5.2 and Table 1] The evaluation metric uses the same semantic-group/geodesic-distance structure that the training loss optimizes (distance of the prediction to the nearest ground-truth semantic group). This alignment can inflate DenseMatcher's margin over baselines that were not trained with this objective, so the headline 43.5% improvement is partly self-referential. Independent grounding is needed, for example human-annotated keypoint transfer accuracy, per-pair success statistics in the robot experiments that isolate correspondence quality, or a held-out metric not used in the loss. Additionally, Table 1 reports single point estimates without variance over dataset splits, seeds, or mesh pairs, so the 'significant outperformance' language is not statistically supported.
minor comments (5)
  1. [Table 2] The header 'Multiple Objets' should be 'Multiple Objects'.
  2. [§6.4] The ablation list is numbered '(i), (ii), (iii), (ii)'; the fourth item should be '(iv)'.
  3. [Equation (1)] The weights α and β are used in Equation (1) but their values are only given in A.2.2; please define them at first use and also state the weights of the two new regularizers in the main text.
  4. [Reproducibility] The paper does not state whether code, benchmark data, and model weights will be released; the project website link alone is not sufficient for reproducibility. Please add a reproducibility statement.
  5. [A.3.2] The text says 'We pre-train FeatUp parameters for 10,000 steps on ImageNet'; FeatUp normally learns a per-image upsampler, so please clarify which subset of FeatUp is pre-trained and whether the pre-training uses ImageNet images or the rendered multiview images.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity found: the method's outputs are not read off from the ground-truth semantic groups, and the held-out categories plus real-robot and color-transfer experiments provide evidence outside the training objective.

full rationale

The derivation chain is: frozen SD-DINO multiview features are projected onto each mesh, refined by a trainable DiffusionNet, and then converted to dense correspondences by a functional-map optimization. At inference, the inputs are only the source/target meshes and the pretrained 2D features; the ground-truth semantic groups are not fed into the pipeline. The training loss Lsemantic (Section 4.3.1) does directly fit pairwise feature distances to the semantic distance Dsemantic, and Section 5.2 scores matches by distance to the nearest vertex in the ground-truth semantic group, so the evaluation metric is well aligned with the learning objective. That alignment is standard supervised learning rather than a circular prediction: the held-out categories (celery, cucumber, eggplant) have zero training instances in DenseCorr3D, so the reported held-out AUC 0.775 must come from extrapolation through the frozen SD-DINO features and geometric priors, not from the loss directly. The real-robot manipulation results and the zero-shot color transfer examples are outside the benchmark objective and give independent, if anecdotal, grounding. The paper contains self-citations (SD-DINO by coauthor Junyi Zhang; Robo-ABC by coauthors Ju, Hu, and Xu), but neither is load-bearing in a circular way: SD-DINO has its own published evaluation and is used as an off-the-shelf frozen backbone, while Robo-ABC is a comparison baseline. The proof in A.4.2 is a restatement of the loss design rather than an independent first-principles theorem, and it contains a looseness (the inequality is only an equality with full-rank bases), but this is a rigor concern, not circularity. Finally, the abstract's cross-category claim is supported more thinly than the headline suggests because Table 1 evaluates only within-category pairs and the one cross-category robot pair is folded into a task-level success rate; that is an evidence gap to weigh in correctness assessment, not a circularity of the derivation.

Assumptions & free parameters 8 free parameters · 7 assumptions · 0 invented entities

The central claim rests on standard spectral geometry, a hand-annotated semantic-group dataset, and an unverified assumption that frozen 2D features transfer across categories. The main free parameters are functional-map regularization weights and representation sizes; none are fit to the test benchmark, but several are chosen by hand.

free parameters (8)
  • Functional map isometry regularization weight alpha = 1e-2
    Set in A.2.2 to balance the isometry commutativity term in Eq. (1); chosen by hand, not learned or swept.
  • Functional map point-to-point regularization weight beta = 1e-4
    Set in A.2.2 for commutativity with product operators.
  • Entropy sparsity penalty weight = 1e-5
    Added in Section 4.4 to promote sparse soft-assignment Pi; chosen by hand.
  • Row/column sum-to-1 assignment penalty weight = 1e-3
    Enforces soft-assignment marginals in Section 4.4; chosen by hand.
  • Number of Laplace-Beltrami eigenfunctions k = 10
    Used as spectral basis for functional map; set in A.2.2.
  • Number of rendered views per mesh = 5 (3 lateral, 1 top, 1 bottom)
    Used for multiview feature aggregation in Section 4.2.1 and A.3.2; fewer views than Diff3F's 100, enabled by 3D refiner.
  • Remesh target vertex count = ~2000 nominal; 500-2500 during training
    Simplified geometry fed to DiffusionNet; randomization used for robustness (Section 4.2.2, A.3.2).
  • DiffusionNet training steps = 6000 steps, batch size 8, ~50 epochs
    Training schedule reported in A.3.2; not swept, but part of the trained model.
assumptions (7)
  • standard math Laplace-Beltrami eigenfunctions are orthonormal with respect to the area-weighted inner product <x,y> = x^T A y.
    Used in A.4.1 to define spectral projection and pseudo-inverse Phi^+ = Phi^T A.
  • standard math A diffeomorphism between manifolds is isometric if and only if it commutes with the Laplace-Beltrami operator.
    Used in A.4.3 to justify the isometry constraint alpha * ||Lambda_N C - C Lambda_M||^2.
  • domain assumption A low-rank functional map Pi ~= Phi_N C Phi_M^+ with k eigenvectors captures the correspondence well enough for daily objects.
    The entire functional-map solver (Section 4.1) relies on this approximation; k=10 is used.
  • domain assumption Semantic group annotations in DenseCorr3D are consistent across instances within a category and reflect the intended correspondence.
    The dataset and benchmark are built from these annotations (Section 5); if annotations are noisy, the training loss and evaluation metric are both affected.
  • domain assumption Pretrained 2D foundation models (DINOv2 + Stable Diffusion, combined as SD-DINO) provide semantically meaningful, category-agnostic features when lifted to 3D.
    The method's generalization relies on this; see Section 4.2.1 and the claim of generalization due to 'pre-trained 2D backbones' in Section 6.1.2.
  • ad hoc to paper After training, feature L2 distances become exactly proportional to semantic distances with a constant s.
    Assumed in A.4.2 to prove that functional-map feature matching minimizes semantic distance; this is a strong idealization and is not empirically demonstrated.
  • domain assumption Geodesic distances computed on remeshed geometry are accurate enough for semantic distance and evaluation.
    Used for Dsemantic in A.2.1 and for evaluation metric in Section 5.2; error from remeshing is not quantified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DenseMatcher: Learning 3D Semantic Correspondence for Category-Level Manipulation from a Single Demo." pith.science (2026). https://pith.science/paper/VNSQKKPT

@misc{pith2026241205268,
  author       = {Pith},
  title        = {Pith review of: DenseMatcher: Learning 3D Semantic Correspondence for Category-Level Manipulation from a Single Demo},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VNSQKKPT}},
  note         = {Machine review of arXiv:2412.05268}
}
read the original abstract

Dense 3D correspondence can enhance robotic manipulation by enabling the generalization of spatial, functional, and dynamic information from one object to an unseen counterpart. Compared to shape correspondence, semantic correspondence is more effective in generalizing across different object categories. To this end, we present DenseMatcher, a method capable of computing 3D correspondences between in-the-wild objects that share similar structures. DenseMatcher first computes vertex features by projecting multiview 2D features onto meshes and refining them with a 3D network, and subsequently finds dense correspondences with the obtained features using functional map. In addition, we craft the first 3D matching dataset that contains colored object meshes across diverse categories. In our experiments, we show that DenseMatcher significantly outperforms prior 3D matching baselines by 43.5%. We demonstrate the downstream effectiveness of DenseMatcher in (i) robotic manipulation, where it achieves cross-instance and cross-category generalization on long-horizon complex manipulation tasks from observing only one demo; (ii) zero-shot color mapping between digital assets, where appearance can be transferred between different objects with relatable geometry.

Figures

Figures reproduced from arXiv: 2412.05268 by the authors.

Figure 1
Figure 1. (a) Zero-shot color transfer between 3D assets. (b) In real-world robotic experi￾ments, we use DenseMatcher to transfer a manipulation sequence to the robot from a single human demonstration. Circles represent the contact points in the human demo / grasping points for robot manipulation. ABSTRACT Dense 3D correspondence can enhance robotic manipulation by enabling the gen￾eralization of spatial, functional, and dyna… view at source ↗
Figure 2
Figure 2. The 4 types of correspon￾dence. The reference image is on the left, while the right side demonstrates 1) 3D dense, 2) 3D sparse, 3) 2D dense, and 4) 2D sparse correspondences. Among these types, 3D dense correspondence is particu￾larly advantageous for robotic manipulation, as it ensures continuity by smooth mappings between surfaces. This is crucial for tasks requiring precise multi-point contact and positioning. A… view at source ↗
Figure 3
Figure 3. Predicted correspondences on few-shot categories. DenseMatcher can generalize across diverse topological variations, given only 5 training examples per category. To ensure that the model is not reliant on canonical spatial poses, we randomly rotate the mesh before the test procedure [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Semantic group annotations examples of apple, banana, animals (deer, tiger, elephant), and chairs. Different colors represent different semantic groups across the same category. DenseC￾orr3D contains objects of varying topologies and structures, both across and within …
Figure 5
Figure 5. Figure 5: Two possible partitioning schemes for a hand are shown. The definition of correspondence is inherently subjective. For instance, elephant tusks and rhino tusks can corre￾spond based on function, while an elephant’s nose can correspond to a rhino’s tusk based on locatio…
Figure 6
Figure 6. Figure 6: DenseMatcher model architecture. SD-DINO (Zhang et al., 2023) fuses 2D features from DINOv2 and Stable Diffusion, which are aggregated and fed into a trainable DiffusionNet. Correspondences are computed from source and target features using functional map. The detailed…
Figure 7
Figure 7. Figure 7: Real-World Experiment Workflow. We obtain template mesh and contact points from a human demonstration with hand-object detector (Shan et al., 2020). We then transfer these contact points onto the target mesh with DenseMatcher. Finally, we use off-the-shelf AnyGrasp (Fa…
Figure 8
Figure 8. Figure 8: KeyFrames of 6 robotic tasks. The tasks from top to bottom are organizing the shoes, pointing object parts with pen, pulling out the carrot, putting flower into a vase, peeling a banana and decorating the the Christmas tree [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: Color transfer results between (i) banana and eggplant, (ii) tomato and kabocha squash, and (iii) wine bottles (iii) gloves. 6.2.2 BASELINE Robo-ABC (Ju et al., 2024) utilizes correspondences found in RGB images to transfer affordances. Since Robo-ABC has its own colle…
Figure 10
Figure 10. Figure 10: Ablation study on dense correspondence results. (a) Effect of using different features (HKS, WKS) with functional maps. (b) Comparison of matching methods using the same features. 6.3 COLOR TRANSFER EXPERIMENTS Ofri-Amar et al. (2023) shows that dense correspondences …
Figure 11
Figure 11. Figure 11: Robot experiments visualization under occlusion conditions. A.5.2 PARTIAL SOURCE AND FULL TARGET In the second case, the source mesh is a partial mesh, and the target mesh is a full mesh. We follow the formulation of partial functional correspondence (Rodola et al. ` …
Figure 12
Figure 12. Figure 12: Correspondence between partial mesh with full mesh. DenseMatcher is capable of matching a partial mesh to a full mesh by utilizing the partial functional correspondence formulation. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_12.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MeshFM: 2D Features Are All You Need for 3D Shape Understanding

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A feedforward network trained only on 2D foundation-model features predicts rotation-robust, general-purpose 3D mesh features that work zero-shot for segmentation, correspondence, and deformation.

  2. AffordGen: Generating Diverse Demonstrations for Generalizable Object Manipulation with Afford Correspondence

    cs.RO 2026-04 unverdicted novelty 6.0 of 10

    AffordGen synthesizes large-scale affordance-aware manipulation trajectories via keypoint correspondence on 3D meshes, enabling zero-shot visuomotor policies for unseen objects from few source demos.

  3. ControlVLA: Few-shot Object-centric Adaptation for Pre-trained Vision-Language-Action Models

    cs.RO 2025-06 conditional novelty 6.0 of 10

    ControlVLA adapts a DROID-pretrained diffusion VLA policy to new manipulation tasks with 10 to 20 demos by injecting object-centric features through zero-initialized cross-attention layers, achieving 76.7% success acr...

Reference graph

Works this paper leans on

81 extracted references · 51 canonical work pages · cited by 3 Pith papers

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...

  3. [3]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...

  4. [4]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...

  5. [5]

    Alliez, E.C

    P. Alliez, E.C. de Verdire, O. Devillers, and M. Isenburg. Isotropic surface remeshing. In 2003 Shape Modeling International., pp.\ 49--58, 2003. doi:10.1109/SMI.2003.1199601

  6. [6]

    Deep vit features as dense visual descriptors

    Shir Amir, Yossi Gandelsman, Shai Bagon, and Tali Dekel. Deep vit features as dense visual descriptors. arXiv preprint arXiv:2112.05814, 2 0 (3): 0 4, 2021

  7. [7]

    The wave kernel signature: A quantum mechanical approach to shape analysis

    Mathieu Aubry, Ulrich Schlickewei, and Daniel Cremers. The wave kernel signature: A quantum mechanical approach to shape analysis. In ICCV workshops, pp.\ 1626--1633. IEEE, 2011

  8. [8]

    Affordances from human videos as a versatile representation for robotics

    Shikhar Bahl, Russell Mendonca, Lili Chen, Unnat Jain, and Deepak Pathak. Affordances from human videos as a versatile representation for robotics. In CVPR, pp.\ 13778--13790, 2023

Show all 81 references
  1. [9]

    Bernardini, J

    F. Bernardini, J. Mittleman, H. Rushmeier, C. Silva, and G. Taubin. The ball-pivoting algorithm for surface reconstruction. IEEE TVCG, 5 0 (4): 0 349--359, 1999. doi:10.1109/2945.817351

  2. [10]

    Faust: Dataset and evaluation for 3d mesh registration

    Federica Bogo, Javier Romero, Matthew Loper, and Michael J Black. Faust: Dataset and evaluation for 3d mesh registration. In CVPR, pp.\ 3794--3801, 2014 a

  3. [11]

    Federica Bogo, Javier Romero, Matthew Loper, and Michael J. Black. Faust: Dataset and evaluation for 3d mesh registration. In CVPR, pp.\ 3794--3801, 2014 b . 10.1109/ CVPR .2014.491

  4. [12]

    Bronstein, Michael M

    Alexander M. Bronstein, Michael M. Bronstein, and Ron Kimmel. Numerical geometry of non-rigid shapes. In Monographs in Computer Science, 2009. URL https://api.semanticscholar.org/CorpusID:33705207

  5. [13]

    Unsupervised deep multi-shape matching

    Dongliang Cao and Florian Bernard. Unsupervised deep multi-shape matching. In ECCV, pp.\ 55--71. Springer, 2022

  6. [14]

    Unsupervised learning of robust spectral shape matching

    Dongliang Cao, Paul Roetzer, and Florian Bernard. Unsupervised learning of robust spectral shape matching. arXiv preprint arXiv:2304.14419, 2023

  7. [15]

    Emerging properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv \'e J \'e gou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In ICCV, pp.\ 9650--9660, 2021

  8. [16]

    Zero-shot image feature consensus with deep functional maps

    Xinle Cheng, Congyue Deng, Adam Harley, Yixin Zhu, and Leonidas Guibas. Zero-shot image feature consensus with deep functional maps. In ECCV, 2024

  9. [17]

    Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots

    Cheng Chi, Zhenjia Xu, Chuer Pan, Eric Cousineau, Benjamin Burchfiel, Siyuan Feng, Russ Tedrake, and Shuran Song. Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots. arXiv preprint arXiv:2402.10329, 2024

  10. [18]

    Reducing the barrier to entry of complex robotic software: a moveit! case study

    David Coleman, Ioan Sucan, Sachin Chitta, and Nikolaus Correll. Reducing the barrier to entry of complex robotic software: a moveit! case study. arXiv preprint arXiv:1404.3785, 2014

  11. [19]

    Geodesics in heat

    Keenan Crane, Clarisse Weischedel, and Max Wardetzky. Geodesics in heat. CoRR, abs/1204.6216, 2012. URL http://arxiv.org/abs/1204.6216

  12. [20]

    Objaverse-xl: A universe of 10m+ 3d objects, 2023

    Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram Voleti, Samir Yitzhak Gadre, Eli VanderBilt, Aniruddha Kembhavi, Carl Vondrick, Georgia Gkioxari, Kiana Ehsani, Ludwig Schmidt, and Ali Farhadi. Objavers...

  13. [21]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, pp.\ 248--255, 2009. 10.1109/ CVPR .2009.5206848

  14. [22]

    Deep geometric functional maps: Robust feature learning for shape correspondence

    Nicolas Donati, Abhishek Sharma, and Maks Ovsjanikov. Deep geometric functional maps: Robust feature learning for shape correspondence. In CVPR, pp.\ 8592--8601, 2020

  15. [23]

    Niladri Shekhar Dutt, Sanjeev Muralikrishnan, and Niloy J. Mitra. Diffusion 3d features (diff3f): Decorating untextured shapes with distilled semantic features, 2024. URL https://arxiv.org/abs/2311.17024

  16. [24]

    Dyke, Caleb Stride, Yu-Kun Lai, and Paul L

    Roberto M. Dyke, Caleb Stride, Yu-Kun Lai, and Paul L. Rosin. Shrec’19: Shape correspondence with isometric and non-isometric deformations, 2019

  17. [25]

    Anygrasp: Robust and efficient grasp perception in spatial and temporal domains

    Hao-Shu Fang, Chenxi Wang, Hongjie Fang, Minghao Gou, Jirong Liu, Hengxu Yan, Wenhai Liu, Yichen Xie, and Cewu Lu. Anygrasp: Robust and efficient grasp perception in spatial and temporal domains. IEEE Transactions on Robotics, 2023

  18. [26]

    Dense object nets: Learning dense visual object descriptors by and for robotic manipulation

    Peter R Florence, Lucas Manuelli, and Russ Tedrake. Dense object nets: Learning dense visual object descriptors by and for robotic manipulation. arXiv preprint arXiv:1806.08756, 2018

  19. [27]

    Brandt, Axel Feldmann, Zhoutong Zhang, and William T

    Stephanie Fu, Mark Hamilton, Laura E. Brandt, Axel Feldmann, Zhoutong Zhang, and William T. Freeman. Featup: A model-agnostic framework for features at any resolution. In ICLR, 2024. URL https://openreview.net/forum?id=GkJiNn2QDF

  20. [28]

    Riemann: Near real-time se (3)-equivariant robot manipulation without point cloud segmentation

    Chongkai Gao, Zhengrong Xue, Shuying Deng, Tianhai Liang, Siqi Yang, Lin Shao, and Huazhe Xu. Riemann: Near real-time se (3)-equivariant robot manipulation without point cloud segmentation. CoRL, 2024 a

  21. [29]

    Can pre-trained text-to-image models generate visual goals for reinforcement learning? NeurIPS, 36, 2024 b

    Jialu Gao, Kaizhe Hu, Guowei Xu, and Huazhe Xu. Can pre-trained text-to-image models generate visual goals for reinforcement learning? NeurIPS, 36, 2024 b

  22. [30]

    3 d - coded : 3 d c orrespondences by d eep d eformation

    Thibault Groueix, Matthew Fisher, Vladimir G Kim, Bryan C Russell, and Mathieu Aubry. 3 d - coded : 3 d c orrespondences by d eep d eformation. In ECCV, pp.\ 230--246, 2018 a

  23. [31]

    Kim, Bryan C

    Thibault Groueix, Matthew Fisher, Vladimir G. Kim, Bryan C. Russell, and Mathieu Aubry. 3d-coded : 3d correspondences by deep deformation, 2018 b . URL https://arxiv.org/abs/1806.05228

  24. [32]

    Unsupervised learning of dense shape correspondence

    Oshri Halimi, Or Litany, Emanuele Rodola, Alex M Bronstein, and Ron Kimmel. Unsupervised learning of dense shape correspondence. In CVPR, pp.\ 4370--4379, 2019

  25. [33]

    Proposal flow: Semantic correspondences from object proposals

    Bumsub Ham, Minsu Cho, Cordelia Schmid, and Jean Ponce. Proposal flow: Semantic correspondences from object proposals. IEEE TPAMI, 40 0 (7): 0 1711--1725, 2017

  26. [34]

    Unsupervised semantic correspondence using stable diffusion

    Eric Hedlin, Gopal Sharma, Shweta Mahajan, Hossam Isack, Abhishek Kar, Andrea Tagliasacchi, and Kwang Moo Yi. Unsupervised semantic correspondence using stable diffusion. In NeurIPS, 2023

  27. [35]

    Stem-ob: Generalizable visual imitation learning with stem-like convergent observation through diffusion inversion

    Kaizhe Hu, Zihang Rui, Yao He, Yuyao Liu, Pu Hua, and Huazhe Xu. Stem-ob: Generalizable visual imitation learning with stem-like convergent observation through diffusion inversion. arXiv preprint arXiv:2411.04919, 2024

  28. [36]

    Robo-abc: Affordance generalization beyond categories via semantic correspondence for robot manipulation

    Yuanchen Ju, Kaizhe Hu, Guowei Zhang, Gu Zhang, Mingrun Jiang, and Huazhe Xu. Robo-abc: Affordance generalization beyond categories via semantic correspondence for robot manipulation. ECCV, 2024

  29. [37]

    Kazhdan, Matthew Bolitho, and Hugues Hoppe

    Michael M. Kazhdan, Matthew Bolitho, and Hugues Hoppe. Poisson surface reconstruction. In Alla Sheffer and Konrad Polthier (eds.), Proceedings of the Fourth Eurographics Symposium on Geometry Processing, volume 256 of SGP '06, pp.\ 61--70, Aire-la-Ville, Switzerland, Switzerla...

  30. [38]

    One-shot imitation under mismatched execution

    Kushal Kedia, Prithwish Dan, and Sanjiban Choudhury. One-shot imitation under mismatched execution. arXiv preprint arXiv:2409.06615, 2024

  31. [39]

    Robot see robot do: Imitating articulated object manipulation with monocular 4d reconstruction

    Justin Kerr, Chung Min Kim, Mingxuan Wu, Brent Yi, Qianqian Wang, Ken Goldberg, and Angjoo Kanazawa. Robot see robot do: Imitating articulated object manipulation with monocular 4d reconstruction. arXiv preprint arXiv:2409.18121, 2024

  32. [40]

    Blended intrinsic maps

    Vladimir G Kim, Yaron Lipman, and Thomas Funkhouser. Blended intrinsic maps. TOG, 30 0 (4): 0 1--12, 2011

  33. [41]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. CoRR, abs/1412.6980, 2014. URL https://api.semanticscholar.org/CorpusID:6628106

  34. [42]

    Ram: Retrieval-based affordance transfer for generalizable zero-shot robotic manipulation, 2024

    Yuxuan Kuang, Junjie Ye, Haoran Geng, Jiageng Mao, Congyue Deng, Leonidas Guibas, He Wang, and Yue Wang. Ram: Retrieval-based affordance transfer for generalizable zero-shot robotic manipulation, 2024. URL https://arxiv.org/abs/2407.04689

  35. [43]

    Ag2manip: Learning novel manipulation skills with agent-agnostic visual and action representations

    Puhao Li, Tengyu Liu, Yuyang Li, Muzhi Han, Haoran Geng, Shu Wang, Yixin Zhu, Song-Chun Zhu, and Siyuan Huang. Ag2manip: Learning novel manipulation skills with agent-agnostic visual and action representations. arXiv preprint arXiv:2404.17521, 2024

  36. [44]

    Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis

    Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis. In 3DV, 2024

  37. [45]

    Diffusion hyperfeatures: Searching through time and space for semantic correspondence

    Grace Luo, Lisa Dunlap, Dong Huk Park, Aleksander Holynski, and Trevor Darrell. Diffusion hyperfeatures: Searching through time and space for semantic correspondence. In NeurIPS, 2023

  38. [46]

    Diffusion hyperfeatures: Searching through time and space for semantic correspondence

    Grace Luo, Lisa Dunlap, Dong Huk Park, Aleksander Holynski, and Trevor Darrell. Diffusion hyperfeatures: Searching through time and space for semantic correspondence. NeurIPS, 36, 2024

  39. [47]

    Discrete differential-geometry operators for triangulated 2-manifolds

    Mark Meyer, Mathieu Desbrun, Peter Schr \"o der, and Alan H Barr. Discrete differential-geometry operators for triangulated 2-manifolds. In Visualization and mathematics III, pp.\ 35--57. Springer, 2003

  40. [48]

    Nerf: Representing scenes as neural radiance fields for view synthesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65 0 (1): 0 99--106, 2021

  41. [49]

    PyMeshLab , January 2021

    Alessandro Muntoni and Paolo Cignoni. PyMeshLab , January 2021

  42. [50]

    Informative descriptor preservation via commutativity for shape matching

    Dorian Nogneng and Maks Ovsjanikov. Informative descriptor preservation via commutativity for shape matching. In Computer Graphics Forum, volume 36, pp.\ 259--267. Wiley Online Library, 2017

  43. [51]

    Neural congealing: Aligning images to a joint semantic atlas

    Dolev Ofri-Amar, Michal Geyer, Yoni Kasten, and Tali Dekel. Neural congealing: Aligning images to a joint semantic atlas. In CVPR, pp.\ 19403--19412, 2023

  44. [53]

    Dinov2: Learning robust visual features without supervision, 2024

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Mahmoud Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabba...

  45. [54]

    Functional maps: a flexible representation of maps between shapes

    Maks Ovsjanikov, Mirela Ben-Chen, Justin Solomon, Adrian Butscher, and Leonidas Guibas. Functional maps: a flexible representation of maps between shapes. TOG, 31 0 (4): 0 1--11, 2012

  46. [55]

    Learning so (3)-invariant semantic correspondence via local shape transform

    Chunghyun Park, Seungwook Kim, Jaesik Park, and Minsu Cho. Learning so (3)-invariant semantic correspondence via local shape transform. In CVPR, pp.\ 22978--22987, 2024

  47. [56]

    Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas K \" o pf, Edward Z. Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner...

  48. [57]

    Ioannis Pratikakis, Michalis A. Savelonas, Fotis Arnaou TOG lou, George Alexios Ioannakis, Anestis Koutsoudis, Theoharis Theoharis, Minh-Triet Tran, Vinh-Tiep Nguyen, Viet-Khoi Pham, Hai-Dang Nguyen, Ho \`a ng- \^A n L \^e , Ba-Huu Tran, Quan H. To, Minh-Bao Truong, Thuyen V. ...

  49. [58]

    Charles Ruizhongtai Qi, Hao Su, Kaichun Mo, and Leonidas J. Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. CoRR, abs/1612.00593, 2016. URL http://arxiv.org/abs/1612.00593

  50. [59]

    Continuous and orientation-preserving correspondences via functional maps

    Jing Ren, Adrien Poulenard, Peter Wonka, and Maks Ovsjanikov. Continuous and orientation-preserving correspondences via functional maps. TOG, 37 0 (6): 0 1--16, 2018

  51. [60]

    Partial functional correspondence

    Emanuele Rodol \`a , Luca Cosmo, Michael M Bronstein, Andrea Torsello, and Daniel Cremers. Partial functional correspondence. In Computer graphics forum, volume 36, pp.\ 222--236. Wiley Online Library, 2017

  52. [61]

    Spidermatch: 3d shape matching with global optimality and geometric consistency

    Paul Roetzer and Florian Bernard. Spidermatch: 3d shape matching with global optimality and geometric consistency. In CVPR, 2024

  53. [62]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj \" o rn Ommer. High-resolution image synthesis with latent diffusion models. CoRR, abs/2112.10752, 2021. URL https://arxiv.org/abs/2112.10752

  54. [63]

    Understanding human hands in contact at internet scale

    Dandan Shan, Jiaqi Geng, Michelle Shu, and David F Fouhey. Understanding human hands in contact at internet scale. In CVPR, pp.\ 9869--9878, 2020

  55. [64]

    Diffusionnet: Discretization agnostic learning on surfaces

    Nicholas Sharp, Souhaib Attaiki, Keenan Crane, and Maks Ovsjanikov. Diffusionnet: Discretization agnostic learning on surfaces. TOG, 41 0 (3): 0 1--16, 2022

  56. [65]

    A concise and provably informative multi-scale signature based on heat diffusion

    Jian Sun, Maks Ovsjanikov, and Leonidas Guibas. A concise and provably informative multi-scale signature based on heat diffusion. In Computer graphics forum, volume 28, pp.\ 1383--1392. Wiley Online Library, 2009

  57. [66]

    Emergent correspondence from image diffusion

    Luming Tang, Menglin Jia, Qianqian Wang, Cheng Perng Phoo, and Bharath Hariharan. Emergent correspondence from image diffusion. NeurIPS, 36: 0 1363--1389, 2023

  58. [67]

    Robotap: Tracking arbitrary points for few-shot visual imitation

    Mel Vecerik, Carl Doersch, Yi Yang, Todor Davchev, Yusuf Aytar, Guangyao Zhou, Raia Hadsell, Lourdes Agapito, and Jon Scholz. Robotap: Tracking arbitrary points for few-shot visual imitation. In ICRA, pp.\ 5397--5403. IEEE, 2024

  59. [68]

    One-shot imitation learning: A pose estimation perspective

    Pietro Vitiello, Kamil Dreczkowski, and Edward Johns. One-shot imitation learning: A pose estimation perspective. arXiv preprint arXiv:2310.12077, 2023

  60. [69]

    Mimicplay: Long-horizon imitation learning by watching human play

    Chen Wang, Linxi Fan, Jiankai Sun, Ruohan Zhang, Li Fei-Fei, Danfei Xu, Yuke Zhu, and Anima Anandkumar. Mimicplay: Long-horizon imitation learning by watching human play. arXiv preprint arXiv:2302.12422, 2023 a

  61. [70]

    Dexcap: Scalable and portable mocap data collection system for dexterous manipulation

    Chen Wang, Haochen Shi, Weizhuo Wang, Ruohan Zhang, Li Fei-Fei, and C Karen Liu. Dexcap: Scalable and portable mocap data collection system for dexterous manipulation. arXiv preprint arXiv:2403.07788, 2024

  62. [71]

    Learning and Reasoning with Visual Correspondence in Time

    Xiaolong Wang. Learning and Reasoning with Visual Correspondence in Time. PhD thesis, Carnegie Mellon University, Pittsburgh, PA, September 2019

  63. [72]

    D ^3 fields: Dynamic 3d descriptor fields for zero-shot generalizable robotic manipulation, 2023 b

    Yixuan Wang, Zhuoran Li, Mingtong Zhang, Katherine Driggs-Campbell, Jiajun Wu, Li Fei-Fei, and Yunzhu Li. D ^3 fields: Dynamic 3d descriptor fields for zero-shot generalizable robotic manipulation, 2023 b . URL https://arxiv.org/abs/2309.16118

  64. [73]

    Dynamic graph cnn for learning on point clouds

    Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. Dynamic graph cnn for learning on point clouds. TOG, 38 0 (5): 0 1--12, 2019

  65. [74]

    Omniobject3d: Large-vocabulary 3d object dataset for realistic perception, reconstruction and generation

    Tong Wu, Jiarui Zhang, Xiao Fu, Yuxin Wang, Jiawei Ren, Liang Pan, Wayne Wu, Lei Yang, Jiaqi Wang, Chen Qian, Dahua Lin, and Ziwei Liu. Omniobject3d: Large-vocabulary 3d object dataset for realistic perception, reconstruction and generation. CVPR, pp.\ 803--814, 2023. URL http...

  66. [75]

    Useek: Unsupervised se (3)-equivariant 3d keypoints for generalizable manipulation

    Zhengrong Xue, Zhecheng Yuan, Jiashun Wang, Xueqian Wang, Yang Gao, and Huazhe Xu. Useek: Unsupervised se (3)-equivariant 3d keypoints for generalizable manipulation. In ICRA, pp.\ 1715--1722. IEEE, 2023

  67. [76]

    Sam3d: Segment anything in 3d scenes, 2023

    Yunhan Yang, Xiaoyang Wu, Tong He, Hengshuang Zhao, and Xihui Liu. Sam3d: Segment anything in 3d scenes, 2023. URL https://arxiv.org/abs/2306.03908

  68. [77]

    Learning to manipulate anywhere: A visual generalizable framework for reinforcement learning

    Zhecheng Yuan, Tianming Wei, Shuiqi Cheng, Gu Zhang, Yuanpei Chen, and Huazhe Xu. Learning to manipulate anywhere: A visual generalizable framework for reinforcement learning. arXiv preprint arXiv:2407.15815, 2024

  69. [78]

    3d diffusion policy: Generalizable visuomotor policy learning via simple 3d representations

    Yanjie Ze, Gu Zhang, Kangning Zhang, Chenyuan Hu, Muhan Wang, and Huazhe Xu. 3d diffusion policy: Generalizable visuomotor policy learning via simple 3d representations. In RSS, 2024

  70. [79]

    Corrnet3d: Unsupervised end-to-end learning of dense correspondence for 3d point clouds

    Yiming Zeng, Yue Qian, Zhiyu Zhu, Junhui Hou, Hui Yuan, and Ying He. Corrnet3d: Unsupervised end-to-end learning of dense correspondence for 3d point clouds. In CVPR, pp.\ 6052--6061, 2021

  71. [80]

    A tale of two features: Stable diffusion complements dino for zero-shot semantic correspondence

    Junyi Zhang, Charles Herrmann, Junhwa Hur, Luisa Polania Cabrera, Varun Jampani, Deqing Sun, and Ming-Hsuan Yang. A tale of two features: Stable diffusion complements dino for zero-shot semantic correspondence. In NeurIPS, 2023

  72. [81]

    Telling left from right: Identifying geometry-aware semantic correspondence

    Junyi Zhang, Charles Herrmann, Junhwa Hur, Eric Chen, Varun Jampani, Deqing Sun, and Ming-Hsuan Yang. Telling left from right: Identifying geometry-aware semantic correspondence. In CVPR, pp.\ 3076--3085, June 2024

  73. [82]

    Silvia Zuffi, Angjoo Kanazawa, David Jacobs, and Michael J. Black. 3D menagerie: Modeling the 3D shape and pose of animals. In CVPR, July 2017

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.