Pith. sign in

REVIEW 4 major objections 6 minor 4 cited by

PatchAlign3D claims that a single feed-forward 3D transformer, trained by distilling 2D visual features into patch tokens and aligning them with text, can outperform multi-view rendering pipelines at zero-shot 3D part segmentation.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 06:24 UTC pith:VF6AR3IN

load-bearing objection A well-ablated and honest two-stage recipe for text-aligned 3D patch features; the benchmark numbers are strong, but the open-world claim rests on a weak 14-category unseen split. the 4 major comments →

arxiv 2601.02457 v2 pith:VF6AR3IN submitted 2026-01-05 cs.CV

PatchAlign3D: Local Feature Alignment for Dense 3D Shape Understanding

Souhail Hadgi , Bingchen Gong , Ramana Sundararaman , Emery Pierson , Lei Li , Peter Wonka , Maks Ovsjanikov This is my paper
classification cs.CV
keywords 3D part segmentationzero-shot learningpoint cloud transformer2D-to-3D distillationcontrastive learningpatch-level featuresopen-vocabulary segmentationfoundation models
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

PatchAlign3D argues that a point cloud transformer can be turned into a language-aligned local feature extractor through two stages of patch-level pre-training, and that this encoder alone matches or beats multi-view rendering pipelines for zero-shot 3D part segmentation. On ShapeNetPart it reports 56.9 mIoU, a gain of more than 31 points over the strongest rendering-based approach and more than 33 over the best prior feed-forward baseline trained on the same data, with consistent gains on four other benchmarks. The practical stake is that dense 3D understanding no longer needs test-time rendering, LLM prompt engineering, or per-view feature fusion; a single forward pass over point coordinates suffices. The paper's method deliberately operates on 128 patches of 32 points each, arguing that patch-level aggregation absorbs the noise in automatically generated part annotations.

Core claim

The central claim is that language-aligned local features for 3D shapes can be learned entirely in the encoder, without any decoding or rendering at inference. The paper proposes a two-stage pre-training: first regress each 3D patch token toward back-projected dense features from a pre-trained 2D visual encoder, then freeze most of the transformer and align only the last block's patch embeddings to part-name text embeddings with a multi-positive sigmoid contrastive loss. Fractional labels encode the degree to which a patch overlaps each part, making the objective robust to fragmented and inconsistent annotations. At test time, each patch is labeled by maximum similarity to the query text and

What carries the argument

The carrying mechanism is a point cloud transformer operating on tokens: farthest-point sampling selects 128 patch centers, each patch of 32 points is encoded by a lightweight PointNet, and a 12-layer transformer produces patch tokens. Stage 1 uses a cosine-similarity regression against cached, multi-view-averaged features from a dense 2D visual encoder, transferring visual priors into the 3D encoder. Stage 2 applies a sigmoid binary cross-entropy loss with fractional multi-positive labels and treats negatives only within each shape, which prevents identical parts in different shapes from being pushed apart. The key design choice is decoupling the two objectives in sequence rather than joint

Load-bearing premise

The load-bearing premise is that the automatically generated part annotations—masks paired with single-word vision-language captions back-projected onto the point cloud—are noisy but not systematically biased, so that patch-level aggregation with fractional multi-positive labels converges to a semantically correct local feature space; the paper supports this only with a self-reported ablation on the same noisy labels.

What would settle it

Systematically mislabel a controlled fraction of the pseudo-annotations in a geometry-correlated way (for example, rename half of all 'wheel' captions to 'tire' or shift part captions by one category) and re-run the two-stage training; if ShapeNetPart zero-shot mIoU drops proportionally to the injected noise, the method is memorizing the label distribution rather than learning geometry-backed part semantics.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the claim holds, open-vocabulary 3D part segmentation no longer requires multiple rendered views; any point cloud can be labeled from text in a single fast pass, making the approach suitable for robotic and interactive settings.
  • Rendering-based dense pipelines lose their accuracy advantage on these benchmarks, shifting the default for 3D local understanding toward geometry-native encoders and away from appearance-based 2D transfer.
  • Because the model is trained on automatically generated pseudo-part labels and still transfers to clean benchmarks, it suggests that noisy large-scale 3D data engines can substitute for costly manual part annotation in pre-training.
  • The same patch features support zero-shot and few-shot keypoint detection in the paper's supplementary experiments, indicating the representation is not segmentation-specific.
  • Stage 2's compatibility with a purely text-based encoder suggests the geometry-to-language alignment is robust to the choice of text tower.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the pseudo-label distribution is systematically biased toward coarse or view-dependent part names, the reported margins could shrink on datasets whose part vocabularies are outside the training distribution; the paper's ablation does not yet audit per-category label noise.
  • Patch size fixes a resolution ceiling: 128 patches of 32 points may blur thin or small parts. An adaptive or hierarchical partitioning, flagged as future work by the paper, would be a natural test of whether the gains persist at finer granularity.
  • One can test the method's inductive bias by applying it to text queries that name materials, functions, or abstract properties rather than parts; the contrastive objective should respond, suggesting a route from part segmentation to open-vocabulary 3D grounding.
  • The single-pass speed opens a practical extension to interactive segmentation where a user names a part and sees the region update in real time; the paper does not build this interface but its latency enables it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes PatchAlign3D, a feed-forward point-cloud transformer that learns language-aligned patch-level features through two-stage pre-training. Stage 1 distills dense 2D features (DINOv2) into 3D patch tokens via a cosine-similarity regression loss; Stage 2 freezes all but the last transformer block and aligns patch tokens with text embeddings using a multi-positive sigmoid contrastive loss with fractional labels. Training data comes from the Find3D data engine: 32K Objaverse shapes annotated by back-projecting SAM masks and Gemini captions. At inference, patch--text similarities are computed with a fixed template and upsampled to points. The paper reports state-of-the-art zero-shot segmentation on ShapeNetPart, PartNetE, ScanObjectNN, FAUST, and Objaverse-General, with a single feed-forward pass, and provides ablations showing that Stage 2 alone already outperforms prior baselines.

Significance. If the results hold, the paper makes a strong practical contribution: it suggests that an encoder-only 3D model can beat multi-view rendering pipelines on dense part segmentation while being far faster. The patch-level contrastive formulation with fractional labels is a clean and well-motivated way to handle noisy pseudo-labels. The paper also deserves credit for honest ablations (e.g., Table 6 showing Stage-2-only performance, Table 9 on freezing), for using a unified prompt template to reduce prompt-engineering bias, and for reporting inference speed. However, the central claim of open-world zero-shot generalization is not yet convincingly established because the only strict category-holdout evaluation is small, unreleased, and shows a marginal advantage over the strongest feed-forward baseline.

major comments (4)
  1. [§4.1, Table 4] The only evaluation against truly unseen categories is the author-defined Objaverse–General split with 14 unseen categories. The margin over Find3D on unseen categories is 35.61 vs 34.6 mIoU — about 1 point — whereas the ShapeNetPart margin is +31.3 mIoU (Table 1). Since ShapeNetPart, PartNetE, FAUST, and ScanObjectNN are composed of common object classes that plausibly fall within the 761-class Objaverse training set, the large reported gains may reflect category-level familiarity rather than generalizable local 3D–text alignment. The split is not released and is too small to support strong conclusions. Please release the split, provide a larger strictly held-out category benchmark (or a detailed overlap analysis and per-category results on the existing benchmarks), and report confidence intervals for the unseen-category numbers.
  2. [§4.2, Table 3] The PartNetE protocol is ambiguous and possibly unfair. The text states: “we assign a ”body“ label to unlabeled points since our approach relies on patch-text similarity,” while the baselines are “evaluated using part labels only.” If unlabeled points are excluded from the baselines’ metric but assigned to “body” for PatchAlign3D, the comparison is not like-for-like and could inflate the reported mIoU. Please specify exactly how unlabeled points are treated for every method, and either exclude them consistently or introduce a background class for all methods.
  3. [All tables] No error bars or statistical significance tests are reported. This is especially important for the Objaverse–General unseen split, where the 1.0 mIoU difference could be within noise. Please provide standard deviations over at least 3 seeds (or bootstrapped confidence intervals) for all benchmarks. For the small 14-category split, also report per-category numbers.
  4. [§3.2, §3.3, Table 6] The claim that patch-level aggregation mitigates noisy pseudo-labels is supported mainly by the Stage-2-only ablation (Table 6), which is measured on the same noisy training pipeline, and by external benchmarks that may suffer from category leakage. A direct label-noise audit is missing: e.g., human evaluation of a subset of Find3D pseudo-annotations, fragmentation statistics, or agreement rates with ShapeNetPart/PartNetE labels. Without such evidence, it is hard to rule out that the method is memorizing annotation biases rather than learning semantically robust local features.
minor comments (6)
  1. [§1, Abstract] The phrase “significantly outperforming” is used without significance testing. Consider phrasing as “outperforms” or adding statistical support.
  2. [§1, Contributions] The claim of “the first 3D encoder that produces language-aligned, patch-level features” is slightly overstated given Find3D is also feed-forward (though point-level). Clarify that the novelty is the patch-level two-stage distillation and fractional-label contrastive objective.
  3. [Table 1] The “Improvement” row does not state the baseline (apparently COPS). Also, the motorcycle category shows a negative improvement; the text says “consistent gains across 15 of the 16,” which is accurate but the row could be misinterpreted. Specify the baseline in the caption or the row header.
  4. [§4.1] The FAUST coarse annotations are attributed to SATR [1]; its companion reference appears in the text but the exact split name could be made more explicit. Also note that Table 5 lacks hardware details; please specify GPU/model and batch size for a reproducible runtime comparison.
  5. [§3.3, Eq. (5)] The negative set is defined as within-sample entries with y_{i,j}=0. It would help to clarify whether all part names appearing in the shape are used as negatives, or only a subset, and how this interacts with the fact that some true parts may be absent from a given shape's annotation set.
  6. [Supplementary, Table 10] The keypoint detection results are interesting but not connected to the main narrative; a sentence in the main text referring to the supplementary would help readers find this additional validation.

Circularity Check

0 steps flagged

No significant circularity: external benchmarks and frozen text tower make the core claim independent of its training inputs.

full rationale

PatchAlign3D's derivation chain is self-contained and non-circular. Stage 1 transfers fixed DINOv2 multi-view features into 3D patch tokens through a cosine-similarity regression loss (Eq. 3); Stage 2 aligns the Stage-1-initialized encoder's patch tokens with fixed OpenCLIP text embeddings using a sigmoid contrastive loss (Eq. 5) on Find3D's SAM+Gemini pseudo-labels. No test-set constants are fitted: ShapeNetPart, PartNetE, ScanObjectNN, and FAUST are external human-annotated benchmarks, and the CLIP text tower is pre-trained and frozen at inference. The pseudo-label supervision is explicitly acknowledged as noisy ('inherently noisy and inconsistent across parts'), and the method's advantage over Find3D is an architectural/loss claim evaluated against external ground truth, not a relabeling of training targets. The paper's own limitation statement — 'pre-trained on a curated Objaverse subset with imperfect pseudo-part annotations derived from SAM and a language model' — confirms that the semantic supervision is declared distillation rather than hidden circularity. The author-defined Objaverse-General split (Table 4) is not released and the large ShapeNetPart gains may benefit from category overlap with the 761-class training set, but this is a generalization/auditability concern, not a by-construction equivalence. Minor self-citations (e.g., Zerokey [13], SATR [1]) appear in related work or the keypoint evaluation protocol and are not load-bearing for the central claims.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

This paper introduces no invented entities in the physics sense. Its load-bearing inputs are: hand-set architectural constants (patch count 128, patch size 32, 10 render views) that define the spatial quantization of the entire pipeline; learnable sigmoid temperature/bias with hand-set initializations; a freezing depth selected by ablation on the evaluation benchmark; and three domain assumptions about upstream models — DINOv2 back-projected features are coherent targets, CLIP text embeddings form a usable part-name space, and the Find3D SAM+Gemini pseudo-labels are noisy-but-benign. The method computes nothing from first principles; it is a distillation-and-alignment recipe whose semantic content is inherited from 2D foundation models.

free parameters (5)
  • Patch count G = 128
    Number of patch tokens per shape (§3.2); hand-set; defines feature granularity and the spatial scale of label propagation at inference.
  • Patch size k = 32
    Points per patch (§3.2); hand-set; jointly with G determines how coarse part boundaries can be.
  • Render view count = 10
    Views used for Stage-1 DINOv2 distillation and target caching (§3.2); inherited from the Find3D/COPS data pipeline; controls back-projection coverage in Eq. (1).
  • Sigmoid temperature/bias initialization = τ=0.1, b=−10 (learnable)
    Learnable parameters in Eq. (4) initialized by hand following SigLIP; final trained values are not reported.
  • Stage-2 freezing depth = freeze all but last transformer block + head
    Selected by ablation (Table 9) on the evaluation protocol; a model-selection choice coupled to the test benchmark.
axioms (5)
  • domain assumption DINOv2 dense features back-projected via Eq. (1) are per-point coherent semantic targets suitable for cosine regression
    Stage-1 supervision quality rests on multi-view feature agreement and on nearest-neighbor interpolation being harmless for points invisible in all views (§3.2).
  • domain assumption OpenCLIP ViT-bigG-14 text embeddings provide a discriminative shared space for simple part names under the unified template
    Stage 2 and zero-shot inference assume part names like 'wing' map to separable CLIP text vectors; the Gemma-2-9B-it ablation (Table 8) indicates the text tower itself is not the limiting factor.
  • domain assumption Find3D's SAM + Gemini pseudo-annotations are noisy but not systematically wrong, so patch-level fractional labels converge to a useful signal
    The entire training signal comes from this pipeline (§3.2); the paper acknowledges imperfect pseudo-annotations (Limitations) but provides no noise-rate audit.
  • domain assumption FPS + kNN patch decomposition and 1-NN center matching align online patches with cached features during Stage 1
    Stage-1 correspondence assumes cached centers and online centers match through nearest-centroid lookup (§3.3); mismatch would inject target noise.
  • standard math Standard transformer, cosine-similarity, and sigmoid binary cross-entropy machinery
    Eqs. (3)–(5) use standard definitions; no novel mathematics is claimed.

pith-pipeline@v1.3.0-alltime-deepseek · 17710 in / 20760 out tokens · 199350 ms · 2026-08-04T06:24:43.579771+00:00 · methodology

0 comments
read the original abstract

Current foundation models for 3D shapes excel at global tasks (retrieval, classification) but transfer poorly to local part-level reasoning. Recent approaches leverage vision and language foundation models to directly solve dense tasks through multi-view renderings and text queries. While promising, these pipelines require expensive inference over multiple renderings, depend heavily on large language-model (LLM) prompt engineering for captions, and fail to exploit the inherent 3D geometry of shapes. We address this gap by introducing an encoder-only 3D model that produces language-aligned patch-level features directly from point clouds. Our pre-training approach builds on existing data engines that generate part-annotated 3D shapes by pairing multi-view SAM regions with VLM captioning. Using this data, we train a point cloud transformer encoder in two stages: (1) distillation of dense 2D features from visual encoders such as DINOv2 into 3D patches, and (2) alignment of these patch embeddings with part-level text embeddings through a multi-positive contrastive objective. Our 3D encoder achieves zero-shot 3D part segmentation with fast single-pass inference without any test-time multi-view rendering, while significantly outperforming previous rendering-based and feed-forward approaches across several 3D part segmentation benchmarks. Project website: https://souhail-hadgi.github.io/patchalign3dsite/

Figures

Figures reproduced from arXiv: 2601.02457 by Bingchen Gong, Emery Pierson, Lei Li, Maks Ovsjanikov, Peter Wonka, Ramana Sundararaman, Souhail Hadgi.

Figure 1
Figure 1. Figure 1: PatchAlign3D is a point cloud transformer encoder that produces language-aligned patch-level features. Through two-stage training, [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: PatchAlign3D pre-training. Given an input point cloud, we extract multi-view visual features using a 2D backbone and back￾project them into 3D space. In Stage 1, the 3D transformer encoder operates on sampled point cloud patches and learns to align its output patch tokens with the back-projected visual features. In Stage 2, we initialize from Stage 1, freeze all earlier layers, and train only the last tran… view at source ↗
Figure 3
Figure 3. Figure 3: Qualitative comparisons on ShapeNetPart [51]. We show ground truth (top row) and predictions from COPS [12], Find3D [23], and PatchAlign3D (rows 2–4) across six representative shapes. The part legends below each column indicate the semantic labels used for zero-shot prediction. PatchAlign3D produces noticeably more precise and coherent segmentations, despite relying solely on an encoder and patch-level fea… view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative comparison on non-rigid human shapes from FAUST [1, 5]. We show ground truth and predictions from COPS [12], Find3D [23], and PatchAlign3D across three repre￾sentative shapes (columns). The part legend below specifies the semantic labels used for zero-shot prediction. PatchAlign3D pro￾duces cleaner segmentations than prior methods and is less noisy than Find3D’s encoder–decoder outputs. gories … view at source ↗
Figure 5
Figure 5. Figure 5: Feature comparison across stages. We visualize features from DINOv2, Stage 1, and Stage 1 + Stage 2 of our approach on example point clouds from the validation split of the training data. Stage 1 refines DINOv2 features, and Stage 2 further preserves them while assigning downstream text capabilities. ficiently effective to learn semantically meaningful local features even without dense 2D supervision. Neve… view at source ↗
Figure 7
Figure 7. Figure 7: Anchor-based feature similarity. For a selected anchor point or patch on a shape (e.g., wing, body, motor), we visualize the similarity of all other points/patches to the anchor. PatchAlign3D shows stronger geometric coherence than DINOv2, Stage 1, and Find3D. separation between fine-grained regions. PatchAlign3D pro￾duces the cleanest and most discriminative part clusters, demonstrating that Stage 2 refin… view at source ↗
Figure 6
Figure 6. Figure 6: Text-to-feature similarity visualization. We compare PatchAlign3D to Find3D by visualizing similarities between a tex￾tual query (e.g., “ear”, “nose”) and the dense features on a valida￾tion point cloud. Yellow indicates higher similarity. PatchAlign3D produces sharper and more localized responses, while Find3D often shows diffuse signals with weaker semantic localization. noisy and lack spatial precision,… view at source ↗
Figure 8
Figure 8. Figure 8: Visualization of keypoints detected in zero-shot key￾point detection. In these experiments, the input to our method is a point cloud containing 2048 points. The detected keypoints given a text prompt are highlighted as larger green dots. ically designed for this dataset, such as B2-3D [45] and FSKD [3]. These results emphasize that our feature improved point-level understanding of both text semantics and g… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. TORA: Topological Representation Alignment for 3D Shape Assembly

    cs.CV 2026-04 unverdicted novelty 7.0

    TORA distills topological structure from pretrained 3D encoders into flow-matching backbones via cosine matching and CKA loss, delivering up to 6.9x faster convergence and better accuracy on 3D shape assembly benchmar...

  2. MV-GEL: Language-Driven Multi-View Geometric Entity Localization on Meshes

    cs.CV 2026-06 unverdicted novelty 6.0

    MV-GEL localizes fine-grained geometric entities on 3D meshes from natural language by ranking informative views with GELviews, applying VLM segmentation, and lifting masks via geometry-aware ray casting, reporting up...

  3. MV-GEL: Language-Driven Multi-View Geometric Entity Localization on Meshes

    cs.CV 2026-06 conditional novelty 6.0

    MV-GEL uses a learned view selector and a fine-tuned vision-language segmentation model to localize text-described faces and edges on 3D meshes.

  4. TORA: Topological Representation Alignment for 3D Shape Assembly

    cs.CV 2026-04 unverdicted novelty 5.0

    Aligning flow-matching assembly networks to frozen 3D encoder topology via cosine and CKA losses speeds training up to 6.9× and improves in- and out-of-distribution assembly accuracy with zero inference cost.

Reference graph

Works this paper leans on

62 extracted references · 18 linked inside Pith · cited by 2 Pith papers

  1. [1]

    Satr: Zero-shot semantic segmentation of 3d shapes

    Ahmed Abdelreheem, Ivan Skorokhodov, Maks Ovsjanikov, and Peter Wonka. Satr: Zero-shot semantic segmentation of 3d shapes. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 15166–15179, 2023. 2, 5, 6, 7

    2023

  2. [2]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023. 13, 14

    Pith/arXiv arXiv 2023

  3. [3]

    Ncp: Neural cor- respondence prior for effective unsupervised shape match- ing.Advances in Neural Information Processing Systems, 35: 28842–28857, 2022

    Souhaib Attaiki and Maks Ovsjanikov. Ncp: Neural cor- respondence prior for effective unsupervised shape match- ing.Advances in Neural Information Processing Systems, 35: 28842–28857, 2022. 14

    2022

  4. [4]

    D3feat: Joint learning of dense detec- tion and description of 3d local features

    Xuyang Bai, Zixin Luo, Lei Zhou, Hongbo Fu, Long Quan, and Chiew-Lan Tai. D3feat: Joint learning of dense detec- tion and description of 3d local features. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6359–6367, 2020. 14

    2020

  5. [5]

    Faust: Dataset and evaluation for 3d mesh registration

    Federica Bogo, Javier Romero, Matthew Loper, and Michael J Black. Faust: Dataset and evaluation for 3d mesh registration. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 3794–3801, 2014. 5, 6, 7

    2014

  6. [6]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. InPro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 9650–9660, 2021. 12

    2021

  7. [7]

    Yin, Matthew Fisher, Siddhartha Chaudhuri, and Hao Zhang

    Zhiqin Chen, K. Yin, Matthew Fisher, Siddhartha Chaudhuri, and Hao Zhang. Bae-net: Branched autoencoder for shape co-segmentation.2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 8489–8498, 2019. 2

    2019

  8. [8]

    Chang, Manolis Savva, Maciej Hal- ber, Thomas A

    Angela Dai, Angel X. Chang, Manolis Savva, Maciej Hal- ber, Thomas A. Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes.2017 IEEE Conference on Computer Vision and Pattern Recogni- tion (CVPR), pages 2432–2443, 2017. 2

    2017

  9. [9]

    3d highlighter: Localizing regions on 3d shapes via text descriptions

    Dale Decatur, Itai Lang, and Rana Hanocka. 3d highlighter: Localizing regions on 3d shapes via text descriptions. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20930–20939, 2023. 6, 7

    2023

  10. [10]

    Objaverse: A universe of annotated 3d objects

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13142–13153, 2023. 3, 5

    2023

  11. [11]

    An image is worth 16x16 words: Trans- formers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020

    Alexey Dosovitskiy. An image is worth 16x16 words: Trans- formers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020. 12

    Pith/arXiv arXiv 2010

  12. [12]

    3d part segmenta- tion via geometric aggregation of 2d visual features

    Marco Garosi, Riccardo Tedoldi, Davide Boscaini, Massimil- iano Mancini, Nicu Sebe, and Fabio Poiesi. 3d part segmenta- tion via geometric aggregation of 2d visual features. In2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 3257–3267. IEEE, 2025. 2, 4, 5, 6, 7

    2025

  13. [13]

    Zerokey: Point-level reasoning and zero- shot 3d keypoint detection from large language models

    Bingchen Gong, Diego Gomez, Abdullah Hamdi, Abdelrah- man Eldesokey, Ahmed Abdelreheem, Peter Wonka, and Maks Ovsjanikov. Zerokey: Point-level reasoning and zero- shot 3d keypoint detection from large language models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 22089–22099, 2025. 14

    2025

  14. [14]

    Unim-ov3d: Uni-modality open-vocabulary 3d scene understanding with fine-grained feature representation.arXiv preprint arXiv:2401.11395,

    Qingdong He, Jinlong Peng, Zhengkai Jiang, Kai Wu, Xi- aozhong Ji, Jiangning Zhang, Yabiao Wang, Chengjie Wang, Mingang Chen, and Yunsheng Wu. Unim-ov3d: Uni-modality open-vocabulary 3d scene understanding with fine-grained feature representation.arXiv preprint arXiv:2401.11395,

    Pith/arXiv arXiv

  15. [15]

    Learning and memorizing representative prototypes for 3d point cloud semantic and instance segmentation.ArXiv, abs/2001.01349,

    Tong He, Dong Gong, Zhi Tian, and Chunhua Shen. Learning and memorizing representative prototypes for 3d point cloud semantic and instance segmentation.ArXiv, abs/2001.01349,

    Pith/arXiv arXiv 2001

  16. [16]

    Open-vocabulary 3d semantic segmentation with foundation models

    Li Jiang, Shaoshuai Shi, and Bernt Schiele. Open-vocabulary 3d semantic segmentation with foundation models. In2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 21284–21294, 2024. 2

    2024

  17. [17]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. InProceedings of the IEEE/CVF international confer- ence on computer vision, pages 4015–4026, 2023. 2, 3

    2023

  18. [18]

    Usip: Unsupervised stable interest point detection from 3d point clouds

    Jiaxin Li and Gim Hee Lee. Usip: Unsupervised stable interest point detection from 3d point clouds. InProceedings of the IEEE/CVF international conference on computer vision, pages 361–370, 2019. 14

    2019

  19. [19]

    Grounded language-image pre-training.2022 IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR), pages 10955–10965, 2021

    Liunian Harold Li, Pengchuan Zhang, Haotian Zhang, Jianwei Yang, Chunyuan Li, Yiwu Zhong, Lijuan Wang, Lu Yuan, Lei Zhang, Jenq-Neng Hwang, Kai-Wei Chang, and Jianfeng Gao. Grounded language-image pre-training.2022 IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR), pages 10955–10965, 2021. 2

    2022

  20. [20]

    Openshape: Scaling up 3d shape representation towards open-world understanding.Advances in neural information processing systems, 36:44860–44879, 2023

    Minghua Liu, Ruoxi Shi, Kaiming Kuang, Yinhao Zhu, Xu- anlin Li, Shizhong Han, Hong Cai, Fatih Porikli, and Hao Su. Openshape: Scaling up 3d shape representation towards open-world understanding.Advances in neural information processing systems, 36:44860–44879, 2023. 1, 3, 4, 6

    2023

  21. [21]

    Partslip: Low-shot part seg- mentation for 3d point clouds via pretrained image-language models

    Minghua Liu, Yinhao Zhu, Hong Cai, Shizhong Han, Zhan Ling, Fatih Porikli, and Hao Su. Partslip: Low-shot part seg- mentation for 3d point clouds via pretrained image-language models. InProceedings of the IEEE/CVF conference on com- puter vision and pattern recognition, pages 21736–21746,

  22. [22]

    Partfield: Learning 3d feature fields for part segmentation and beyond

    Minghua Liu, Mikaela Angelina Uy, Donglai Xiang, Hao Su, Sanja Fidler, Nicholas Sharp, and Jun Gao. Partfield: Learning 3d feature fields for part segmentation and beyond. ArXiv, abs/2504.11451, 2025. 2 9

    Pith/arXiv arXiv 2025

  23. [23]

    Find any part in 3d

    Ziqi Ma, Yisong Yue, and Georgia Gkioxari. Find any part in 3d. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 7818–7827, 2025. 1, 2, 3, 5, 6, 7, 12, 13

    2025

  24. [24]

    Partnet: A large- scale benchmark for fine-grained and hierarchical part-level 3d object understanding

    Kaichun Mo, Shilin Zhu, Angel X Chang, Li Yi, Subarna Tripathi, Leonidas J Guibas, and Hao Su. Partnet: A large- scale benchmark for fine-grained and hierarchical part-level 3d object understanding. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 909–918, 2019. 2

    2019

  25. [25]

    Ngo, Chuang Gan, Evangelos Kaloger- akis, Anh Dat Tran, Cuong Pham, and Khoi Nguyen

    Phuc Nguyen, T.D. Ngo, Chuang Gan, Evangelos Kaloger- akis, Anh Dat Tran, Cuong Pham, and Khoi Nguyen. Open3dis: Open-vocabulary 3d instance segmentation with 2d mask guidance.2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4018–4028,

    2024

  26. [26]

    Lam, Yan-Pei Cao, and Xihui Liu

    Yu nuo Yang, Yukun Huang, Yuanchen Guo, Liangjun Lu, Xiaoyang Wu, Edmund Y . Lam, Yan-Pei Cao, and Xihui Liu. Sampart3d: Segment any part in 3d objects.ArXiv, abs/2411.07184, 2024. 2

    Pith/arXiv arXiv 2024

  27. [27]

    Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193, 2023

    Maxime Oquab, Timoth´ee Darcet, Th´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193, 2023. 2, 4, 6, 12, 13

    Pith/arXiv arXiv 2023

  28. [28]

    Pytorch: An imperative style, high-performance deep learning library.Ad- vances in neural information processing systems, 32, 2019

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library.Ad- vances in neural information processing systems, 32, 2019. 7

    2019

  29. [29]

    Funkhouser

    Songyou Peng, Kyle Genova, ChiyuMaxJiang, Andrea Tagliasacchi, Marc Pollefeys, and Thomas A. Funkhouser. Openscene: 3d scene understanding with open vocabularies. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 815–824, 2022. 2

    2023

  30. [30]

    Qi, Hao Su, Kaichun Mo, and Leonidas J

    C. Qi, Hao Su, Kaichun Mo, and Leonidas J. Guibas. Pointnet: Deep learning on point sets for 3d classification and segmenta- tion.2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 77–85, 2016. 2

    2017

  31. [31]

    C. Qi, L. Yi, Hao Su, and Leonidas J. Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space.ArXiv, abs/1706.02413, 2017. 2

    Pith/arXiv arXiv 2017

  32. [32]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. InInternational conference on machine learning, pages 8748–8763. PmLR, 2021. 2, 6, 12

    2021

  33. [33]

    Sam 2: Segment any- thing in images and videos.arXiv preprint arXiv:2408.00714,

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman R¨adle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment any- thing in images and videos.arXiv preprint arXiv:2408.00714,

    Pith/arXiv arXiv

  34. [34]

    V ol- umetric image registration from invariant keypoints.IEEE Transactions on Image Processing, 26(10):4900–4910, 2017

    Blaine Rister, Mark A Horowitz, and Daniel L Rubin. V ol- umetric image registration from invariant keypoints.IEEE Transactions on Image Processing, 26(10):4900–4910, 2017. 14

    2017

  35. [35]

    What does clip know about a red circle? vi- sual prompt engineering for vlms

    Aleksandar Shtedritski, Christian Rupprecht, and Andrea Vedaldi. What does clip know about a red circle? vi- sual prompt engineering for vlms. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 11987–11997, 2023. 14

    2023

  36. [36]

    Di- nov3.arXiv preprint arXiv:2508.10104, 2025

    Oriane Sim ´eoni, Huy V V o, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Marc Szafraniec, Seungeun Yi, Micha¨el Ramamonjisoa, et al. Di- nov3.arXiv preprint arXiv:2508.10104, 2025. 12

    Pith/arXiv arXiv 2025

  37. [37]

    Harris 3d: a robust exten- sion of the harris operator for interest point detection on 3d meshes.The Visual Computer, 27:963–976, 2011

    Ivan Sipiran and Benjamin Bustos. Harris 3d: a robust exten- sion of the harris operator for interest point detection on 3d meshes.The Visual Computer, 27:963–976, 2011. 14

    2011

  38. [38]

    Paligemma 2: A family of versatile vlms for transfer.arXiv preprint arXiv:2412.03555, 2024

    Andreas Steiner, Andr´e Susano Pinto, Michael Tschannen, Daniel Keysers, Xiao Wang, Yonatan Bitton, Alexey Grit- senko, Matthias Minderer, Anthony Sherbondy, Shangbang Long, et al. Paligemma 2: A family of versatile vlms for transfer.arXiv preprint arXiv:2412.03555, 2024. 13, 14

    Pith/arXiv arXiv 2024

  39. [39]

    Sumner, Marc Pollefeys, Federico Tombari, and Francis Engelmann

    Ayca Takmaz, Elisabetta Fedele, Robert W. Sumner, Marc Pollefeys, Federico Tombari, and Francis Engelmann. Open- mask3d: Open-vocabulary 3d instance segmentation.ArXiv, abs/2306.13631, 2023. 2

    Pith/arXiv arXiv 2023

  40. [40]

    Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, An- drew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023. 2, 3

    Pith/arXiv arXiv 2023

  41. [41]

    Gemma 2: Improving open language models at a practical size.arXiv preprint arXiv:2408.00118,

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupati- raju, L´eonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ram´e, et al. Gemma 2: Improving open language models at a practical size.arXiv preprint arXiv:2408.00118,

    Pith/arXiv arXiv

  42. [42]

    Partdistill: 3d shape part segmenta- tion by vision-language model distillation.2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3470–3479, 2023

    Ardian Umam, Cheng-Kun Yang, Min-Hung Chen, Jen-Hui Chuang, and Yen-Yu Lin. Partdistill: 3d shape part segmenta- tion by vision-language model distillation.2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3470–3479, 2023. 3

    2024

  43. [43]

    Revisiting point cloud classification: A new benchmark dataset and classification model on real-world data

    Mikaela Angelina Uy, Quang-Hieu Pham, Binh-Son Hua, Thanh Nguyen, and Sai-Kit Yeung. Revisiting point cloud classification: A new benchmark dataset and classification model on real-world data. InProceedings of the IEEE/CVF international conference on computer vision, pages 1588– 1597, 2019. 2, 5, 7

    2019

  44. [44]

    Attention is all you need.Advances in neural information processing systems, 30, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural information processing systems, 30, 2017. 4

    2017

  45. [45]

    Back to 3d: Few-shot 3d keypoint detection with back-projected 2d features

    Thomas Wimmer, Peter Wonka, and Maks Ovsjanikov. Back to 3d: Few-shot 3d keypoint detection with back-projected 2d features. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4154–4164,

  46. [46]

    Point transformer v3: Simpler faster stronger

    Xiaoyang Wu, Li Jiang, Peng-Shuai Wang, Zhijian Liu, Xihui Liu, Yu Qiao, Wanli Ouyang, Tong He, and Hengshuang Zhao. Point transformer v3: Simpler faster stronger. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4840–4851, 2024. 2 10

    2024

  47. [47]

    Clip-dinoiser: Teaching clip a few dino tricks for open- vocabulary semantic segmentation

    Monika Wysocza´nska, Oriane Sim´eoni, Micha¨el Ramamon- jisoa, Andrei Bursuc, Tomasz Trzci´nski, and Patrick P ´erez. Clip-dinoiser: Teaching clip a few dino tricks for open- vocabulary semantic segmentation. InEuropean Conference on Computer Vision, pages 320–337. Springer, 2024. 14

    2024

  48. [48]

    Ulip-2: To- wards scalable multimodal pre-training for 3d understanding

    L Xue, N Yu, S Zhang, A Panagopoulou, J Li, R Mart ´ın- Mart´ın, J Wu, C Xiong, R Xu, and JC Niebles. Ulip-2: To- wards scalable multimodal pre-training for 3d understanding. arxiv.arXiv preprint arXiv:2305.08275, 2023. 1, 2, 3, 4, 14

    Pith/arXiv arXiv 2023

  49. [49]

    Ze- rops: High-quality cross-modal knowledge transfer for zero- shot 3d part segmentation.2025 International Conference on 3D Vision (3DV), pages 1328–1339, 2023

    Yuheng Xue, Nenglun Chen, Jun Liu, and Wenyun Sun. Ze- rops: High-quality cross-modal knowledge transfer for zero- shot 3d part segmentation.2025 International Conference on 3D Vision (3DV), pages 1328–1339, 2023. 2

    2025

  50. [50]

    Scannet++: A high-fidelity dataset of 3d in- door scenes.2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 12–22, 2023

    Chandan Yeshwanth, Yueh-Cheng Liu, Matthias Nießner, and Angela Dai. Scannet++: A high-fidelity dataset of 3d in- door scenes.2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 12–22, 2023. 2

    2023

  51. [51]

    A scalable active framework for region annotation in 3d shape collections.ACM Transactions on Graphics (ToG), 35(6):1–12, 2016

    Li Yi, Vladimir G Kim, Duygu Ceylan, I-Chao Shen, Mengyan Yan, Hao Su, Cewu Lu, Qixing Huang, Alla Sheffer, and Leonidas Guibas. A scalable active framework for region annotation in 3d shape collections.ACM Transactions on Graphics (ToG), 35(6):1–12, 2016. 2, 5, 6

    2016

  52. [52]

    Yi, Wang Zhao, He Wang, Minhyuk Sung, and Leonidas J

    L. Yi, Wang Zhao, He Wang, Minhyuk Sung, and Leonidas J. Guibas. Gspn: Generative shape proposal network for 3d instance segmentation in point cloud.2019 IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR), pages 3942–3951, 2018. 2

    2019

  53. [53]

    Keypointnet: A large-scale 3d keypoint dataset aggregated from numerous human annotations

    Yang You, Yujing Lou, Chengkun Li, Zhoujun Cheng, Liangwei Li, Lizhuang Ma, Cewu Lu, and Weiming Wang. Keypointnet: A large-scale 3d keypoint dataset aggregated from numerous human annotations. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13647–13656, 2020. 14

    2020

  54. [54]

    Ukpgan: A general self-supervised key- point detector

    Yang You, Wenhai Liu, Yanjie Ze, Yong-Lu Li, Weiming Wang, and Cewu Lu. Ukpgan: A general self-supervised key- point detector. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022. 14

    2022

  55. [55]

    Point-bert: Pre-training 3d point cloud transformers with masked point modeling

    Xumin Yu, Lulu Tang, Yongming Rao, Tiejun Huang, Jie Zhou, and Jiwen Lu. Point-bert: Pre-training 3d point cloud transformers with masked point modeling. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 19313–19322, 2022. 3, 4

    2022

  56. [56]

    Dino in the room: Leveraging 2d foundation models for 3d segmentation.arXiv preprint arXiv:2503.18944, 2025

    Karim Abou Zeid, Kadir Yilmaz, Daan de Geus, Alexander Hermans, David Adrian, Timm Linder, and Bastian Leibe. Dino in the room: Leveraging 2d foundation models for 3d segmentation.arXiv preprint arXiv:2503.18944, 2025. 3

    arXiv 2025

  57. [57]

    Sigmoid loss for language image pre-training

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. InProceedings of the IEEE/CVF international conference on computer vision, pages 11975–11986, 2023. 4, 12

    2023

  58. [58]

    Intrinsic shape signatures: A shape descriptor for 3d object recognition

    Yu Zhong. Intrinsic shape signatures: A shape descriptor for 3d object recognition. In2009 IEEE 12th international conference on computer vision workshops, ICCV Workshops, pages 689–696. IEEE, 2009. 14

    2009

  59. [59]

    Uni3d: Exploring unified 3d representation at scale.arXiv preprint arXiv:2310.06773,

    Junsheng Zhou, Jinsheng Wang, Baorui Ma, Yu-Shen Liu, Tiejun Huang, and Xinlong Wang. Uni3d: Exploring unified 3d representation at scale.arXiv preprint arXiv:2310.06773,

    Pith/arXiv arXiv

  60. [60]

    Partslip++: Enhancing low-shot 3d part segmentation via multi-view instance segmentation and max- imum likelihood estimation.ArXiv, abs/2312.03015, 2023

    Yuchen Zhou, Jiayuan Gu, Xuanlin Li, Minghua Liu, Yunhao Fang, and Hao Su. Partslip++: Enhancing low-shot 3d part segmentation via multi-view instance segmentation and max- imum likelihood estimation.ArXiv, abs/2312.03015, 2023. 2

    Pith/arXiv arXiv 2023

  61. [61]

    Yi, Leonidas J

    Chenyang Zhu, Kai Xu, Siddhartha Chaudhuri, L. Yi, Leonidas J. Guibas, and Hao Zhang. Adacoseg: Adaptive shape co-segmentation with group consistency loss.2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8540–8549, 2019. 2

    2020

  62. [62]

    ear”,“nose

    Xiangyang Zhu, Renrui Zhang, Bowei He, Ziyao Zeng, Shanghang Zhang, and Peng Gao. Pointclip v2: Adapting clip for powerful 3d open-world learning.arXiv preprint arXiv:2211.11682, 3(4), 2022. 1, 2, 6, 7 11 PatchAlign3D: Local Feature Alignment for Dense 3D Shape Understanding Supplementary Material 2D encoder mIoU cIoU DINOv1 [6] 51.82 54.39 DINOv3 [36] 46...

    Pith/arXiv arXiv 2022