Pith. sign in

REVIEW 3 major objections 6 minor 12 references

3D Part Segmentation via Geometric Aggregation of 2D Visual Features

T0 review · 3 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A training-free pipeline that separates part decomposition from semantic labelling reaches zero-shot state-of-the-art mIoU across five 3D part segmentation datasets.

desk verdict Strong training-free pipeline with a credible GFA module, but the paper never says where the part count P comes from—if it is read from ground truth, the headline numbers are not apples-to-apples. read the letter →

arxiv 2412.04247 v2 pith:X77Q2U3N submitted 2024-12-05 cs.CV

Marco Garosi , Riccardo Tedoldi , Davide Boscaini , Massimiliano Mancini , Nicu Sebe , Fabio Poiesi This is my paper · ORCID
classification cs.CV
keywords 3Dpartsegmentationzero-shotpointcloudvisionfoundationmodelDINOv2CLIPgeometricfeatureaggregationmulti-viewrendering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

COPS claims that open-vocabulary 3D part segmentation can be solved without any training by combining frozen 2D vision features with 3D geometry. It renders the point cloud from multiple views, lifts DINOv2 features to 3D, refines them with a geometric feature aggregation module, and only then clusters into the known number of parts and labels them with CLIP. Because part discovery and labelling are separated, segmentation quality no longer depends on prompt engineering. On ShapeNetPart it reports 62.5 instance-average mIoU versus 51.8 for PointCLIPv2, with improvements across five datasets spanning synthetic, real, textured, texture-less, rigid, and non-rigid shapes.

What carries the argument

The geometric feature aggregation (GFA) module: it samples 256 super points via farthest point sampling, then performs spatially-consistent aggregation (averaging features of the 10 nearest neighbours in 3D) followed by semantically-consistent aggregation (averaging over the 90 nearest neighbours in feature space), and interpolates back to all points. This makes features locally coherent and brings distant same-part points (e.g., chair armrests) together in latent space, enabling K-means to recover clean clusters.

What would settle it

Measure the ratio of average intra-part to inter-part cosine distances in the back-projected DINOv2 features on ShapeNetPart; if for some categories this ratio is not clearly below 1, GFA cannot recover part structure, and the claim that DINOv2 features carry part-level semantics for those categories fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that decoupling part decomposition from semantic labelling, and injecting 3D geometry into lifted 2D features, yields accurate zero-shot part segmentation from frozen components. The authors show that naive VLM prompting is fragile; instead, COPS first produces geometrically and semantically consistent point features via the GFA module, then applies K-means to cluster the features (given the part count), and finally assigns labels by Hungarian-matching clusters to CLIP's view-based part masks. The GFA module contributes roughly 3 mIoU points over the no-GFA baseline, and the full pipeline surpasses the previous state of the art by +10.7 points instance-average on ShapeNetPart and +16.8 on FAUST.

Load-bearing premise

The entire pipeline relies on the premise that DINOv2 features, once projected onto 3D points, place points of the same part close together in feature space and points of different parts far apart; the method also assumes the number of parts is known in advance.

Editorial extensions

If this is right

  • Zero-shot part segmentation becomes feasible without any training data or fine-tuning, using only frozen public models and point clouds.
  • Segmentation quality becomes robust to prompt wording, since prompts are used only for labelling already-separated parts, not for finding them.
  • The method works across depth-only and RGB inputs, as well as synthetic and real scanned point clouds, covering rigid and non-rigid shapes.
  • Part decomposition alone (without semantic labels) achieves much higher IoU, meaning the bottleneck shifts from geometric grouping to semantic naming.
  • The pipeline runs on a consumer GPU, with 9 hours on ShapeNetPart and 1 hour on ScanObjectNN at inference, making it practical for evaluation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the DINOv2 feature-space premise holds broadly, the GFA design could be lifted to other dense 2D-to-3D feature tasks, such as open-vocabulary scene segmentation, where geometry and semantics both matter.
  • The paper's layer ablation shows the best DINOv2 layer differs by category; an adaptive layer-selection mechanism is a direct testable extension that could raise performance on categories like mug and knife.
  • Because the part count P is a required input, the next bottleneck is automatic part-count discovery; the clustering step is a candidate place to integrate a nonparametric method.
  • The decoupling suggests that improving the semantic labeller (e.g., with stronger vision-language models) would mostly affect the final naming stage, leaving the geometric grouping unchanged.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes COPS, a training-free pipeline for zero-shot 3D part segmentation. It renders a point cloud into multiple views, extracts dense DINOv2 features, back-projects them onto 3D points, refines them through a geometric feature aggregation (GFA) module, clusters points via K-means into parts, and assigns semantic labels by matching clusters to vision-language-model anchors with the Hungarian algorithm. The authors report state-of-the-art zero-shot results on ShapeNetPart, PartNet, PartNetE, ScanObjectNN, and FAUST, with ablations on prompt type, number of views, backbone, GFA design, DINOv2 layer, and GFA hyperparameters.

Significance. If the results hold, COPS is a useful and practical contribution: it is training-free, builds on frozen foundation models, explicitly separates part decomposition from semantic labelling, and provides a unified evaluation across five datasets. The GFA module is simple and the ablation study is unusually detailed, including per-category numbers and hyperparameter grids; the release of code and the reported inference timings on a consumer GPU are also strengths. However, the headline zero-shot claims currently rest on an unstated source for the number of parts P and on hyperparameters selected on the test set used for the main tables. Resolving these issues is necessary before the central claims can be accepted.

major comments (3)
  1. [3.1 and 3.4] The decomposition stage requires the exact number of parts P: Sec. 3.1 states "We assume X to have P parts" and Sec. 3.4 says "Given a target of P parts, we apply K-means", yet the manuscript never states how P is obtained in the experiments. On ShapeNetPart and PartNet, the annotated part configuration is instance-dependent, so P cannot be derived solely from the category's part-name list. If P is read from the ground-truth annotation, COPS uses privileged test-time information that PointCLIPv2 and the other baselines do not use, making the headline gaps (+10.7 mIoUI on ShapeNetPart, Table 1, and similar margins in Tables 2-5) not apples-to-apples. Please specify the exact protocol for P; if it is taken from ground truth, provide experiments with a predicted P or with sensitivity to P, and add this limitation to the Limitations section.
  2. [4.4, App. D, App. E] Several central hyperparameters are selected on the ShapeNetPart test set used for the headline numbers, and no variance or multiple-seed statistics are reported. Appendix D chooses the DINOv2 layer that maximizes ShapeNetPart mIoU (Table 1), and Appendix E selects superpoint and neighbour counts the same way (Table 2); the ablations themselves show differences of several mIoU points and that the best layer varies by category. This weakens the robustness of the "zero-shot state-of-the-art" claim. Please either perform model selection on a validation split, or report results across multiple seeds and random point subsamples, and report the sensitivity of Tables 1-5 to these choices.
  3. [4.4, Fig. 5(a)] The paper claims that COPS is prompt-insensitive because prompts are used only for labelling, but Fig. 5(a) shows that the labelled COPS result drops from 63.7 mIoU with PointCLIPv2's original prompts to 28.1 with part names only, a drop of more than 35 points. While this is still substantially better than PointCLIPv2 under the same prompts, the claim in the Introduction that "the segmentation quality of COPS does not depend on prompts" is too strong as stated; prompt sensitivity remains a major performance factor for the labelled output and should be qualified.
minor comments (6)
  1. [4.1 vs 4.4] Sec. 4.1 says "We render 48 RGBD images per point cloud", but the ablation in Fig. 5(b) indicates that 10 views performs best; please state which view count was used to produce Tables 1-5 and whether the default differs from the ablation setting.
  2. [3.4 vs App. C] The main text describes semantic anchors as CLIP score maps, while App. C says labels are assigned "via Hungarian with PointCLIPv2's predictions"; unify this description.
  3. [Table 3] The sentence "COPS surpasses both PartSLIP and ZeroPS by +0.2%" refers only to aIoUC, and PartSLIP's aIoUI is not reported; a 0.2% gap is within typical noise and should be commented on accordingly.
  4. [Fig. 5(a)] The bars for COPS (63.7) exceed the drawn y-axis maximum of 60; adjust the axis limits for readability.
  5. [4.1, Sec. Metrics] The 'COPS↑' rows are described as an upper bound with a perfect labeller; this is reasonable, but they should not be interpreted as a standard zero-shot result because they still rely on the ground-truth part count and omit label assignment errors.
  6. [App. D] The highlighted row in Table 1 is the default used in the main paper, but the main paper does not state which DINOv2 layer is used; add this to the implementation details in Sec. 4.1 or App. C.

Circularity Check

2 steps flagged · score 5.0 of 10

Part-count P is supplied by the ground-truth annotation and the labelled COPS rows inherit PointCLIPv2 labels, but the GFA/clustering core is independently evaluated.

  1. fitted input called prediction [Sec. 3.1 (Problem formulation) and Sec. 3.4 (Shape-level clustering); App. C is silent on P]
    "We assume X to have P parts, i.e., X = ⋃P p=1 Xp ... Given a target of P parts, we apply K-means to split the point cloud into P regions {X1,··· ,XP}."

    P is an input assumption, not a predicted output, yet no experimental section states how P is obtained per instance. ShapeNetPart/PartNet part configurations are instance-dependent, so a category-level part list does not determine P; the value must be read from the ground-truth annotation to reproduce the reported numbers. The K-means head then produces exactly P regions by construction, so the part-count component of the claimed zero-shot decomposition is identical to the ground-truth part count. The headline SOTA gaps therefore encode test-set annotation as a hyperparameter and are not apples-to-apples against methods that do not receive this information.

  2. other [App. C (Point cloud processing); Sec. 3.4 (Semantic anchors extraction and Final segmentation)]
    "we randomly sample 2,048 points to obtain semantic labels via PointCLIPv2. Lastly, we perform clustering on these sampled points and we assign each cluster a semantic label via Hungarian with PointCLIPv2’s predictions."

    In the implementation used for the tables, the semantic labels of the reported COPS prediction are, by construction, PointCLIPv2's predictions after Hungarian alignment with COPS clusters. The paper then compares these labelled COPS numbers against PointCLIPv2 in Tables 1-5. The labelled rows therefore measure a hybrid whose label source is the very baseline being compared; they do not independently validate a novel labelling mechanism. The unlabeled 'COPS↑' rows avoid this, but the headline 'COPS' rows inherit the baseline's labels.

full rationale

The paper is a training-free empirical pipeline built on frozen external models (DINOv2, CLIP). The GFA module and the unlabeled K-means decomposition are self-contained against external benchmarks, and there is no load-bearing self-citation or imported uniqueness theorem: the authors' own prior works [3,4] are background only. The core circularity concern is the role of the ground-truth part count P: Sec. 3.1 assumes X has P parts and Sec. 3.4 runs K-means with exactly P clusters, while no experiment section or appendix states how P is set per instance. On ShapeNetPart/PartNet the annotated part sets are instance-dependent, so reproducing the tables likely requires reading P from the ground-truth annotation; the part-count dimension of the 'zero-shot' prediction then equals a test-set input by construction. A second, partial concern is that the labelled COPS rows use PointCLIPv2's own predictions as semantic anchors (App. C) and then compare against PointCLIPv2, so the label half of those rows is inherited from the baseline itself. The unlabeled 'COPS↑' rows and the GFA ablation are independent evidence, which prevents the whole claim from collapsing; the score reflects that the headline labelled SOTA numbers are partly constructed from the GT part count and the competitor's labels. Appendix D also selects the DINOv2 layer on the test split, a selection-on-test concern that is not circular but should be disclosed.

Assumptions & free parameters 8 free parameters · 4 assumptions · 0 invented entities

The pipeline is an engineering composition of frozen models and hand-set hyperparameters. The central claim does not rest on a mathematical derivation, so the ledger is dominated by empirical assumptions about DINOv2 feature geometry and by hyperparameters selected on the ShapeNetPart test set. No new physical entities are introduced.

free parameters (8)
  • Number of super points M = 256
    Chosen by ablation on ShapeNetPart (Appendix E); controls granularity of GFA.
  • Spatial GFA neighbour count = 10
    Default context window for spatial aggregation; the ablation shows a small window works best (Appendix E).
  • Semantic GFA neighbour count = 90
    Default context window for semantic aggregation; selected on ShapeNetPart (Appendix E).
  • Number of rendered views = 48 (default)
    Main implementation renders 48 RGBD views (Sec. 4.1), while the view-count ablation (Fig. 5b) indicates 10 views can be better with GFA; the choice is dataset and result dependent.
  • Hidden-point neighbour count L = 20
    Used to estimate features for points invisible in all views (Sec. 3.2).
  • DINOv2 feature layer = Layer 12 with normalization
    Selected via ShapeNetPart layer sweep (Appendix D); the optimal layer varies by category.
  • Rendering point size = 0.01 or 0.04 depending on dataset
    Set by dataset density (Sec. 4.1 and Supplementary C).
  • Point sampling counts = 10,000 for back-projection; 2,048 for semantic labelling
    Fixed in Supplementary C; hand-set implementation choices.
assumptions (4)
  • domain assumption DINOv2 feature space groups semantically similar points close together.
    Stated in Sec. 3.3; underlies semantic GFA and K-means clustering.
  • domain assumption Spatially close 3D points tend to belong to the same part.
    Used for hidden-point feature fill-in (Sec. 3.2) and spatial GFA (Sec. 3.3).
  • domain assumption Rendered multi-view depth or RGB images preserve the information DINOv2 needs for part semantics.
    The feature extractor relies on PyTorch3D renders; depth-only renders are used for ShapeNetPart (Supplementary C).
  • domain assumption The number of parts P is known and equals the number of K-means clusters and the number of CLIP or PointCLIPv2 masks.
    Problem formulation in Sec. 3.1 and clustering in Sec. 3.4 assume P is supplied.

how reviews work

0 comments
Cite this review

Pith. "Pith review of 3D Part Segmentation via Geometric Aggregation of 2D Visual Features." pith.science (2026). https://pith.science/paper/X77Q2U3N

@misc{pith2026241204247,
  author       = {Pith},
  title        = {Pith review of: 3D Part Segmentation via Geometric Aggregation of 2D Visual Features},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/X77Q2U3N}},
  note         = {Machine review of arXiv:2412.04247}
}
read the original abstract

Supervised 3D part segmentation models are tailored for a fixed set of objects and parts, limiting their transferability to open-set, real-world scenarios. Recent works have explored vision-language models (VLMs) as a promising alternative, using multi-view rendering and textual prompting to identify object parts. However, naively applying VLMs in this context introduces several drawbacks, such as the need for meticulous prompt engineering, and fails to leverage the 3D geometric structure of objects. To address these limitations, we propose COPS, a COmprehensive model for Parts Segmentation that blends the semantics extracted from visual concepts and 3D geometry to effectively identify object parts. COPS renders a point cloud from multiple viewpoints, extracts 2D features, projects them back to 3D, and uses a novel geometric-aware feature aggregation procedure to ensure spatial and semantic consistency. Finally, it clusters points into parts and labels them. We demonstrate that COPS is efficient, scalable, and achieves zero-shot state-of-the-art performance across five datasets, covering synthetic and real-world data, texture-less and coloured objects, as well as rigid and non-rigid shapes. The code is available at https://3d-cops.github.io.

Figures

Figures reproduced from arXiv: 2412.04247 by the authors.

Figure 1
Figure 1. The quality of part descriptions significantly affects the segmentation performance of methods based on vision-language models. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of COPS’s feature extractor. Φ (top) extracts point-level features by (i) rendering multiple views of the object, (ii) processing them with DINOv2, (iii) lifting them in 3D. The Geometric Feature Aggregation module (GFA, bottom) further refines these features by extracting super points (blue points in the second row) and their neighbouring points (red points in the second row) to obtain spatially consistent… view at source ↗
Figure 3
Figure 3. Qualitative results on ShapeNetPart [52]. Top to bottom: PointCLIPv2 [59], COPS, ground-truth. These results show that PointCLIPv2 often struggles in describing and segmenting some parts, such as the wheels of the skateboard or the wings of the plane. COPS instead produces a better segmentation, with more uniform part segments and sharper part boundaries. Input PointCLIPv2 COPS GT [PITH_FULL_IMAGE:figures/full_fig_… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Qualitative results on ScanObjectNN [45]. Top to bottom: input point cloud with color information; PointCLIPv2’s prediction; COPS’s prediction; ground-truth segmentation. COPS outputs better and sharper segmentations than PointCLIPv2. 4.4. Ablation study In [PITH_FULL…
Figure 5
Figure 5. Figure 5: Ablation on ShapeNetPart [7]. From left to right: (a) Different prompt types, comparing PointCLIPv2 [59] and COPS. (b) Varying the number of views during rendering, with and without our GFA module. (c) Changing the foundation model. (d) Ablating the GFA module. Number …
Figure 1
Figure 1. Figure 1: Detailed visualisation of the steps required by COPS. From left to right: input point cloud, intermediate features obtained by [PITH_FULL_IMAGE:figures/full_fig_p013_1.png]
Figure 2
Figure 2. Figure 2: Cameras visualised in 3D space. (a) shows the 6-camera setting, where the cameras are orthogonal to one another. (b) shows the [PITH_FULL_IMAGE:figures/full_fig_p013_2.png]
Figure 3
Figure 3. Figure 3: Qualitative results on ScanObjectNN [9]. Top to bottom: input point cloud with RGB colours, PointCLIPv2 predictions, COPS predictions, and ground-truth segmentation. mentation for 3d point clouds via pretrained image-language models. In CVPR, 2023. 3 [5] Kaichun Mo, Sh…
Figure 4
Figure 4. Figure 4: Qualitative results on FAUST [2]. Top to bottom: input texture-less point cloud (coloured in yellow for visualisation purposes), PointCLIPv2 predictions, COPS predictions, and ground-truth segmentation provided by SATR [1]. 6 [PITH_FULL_IMAGE:figures/full_fig_p017_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 7 canonical work pages

  1. [1]

    SATR: Zero-Shot Semantic Segmentation of 3D Shapes

    Ahmed Abdelreheem, Ivan Skorokhodov, Maks Ovsjanikov, and Peter Wonka. SATR: Zero-Shot Semantic Segmentation of 3D Shapes. In ICCV, 2023. 3, 6

    work page 2023

  2. [2]

    Federica Bogo, Javier Romero, Matthew Loper, and Michael J. Black. FAUST: Dataset and evaluation for 3D mesh registra- tion. In CVPR, 2014. 1, 3, 6

    work page 2014

  3. [3]

    An image is worth 16x16 words: Transformers for image recognition at scale, 2021

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale, 2021. 3

    2021

  4. [4]

    Minghua Liu, Yinhao Zhu, Hong Cai, Shizhong Han, Zhan Ling, Fatih Porikli, and Hao Su. Partslip: Low-shot part seg- 3 ViT-B layer mIoUI mIoUC Airplane Bag Cap Car Chair Earphone Guitar Knife Lamp Laptop Motorbike Mug Pistol Rocket Skate Table 1 0 49.7 46.4 31.1 53.0 45.7 27.9 47.5 50.8 60.7 66.6 45.5 71.3 21.6 42.9 35.6 31.0 48.8 61.8 2 1 55.9 52.1 31.5 7...

    work page 2023

  5. [5]

    Chang, Li Yi, Subarna Tripathi, Leonidas J

    Kaichun Mo, Shilin Zhu, Angel X. Chang, Li Yi, Subarna Tripathi, Leonidas J. Guibas, and Hao Su. PartNet: A large- scale benchmark for fine-grained and hierarchical part-level 3D object understanding. In CVPR, June 2019. 3

    work page 2019

  6. [6]

    Sup.” is the number of super points sampled from the input point cloud. “Nei

    Maxime Oquab, Timoth´ee Darcet, Th´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel 4 Sup. spat. Sup. sem. Nei. spat. Nei. sem. mIoUI mIoUC Airplane Bag Cap Car Chair Earphone Guitar Knife Lamp Laptop Motorbike Mug Pistol Rocket Skate Table Superpoints 1 512 256 10 90 64.2 60.0 50.6 68.3 69.2 30.1 71.6 65.4 74.2 78.6 48.9 ...

    arXiv 2023

  7. [7]

    Foundpose: Unseen object pose estimation with foundation features.arXiv preprint arXiv:2311.18809, 2023

    Evin Pınar ¨Ornek, Yann Labb ´e, Bugra Tekin, Lingni Ma, Cem Keskin, Christian Forster, and Tomas Hodan. Foundpose: Unseen object pose estimation with foundation features.arXiv preprint arXiv:2311.18809, 2023. 3

    arXiv 2023

  8. [8]

    Accelerating 3d deep learning with pytorch3d

    Nikhila Ravi, Jeremy Reizenstein, David Novotny, Tay- lor Gordon, Wan-Yen Lo, Justin Johnson, and Georgia Gkioxari. Accelerating 3d deep learning with pytorch3d. arXiv:2007.08501, 2020. 1

    arXiv 2007

Show all 12 references
  1. [9]

    Revisiting point cloud classification: A new benchmark dataset and classifica- tion model on real-world data

    Mikaela Angelina Uy, Quang-Hieu Pham, Binh-Son Hua, Duc Thanh Nguyen, and Sai-Kit Yeung. Revisiting point cloud classification: A new benchmark dataset and classifica- tion model on real-world data. In ICCV, 2019. 1, 3, 4

    2019

  2. [10]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. NeurIPS, 2017. 3

    2017

  3. [11]

    A scalable active framework for region annotation in 3d shape collections

    Li Yi, Vladimir G Kim, Duygu Ceylan, I-Chao Shen, Mengyan Yan, Hao Su, Cewu Lu, Qixing Huang, Alla Sheffer, and Leonidas Guibas. A scalable active framework for region annotation in 3d shape collections. ACM Transactions on Graphics, 35(6):1–12, 2016. 1, 3, 4, 5

    2016

  4. [12]

    Point- clip v2: Prompting clip and gpt for powerful 3d open-world learning

    Xiangyang Zhu, Renrui Zhang, Bowei He, Ziyu Guo, Ziyao Zeng, Zipeng Qin, Shanghang Zhang, and Peng Gao. Point- clip v2: Prompting clip and gpt for powerful 3d open-world learning. In ICCV, 2023. 1, 2 5 Ground-truth COPS PointCLIPv2 Input Figure 4. Qualitative results on FAUST ...

    2023

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.