REVIEW 6 cited by
PartSLIP++: Enhancing Low-Shot 3D Part Segmentation via Multi-View Instance Segmentation and Maximum Likelihood Estimation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Open-world 3D part segmentation is pivotal in diverse applications such as robotics and AR/VR. Traditional supervised methods often grapple with limited 3D data availability and struggle to generalize to unseen object categories. PartSLIP, a recent advancement, has made significant strides in zero- and few-shot 3D part segmentation. This is achieved by harnessing the capabilities of the 2D open-vocabulary detection module, GLIP, and introducing a heuristic method for converting and lifting multi-view 2D bounding box predictions into 3D segmentation masks. In this paper, we introduce PartSLIP++, an enhanced version designed to overcome the limitations of its predecessor. Our approach incorporates two major improvements. First, we utilize a pre-trained 2D segmentation model, SAM, to produce pixel-wise 2D segmentations, yielding more precise and accurate annotations than the 2D bounding boxes used in PartSLIP. Second, PartSLIP++ replaces the heuristic 3D conversion process with an innovative modified Expectation-Maximization algorithm. This algorithm conceptualizes 3D instance segmentation as unobserved latent variables, and then iteratively refines them through an alternating process of 2D-3D matching and optimization with gradient descent. Through extensive evaluations, we show that PartSLIP++ demonstrates better performance over PartSLIP in both low-shot 3D semantic and instance-based object part segmentation tasks. Code released at https://github.com/zyc00/PartSLIP2.
Forward citations
Cited by 6 Pith papers
-
MV-GEL: Language-Driven Multi-View Geometric Entity Localization on Meshes
MV-GEL uses a learned view selector and a fine-tuned vision-language segmentation model to localize text-described faces and edges on 3D meshes.
-
PatchAlign3D: Local Feature Alignment for Dense 3D Shape Understanding
A feed-forward 3D encoder aligning patch-level point-cloud features with part-name text embeddings achieves state-of-the-art zero-shot 3D part segmentation, surpassing multi-view rendering pipelines by large margins o...
-
AutoPartGen: Autogressive 3D Part Generation and Discovery
AutoPartGen generates 3D objects as a sequence of latent-space parts, conditioning each new part on previously generated parts, and reports state-of-the-art part completion on PartObjaverse-Tiny.
-
DexVLG: Dexterous Vision-Language-Grasp Model at Scale
DexVLG is a vision-language model trained on 170 million simulated dexterous grasps that generates hand poses aligned with language instructions about which part of an object to grasp.
-
GeoSAM2: Unleashing the Power of SAM2 for 3D Part Segmentation
A prompt-controllable 3D part segmentation method that adapts SAM2 with LoRA and geometry fusion on rendered normal and point maps, then back-projects multi-view masks to the mesh.
-
OpenHuman4D: Open-Vocabulary 4D Human Parsing
OpenHuman4D combines a video tracker with mask validation and attention-based embedding fusion to deliver fast, text-queryable segmentation of 4D human videos.
Discussion (0). Sign in to comment.