LangTail uses entity-level semantic priors from language models aligned via contrastive learning in a hierarchical clustering setup to resolve long-tail ambiguity, yielding +13.5, +12.9, and +8.9 mIoU gains on ScanNet-v2, S3DIS, and nuScenes.
Unsupervised semantic segmenta- tion by distilling feature correspondences
9 Pith papers cite this work. Polarity classification is still indexing.
9
Pith papers citing it
abstract
Unsupervised semantic segmentation aims to discover and localize semantically meaningful categories within image corpora without any form of annotation. To solve this task, algorithms must produce features for every pixel that are both semantically meaningful and compact enough to form distinct clusters. Unlike previous works which achieve this with a single end-to-end framework, we propose to separate feature learning from cluster compactification. Empirically, we show that current unsupervised feature learning frameworks already generate dense features whose correlations are semantically consistent. This observation motivates us to design STEGO ($\textbf{S}$elf-supervised $\textbf{T}$ransformer with $\textbf{E}$nergy-based $\textbf{G}$raph $\textbf{O}$ptimization), a novel framework that distills unsupervised features into high-quality discrete semantic labels. At the core of STEGO is a novel contrastive loss function that encourages features to form compact clusters while preserving their relationships across the corpora. STEGO yields a significant improvement over the prior state of the art, on both the CocoStuff ($\textbf{+14 mIoU}$) and Cityscapes ($\textbf{+9 mIoU}$) semantic segmentation challenges.
representative citing papers
Boxes2Pixels distills noisy SAM pseudo-masks into a compact DINOv2-based student with auxiliary localization and one-sided self-correction, delivering +6.97 anomaly mIoU and +9.71 binary IoU gains over baselines on wind turbine data with 80% fewer parameters.
New CVSI module with IVSP and RAGC losses for single-reference unseen object 6D pose estimation that reports state-of-the-art results on six BOP benchmarks.
TACoS achieves over 96% of fully supervised segmentation performance on 2D material flakes using less than 0.6% annotated pixels via a unified framework of consistency learning, tree energy regularization, and asymmetric contrastive learning.
Casper3D is a backbone-agnostic variational model that infers stable 3D semantics from noisy 2D embeddings by treating them as observations of a latent 3D state and training via held-out viewpoint prediction.
A visualization protocol using unsupervised semantic segmentation outputs reveals positional biases, scaling behaviors, and boundary artifacts in self-supervised ViTs and distinguishes them from locality bias.
Register tokens improve pixel-space Diffusion Transformers by cleaning high-noise feature maps, and Register Guidance amplifies that effect.
SSL clustering is derived as KL-divergence optimization where a teacher-distribution constraint normalizes via inverse cluster priors and simplifies to batch centering by Jensen's inequality.
An unsupervised monocular road segmentation method generates initial labels from horizon and quadrilateral geometric priors then refines them via temporal feature tracking and mutual information to reach 0.86 IoU on Cityscapes.
citing papers explorer
-
Resolving Long-Tail Ambiguity in Unsupervised 3D Point Cloud Segmentation with Language Priors
LangTail uses entity-level semantic priors from language models aligned via contrastive learning in a hierarchical clustering setup to resolve long-tail ambiguity, yielding +13.5, +12.9, and +8.9 mIoU gains on ScanNet-v2, S3DIS, and nuScenes.
-
Boxes2Pixels: Learning Defect Segmentation from Noisy SAM Masks
Boxes2Pixels distills noisy SAM pseudo-masks into a compact DINOv2-based student with auxiliary localization and one-sided self-correction, delivering +6.97 anomaly mIoU and +9.71 binary IoU gains over baselines on wind turbine data with 80% fewer parameters.
-
Learning Cross-View Semantic Priors for Single-Reference Unseen Object Pose Estimation
New CVSI module with IVSP and RAGC losses for single-reference unseen object 6D pose estimation that reports state-of-the-art results on six BOP benchmarks.
-
TACoS: Weakly Supervised Learning of Two-Dimensional Materials from Scribble Annotations to Precise Segmentation
TACoS achieves over 96% of fully supervised segmentation performance on 2D material flakes using less than 0.6% annotated pixels via a unified framework of consistency learning, tree energy regularization, and asymmetric contrastive learning.
-
Lightweight 3D Feature Pretraining by Bayesian Inversion of 2D Foundation Models
Casper3D is a backbone-agnostic variational model that infers stable 3D semantics from noisy 2D embeddings by treating them as observations of a latent 3D state and training via held-out viewpoint prediction.
-
Unsupervised Semantic Segmentation Facilitates Model Understanding
A visualization protocol using unsupervised semantic segmentation outputs reveals positional biases, scaling behaviors, and boundary artifacts in self-supervised ViTs and distinguishes them from locality bias.
-
Registers Matter for Pixel-Space Diffusion Transformers
Register tokens improve pixel-space Diffusion Transformers by cleaning high-noise feature maps, and Register Guidance amplifies that effect.
-
Information theoretic underpinning of self-supervised learning by clustering
SSL clustering is derived as KL-divergence optimization where a teacher-distribution constraint normalizes via inverse cluster priors and simplifies to batch centering by Jensen's inequality.
-
Unsupervised Monocular Road Segmentation for Autonomous Driving via Scene Geometry
An unsupervised monocular road segmentation method generates initial labels from horizon and quadrilateral geometric priors then refines them via temporal feature tracking and mutual information to reach 0.86 IoU on Cityscapes.