REVIEW 8 cited by
Spreading vectors for similarity search
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Discretizing multi-dimensional data distributions is a fundamental step of modern indexing methods. State-of-the-art techniques learn parameters of quantizers on training data for optimal performance, thus adapting quantizers to the data. In this work, we propose to reverse this paradigm and adapt the data to the quantizer: we train a neural net which last layer forms a fixed parameter-free quantizer, such as pre-defined points of a hyper-sphere. As a proxy objective, we design and train a neural network that favors uniformity in the spherical latent space, while preserving the neighborhood structure after the mapping. We propose a new regularizer derived from the Kozachenko--Leonenko differential entropy estimator to enforce uniformity and combine it with a locality-aware triplet loss. Experiments show that our end-to-end approach outperforms most learned quantization methods, and is competitive with the state of the art on widely adopted benchmarks. Furthermore, we show that training without the quantization step results in almost no difference in accuracy, but yields a generic catalyzer that can be applied with any subsequent quantizer.
Forward citations
Cited by 8 Pith papers
-
STAR: A Spatial-Topology Aware Routing Framework for Generalizable 3D Scene Understanding
STAR adds spatial-topology-aware routing and uncertainty-based expert activation to a 3D mixture-of-experts model, reporting gains of 0.4 to 1.2 mIoU over strong baselines.
-
SleepMaMi: A Universal Sleep Foundation Model for Integrating Macro- and Micro-structures
SleepMaMi, a dual-encoder sleep foundation model pretrained on 158K hours of PSG, matches or beats existing sleep foundation models on staging, apnea segmentation, and disease prediction.
-
Beyond Sensor Data: Foundation Models of Behavioral Data from Wearables Improve Health Predictions
A foundation model of wearable behavioral data outperforms simple baselines and complements a PPG sensor model across 57 health detection tasks.
-
Asymmetric Dual Self-Distillation for 3D Self-Supervised Representation Learning
AsymDSD unifies latent masked point modeling and cross-view invariance self-distillation to learn 3D representations, reporting 90.53% on ScanObjectNN and 93.72% with 930k-shape pretraining.
-
Wearable Accelerometer Foundation Models for Health via Knowledge Distillation
A knowledge-distilled accelerometry encoder, taught by an unsupervised PPG teacher on 20 million minutes of paired wearable data, predicts heart rate, heart-rate variability, demographics, and 46 health conditions fro...
-
Sample- and Parameter-Efficient Auto-Regressive Image Models
XTRA shows that block-wise causal autoregressive pre-training improves sample and parameter efficiency over patch-level autoregressive image models such as AIM.
-
Improving Pre-trained Self-Supervised Embeddings Through Effective Entropy Maximization
Continuing SSL pre-training for ten epochs with a criterion that maximizes one-dimensional marginal entropies and minimizes pairwise covariances improves downstream ImageNet classification, especially with few labels.
-
PiPViT: Patch-based Visual Interpretable Prototypes for Retinal Image Analysis
PiPViT combines vision transformers and prototype learning to classify retinal OCT scans while showing the spatial extent of the biomarker that drove the decision.
Discussion (0). Continue with ORCID to comment.