Pith. sign in

REVIEW 5 cited by

Similarity search in the blink of an eye with compressed indices

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2304.04759 v2 pith:RSE7JJKQ submitted 2023-04-07 cs.LG cs.IR

Cecilia Aguerrebere , Ishwar Bhati , Mark Hildebrand , Mariano Tepper , Ted Willke This is my paper · ORCID
classification cs.LGcs.IR
keywords memorysearchsimilarityfootprintgraph-basedindicesvectorsbillions
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Nowadays, data is represented by vectors. Retrieving those vectors, among millions and billions, that are similar to a given query is a ubiquitous problem, known as similarity search, of relevance for a wide range of applications. Graph-based indices are currently the best performing techniques for billion-scale similarity search. However, their random-access memory pattern presents challenges to realize their full potential. In this work, we present new techniques and systems for creating faster and smaller graph-based indices. To this end, we introduce a novel vector compression method, Locally-adaptive Vector Quantization (LVQ), that uses per-vector scaling and scalar quantization to improve search performance with fast similarity computations and a reduced effective bandwidth, while decreasing memory footprint and barely impacting accuracy. LVQ, when combined with a new high-performance computing system for graph-based similarity search, establishes the new state of the art in terms of performance and memory footprint. For billions of vectors, LVQ outcompetes the second-best alternatives: (1) in the low-memory regime, by up to 20.7x in throughput with up to a 3x memory footprint reduction, and (2) in the high-throughput regime by 5.8x with 1.4x less memory.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MERIT: Efficient In-Place Deletion for Dynamic Graph-Based Approximate Nearest Neighbor Indexes

    cs.DB 2026-07 conditional novelty 6.5 of 10

    MERIT makes vector-graph deletions cheap by repairing only a bounded local neighborhood via k_r-MST and invalidating all leftover stale edges with per-target version stamps.

  2. SymphonyQG: Towards Symphonious Integration of Quantization and Graph for Approximate Nearest Neighbor Search

    cs.DB 2024-11 conditional novelty 6.0 of 10

    A graph-based ANN search method that integrates RaBitQ quantization and SIMD batching, with implicit re-ranking and batch-aligned graph refinement, sets a new time-accuracy state of the art.

  3. OneDB: A Distributed Multi-Metric Data Similarity Search System

    cs.DB 2025-07 conditional novelty 5.0 of 10

    A Spark-based system that indexes each data modality separately and combines them with learned weights to support exact distributed similarity search across multi-modal data.

  4. CHASE: A Native Relational Database for Hybrid Queries on Structured and Unstructured Data

    cs.DB 2025-01 conditional novelty 5.0 of 10

    CHASE natively integrates ANN vector search into a compiled relational engine, cuts redundant similarity computation in hybrid queries, and reports up to 7,500x speedups.

  5. Arctic-Embed 2.0: Multilingual Retrieval Without Compromise

    cs.CL 2024-12 conditional novelty 5.0 of 10

    Arctic-Embed 2.0 delivers open multilingual embedding models with competitive MTEB-R and CLEF retrieval scores and strong 256-dimension MRL compression, plus new evidence on cross-lingual transfer.

Pith tools