Lossless compression of vector IDs in IVF and graph indexes, using random-order and random-edge coding plus wavelet trees, reduces ID storage by up to 7x and total index size by 30% at billion scale.
Random Edge Coding: One-Shot Bits-Back Coding of Large Labeled Graphs
1 Pith paper cite this work. Polarity classification is still indexing.
abstract
We present a one-shot method for compressing large labeled graphs called Random Edge Coding. When paired with a parameter-free model based on P\'olya's Urn, the worst-case computational and memory complexities scale quasi-linearly and linearly with the number of observed edges, making it efficient on sparse graphs, and requires only integer arithmetic. Key to our method is bits-back coding, which is used to sample edges and vertices without replacement from the edge-list in a way that preserves the structure of the graph. Optimality is proven under a class of random graph models that are invariant to permutations of the edges and of vertices within an edge. Experiments indicate Random Edge Coding can achieve competitive compression performance on real-world network datasets and scales to graphs with millions of nodes and edges.
citation-role summary
citation-polarity summary
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1roles
baseline 1polarities
baseline 1representative citing papers
citing papers explorer
-
Lossless Compression of Vector IDs for Approximate Nearest Neighbor Search
Lossless compression of vector IDs in IVF and graph indexes, using random-order and random-edge coding plus wavelet trees, reduces ID storage by up to 7x and total index size by 30% at billion scale.