ColChunk adaptively chunks visual document patches into contextual multi-vectors via clustering, cutting storage by over 90% while raising average nDCG@5 by 9 points.
Reducing the footprint of multi-vector retrieval with minimal per- formance impact via token pooling.arXiv preprint arXiv:2409.14683
7 Pith papers cite this work. Polarity classification is still indexing.
7
Pith papers citing it
abstract
Over the last few years, multi-vector retrieval methods, spearheaded by ColBERT, have become an increasingly popular approach to Neural IR. By storing representations at the token level rather than at the document level, these methods have demonstrated very strong retrieval performance, especially in out-of-domain settings. However, the storage and memory requirements necessary to store the large number of associated vectors remain an important drawback, hindering practical adoption. In this paper, we introduce a simple clustering-based token pooling approach to aggressively reduce the number of vectors that need to be stored. This method can reduce the space & memory footprint of ColBERT indexes by 50% with virtually no retrieval performance degradation. This method also allows for further reductions, reducing the vector count by 66%-to-75% , with degradation remaining below 5% on a vast majority of datasets. Importantly, this approach requires no architectural change nor query-time processing, and can be used as a simple drop-in during indexation with any ColBERT-like model.
citation-role summary
background 1
citation-polarity summary
years
2026 7roles
background 1polarities
background 1representative citing papers
Prune-then-Merge combines adaptive pruning of low-signal patches with hierarchical merging to achieve higher compression rates and better performance than prior single-stage methods in visual document retrieval.
A contextual multimodal document retrieval benchmark (CMDR-Bench) and embedding model (CMDR-Embed) that jointly encodes multiple document pages and splits them into page-level representations, trained with a context-aware contrastive objective, outperforming non-contextual baselines by 13–16 nDCG@5.
XTR training does not improve retrieval effectiveness over ColBERT but enhances IVF engine efficiency by flattening token scores to produce more discriminative centroids.
A Voronoi cell estimation framework in embedding space enables principled token pruning for late-interaction models, reducing index size while retaining retrieval quality.
LEMUR accelerates multi-vector retrieval by learning a neural network approximation to MaxSim and reducing it to single-vector search in latent space.
Lightweight pooling-aware fine-tuning with k-means on a single dataset enables up to 83% vector compression in ColBERT models with no retrieval accuracy loss and positive cross-dataset transfer.
citing papers explorer
-
Visual Late Chunking: An Empirical Study of Contextual Chunking for Efficient Visual Document Retrieval
ColChunk adaptively chunks visual document patches into contextual multi-vectors via clustering, cutting storage by over 90% while raising average nDCG@5 by 9 points.
-
Sculpting the Vector Space: Towards Efficient Multi-Vector Visual Document Retrieval via Prune-then-Merge Framework
Prune-then-Merge combines adaptive pruning of low-signal patches with hierarchical merging to achieve higher compression rates and better performance than prior single-stage methods in visual document retrieval.
-
CMDR: Contextual Multimodal Document Retrieval
A contextual multimodal document retrieval benchmark (CMDR-Bench) and embedding model (CMDR-Embed) that jointly encodes multiple document pages and splits them into page-level representations, trained with a context-aware contrastive objective, outperforming non-contextual baselines by 13–16 nDCG@5.
-
A Replicability Study of XTR
XTR training does not improve retrieval effectiveness over ColBERT but enhances IVF engine efficiency by flattening token scores to produce more discriminative centroids.
-
A Voronoi Cell Formulation for Principled Token Pruning in Late-Interaction Retrieval Models
A Voronoi cell estimation framework in embedding space enables principled token pruning for late-interaction models, reducing index size while retaining retrieval quality.
-
LEMUR: Learned Multi-Vector Retrieval
LEMUR accelerates multi-vector retrieval by learning a neural network approximation to MaxSim and reducing it to single-vector search in latent space.
-
Learn to Pool: Lightweight Fine-Tuning for Flexible Multi-Vector Compression
Lightweight pooling-aware fine-tuning with k-means on a single dataset enables up to 83% vector compression in ColBERT models with no retrieval accuracy loss and positive cross-dataset transfer.