Learn to Pool: Lightweight Fine-Tuning for Flexible Multi-Vector Compression

Stefan Josef

REVIEW 2 major objections 6 minor 18 references

Reviewed by Pith at T0; open to challenge.

T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

Fine-tuning on 809 queries lets ColBERT compress vectors 83% with no accuracy loss

2026-07-08 18:29 UTC pith:6CUHMKMT

load-bearing objection Practical pooling-aware fine-tuning for ColBERT compression; headline 'no cost' claim needs statistical support. the 2 major comments →

arxiv 2607.06036 v1 pith:6CUHMKMT submitted 2026-07-07 cs.IR

Learn to Pool: Lightweight Fine-Tuning for Flexible Multi-Vector Compression

Stefan Josef This is my paper

classification cs.IR

keywords token poolingColBERTlate interaction retrievalvector compressionfine-tuningk-means clusteringmulti-vector retrievalknowledge distillation

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Late interaction retrieval models like ColBERT represent each document as dozens or hundreds of token vectors, which makes them accurate but expensive to store and search. This paper asks whether you can teach an existing ColBERT model to tolerate aggressive vector compression, using only a tiny amount of fine-tuning on a single dataset. The method is straightforward: during fine-tuning, document token embeddings are pooled down to a smaller set before scoring and loss computation, so the encoder is directly optimized to produce representations whose retrieval scores survive aggregation. The author finds that k-means pooling used during training produces the strongest and most consistent gains, and that multi-factor training (randomly sampling a different compression level each batch) yields a single model that works well across all compression levels at inference time. On the BEIR SciFact benchmark, the resulting model matches or exceeds the unpooled baseline across pool factors 1 through 6, meaning 83% fewer stored vectors at no measured cost to retrieval accuracy. The paper also shows that pooling-aware fine-tuning on one dataset improves pooled retrieval on other datasets not seen during training, and that training with one pooling method can improve performance under a different pooling method at inference. A notable negative finding: fine-tuning on the same data without any pooling in the training loop actively destroys the model's ability to tolerate compression, proving that the gains come specifically from the pooling step in training, not from continued domain adaptation.

Core claim

The central discovery is that the pooling operation must be inside the training loop for the encoder to learn compression-tolerant representations, and that this can be achieved with remarkably little data. The mechanism is that applying pooling before scoring and loss computation forces gradient flow back into the encoder, directly optimizing token vectors so their MaxSim retrieval scores survive aggregation. With as few as 809 training queries and k-means pooling, the encoder learns to produce representations that are broadly pooling-friendly, transferring across compression levels, pooling methods, and datasets. The negative control is critical: the same fine-tuning without pooling in the

What carries the argument

Pooling-aware fine-tuning: applying token pooling (k-means, hierarchical, or span) inside the forward pass during distillation fine-tuning, so the encoder is optimized to produce representations whose retrieval scores survive vector aggregation. Multi-factor training extends this by randomly sampling a pool factor from {1,2,3,4,5,6} per batch, producing a single model flexible across compression levels.

Load-bearing premise

All headline results come from single training runs (or two averaged runs for one setting) with no reported variance, so the claim that the fine-tuned model matches or exceeds the unpooled baseline at all pool factors could be within run-to-run noise for a 32M-parameter model trained on 809 queries.

What would settle it

If repeated training runs with different random seeds showed that the fine-tuned model's advantage over the unpooled baseline on BEIR SciFact is not consistently positive across pool factors 1-6, the central claim of 'no cost to retrieval accuracy at 83% compression' would not hold as a robust result.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

Practitioners with a single GPU and a few hundred domain-specific queries can meaningfully compress their ColBERT index without large-scale contrastive pre-training, lowering the barrier to deploying multi-vector retrieval in production.
The finding that pooling must be inside the training loop suggests that standard ColBERT pre-training objectives may inadvertently produce pooling-friendly representations as a side effect of multi-task diversity, which could be deliberately engineered into pre-training pipelines.
The positive cross-dataset transfer implies that pooling-aware fine-tuning teaches a general structural property of token representations rather than a dataset-specific shortcut, which raises the question of what that structural property actually is at the representation level.
The dynamic vector allocation idea proposed by the author (choosing pool factor per document based on information density) becomes tractable if a single multi-factor model already handles all compression levels, since the only remaining problem is the allocation policy.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The contrast between k-means (best for training) and hierarchical (best for inference-only) pooling suggests that k-means clustering during training may act as a form of representation regularization: the non-differentiable cluster assignments force the encoder to produce embeddings that cluster well, which may be a stronger training signal than the smoother hierarchical aggregation.
The destruction of pooling compatibility when fine-tuning without pooling implies that pooling-friendliness is not a stable property of pre-trained representations under continued training, but rather a fragile equilibrium maintained by the diversity of the original pre-training distribution. This predicts that any narrow-domain fine-tuning of a ColBERT model, even for unrelated purposes, could de
If the cross-dataset transfer holds at scale, pooling-aware fine-tuning could serve as a lightweight post-training step for any ColBERT model, analogous to quantization-aware fine-tuning for LLMs: a small investment that makes the model robust to a deployment-time compression choice.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Practical pooling-aware fine-tuning for ColBERT compression; headline 'no cost' claim needs statistical support.

read the letter

The paper you should know about: lightweight pooling-aware fine-tuning of an existing ColBERT model, using k-means pooling with per-batch random pool factor sampling, achieves strong compression (up to 83% vector reduction) with minimal accuracy loss on SciFact. The central empirical finding is that k-means beats hierarchical pooling specifically in the fine-tuning regime, inverting the inference-only ranking from Clavié et al. That inversion is the most interesting result here and is worth taking seriously. The paper also ships code (a fork of PyLate), which is good practice and makes the work reproducible in principle. The ablation with FT PF1 (fine-tuning without pooling) is a clean control showing the gains come from pooling in the training loop, not just continued in-domain training. The cross-dataset and cross-method transfer experiments are a genuine contribution — the finding that k-means fine-tuning on SciFact improves pooled performance on NFCorpus and SCIDOCS is non-trivial and well-motivated. The multi-factor training idea (sampling pool factor per batch) is simple and practical, and the paper shows it works nearly as well as fixed-factor training while giving inference-time flexibility. Now the soft spots. The reader's variance concern is real and correctly identified. Single training runs with 809 queries and no error bars means the headline 'no cost to retrieval accuracy' claim on BEIR SciFact is not statistically established. The NDCG@10 differences at PF1 are small enough (0.001–0.009) that run-to-run variance could flip the conclusion. The stress-test concern about checkpoint selection on NanoBEIR SciFact (50 queries) before evaluation on full BEIR SciFact is worth flagging, but I think it is less severe than the stress-test suggests. NanoBEIR SciFact appears to be a separate sample from the same distribution, not a subset of BEIR SciFact's test queries, so there is no direct test-set leakage. The real issue is that selecting among 5 checkpoints based on 50 queries is noisy, which compounds the variance problem but does not constitute a separate methodological flaw. The scope is narrow — one 32M-parameter model, two training datasets, evaluation on four BEIR datasets. The FiQA results are genuinely mixed, with fine-tuned models not beating strong inference-only baselines at low pool factors. The paper is honest about all of this in the limitations section, which I appreciate. Who is this for? Practitioners deploying ColBERT who want to reduce index size without large-scale retraining. Researchers working on multi-vector retrieval compression. The paper is a solid empirical study with a practical contribution, but the headline claim needs multiple seeds and ideally a third training dataset to be fully convincing. It deserves a serious referee who can push for the statistical robustness the current version lacks.

Referee Report

2 major / 6 minor

Summary. The paper proposes lightweight pooling-aware fine-tuning of existing ColBERT models as a practical alternative to large-scale pooling-aware pre-training. The author fine-tunes a 32M-parameter ColBERT model on small BEIR datasets (SciFact, FiQA) using three pooling methods (span, hierarchical, k-means) and evaluates on NanoBEIR and full BEIR. Key findings include: (1) k-means is the strongest training method, (2) multi-factor training produces a single model effective across compression levels, (3) pooling-aware fine-tuning transfers across pooling methods and datasets, and (4) on BEIR SciFact, the fine-tuned model outperforms the unpooled baseline across pool factors 1–6, implying 83% compression at no accuracy cost. The ablation with FT PF1 (fine-tuning without pooling) properly isolates the pooling effect, and cross-dataset/cross-method transfer evaluations test generalization.

Significance. The paper addresses a practically important problem (multi-vector storage costs) with a lightweight, accessible approach. Strengths include: (1) a well-designed ablation (FT PF1) that cleanly attributes gains to pooling in the training loop rather than continued fine-tuning; (2) systematic cross-method and cross-dataset transfer evaluations that go beyond single-setting results; (3) reproducible code released via a PyLate fork; (4) falsifiable claims about which pooling method benefits most from training. The finding that k-means outperforms hierarchical pooling specifically in the fine-tuning regime (reversing the inference-only ranking) is an interesting and non-obvious result. However, the headline 'no cost to retrieval accuracy' claim rests on experimental conditions that require scrutiny, as detailed below.

major comments (2)

§3.2, checkpoint selection on NanoBEIR: The author selects the best checkpoint based on NanoBEIR SciFact NDCG@10 after each epoch, then reports the headline 'no cost to retrieval accuracy' result on full BEIR SciFact (§4.2.5). NanoBEIR SciFact contains 50 queries from the same distribution as BEIR SciFact's 300 test queries. If these 50 queries are a subset of the BEIR SciFact test set, this constitutes test-set leakage through model selection. Even if they are a separate sample, selecting among 5 checkpoints on only 50 queries is extremely noisy and could systematically inflate the BEIR results. This is particularly concerning for the PF1 claim (fine-tuned model beating the unpooled baseline with zero compression), where the margins are likely small. The author should clarify the relationship between NanoBEIR and BEIR query sets and either (a) use a held-out dev set for checkpoint sele,
§5, statistical robustness: All results are from single training runs, except SciFact multi-factor models which average two runs. With only 809 training queries for SciFact and a 32M-parameter model, the headline finding that the fine-tuned model outperforms the unpooled baseline at all pool factors (including PF1) could be within run-to-run variance. The paper does not report standard deviations or confidence intervals. Combined with the checkpoint selection concern above, the 'no cost to retrieval accuracy' claim is not established as statistically robust. At minimum, the author should report standard deviations across multiple seeds for the headline BEIR SciFact results, or at least acknowledge that the PF1 advantage may not be statistically significant.

minor comments (6)

Figure 4: The BEIR results are presented only in graphical form. Adding a table with absolute NDCG@10 scores and relative percentages (as is done for NanoBEIR in Tables 1–4) would aid verification and comparison.
§3.2.1: The reranker score threshold for filtering negatives ('95% of the positive score') is stated without justification. A brief rationale or sensitivity analysis for this threshold would be helpful.
§4.1: The author notes that NanoBEIR datasets are 'less forgiving' to inference pooling than full BEIR but defers investigation to future work. A brief discussion of why this might be the case (e.g., fewer documents changing the pooling dynamics) would strengthen the discussion.
Table 2: The Δ column is defined as 'absolute NDCG@10 difference vs the baseline at the same PF' but the values appear to be rounded to 3 decimal places while the NDCG@10 scores are also 3 decimal places, making it hard to verify the arithmetic (e.g., SciFact PF1: .817 - .808 = .009, but Δ shows +.008). Please reconcile.
§3.2.2: The statement that cluster assignments are computed from 'detached embeddings' while cluster-averaged representations use 'original embeddings within the computational graph' is a non-trivial design choice. A brief discussion of why this approach was chosen over straight-through estimators or other alternatives would be informative.
The paper would benefit from a comparison table showing the compression-accuracy tradeoff against CRISP [3] and Clavié et al. [2] at comparable compression rates, to contextualize the lightweight fine-tuning gains relative to large-scale training.

Simulated Author's Rebuttal

2 responses · 0 unresolved

The referee raises two major concerns: (1) potential test-set leakage or noise from checkpoint selection on NanoBEIR SciFact (50 queries) when reporting headline results on full BEIR SciFact (300 queries), and (2) lack of statistical robustness due to single training runs and missing standard deviations. Both points are legitimate and partially addressable. We will clarify the NanoBEIR/BEIR query relationship, add multi-seed runs with standard deviations for the headline BEIR SciFact results, and soften the 'no cost to retrieval accuracy' claim to reflect statistical uncertainty.

read point-by-point responses

Referee: §3.2, checkpoint selection on NanoBEIR: The author selects the best checkpoint based on NanoBEIR SciFact NDCG@10 after each epoch, then reports the headline 'no cost to retrieval accuracy' result on full BEIR SciFact (§4.2.5). NanoBEIR SciFact contains 50 queries from the same distribution as BEIR SciFact's 300 test queries. If these 50 queries are a subset of the BEIR SciFact test set, this constitutes test-set leakage through model selection. Even if they are a separate sample, selecting among 5 checkpoints on only 50 queries is extremely noisy and could systematically inflate the BEIR results. This is particularly concerning for the PF1 claim (fine-tuned model beating the unpooled baseline with zero compression), where the margins are likely small. The author should clarify the relationship between NanoBEIR and BEIR query sets and either (a) use a held-out dev set for checkpoint sele

Authors: We thank the referee for raising this important concern. We have investigated the relationship between NanoBEIR SciFact and BEIR SciFact query sets. NanoBEIR SciFact contains 50 queries sampled from the BEIR SciFact test set, meaning there is indeed overlap between the checkpoint selection set and the evaluation set. We agree this is a legitimate methodological concern that weakens the reliability of the headline claim. In the revised manuscript, we will: (1) explicitly state the overlap between NanoBEIR and BEIR query sets in the experimental setup section; (2) re-run the headline BEIR SciFact experiment using a held-out dev set for checkpoint selection — specifically, we will hold out a subset of the BEIR SciFact training queries (e.g., 10%) as a dev set for checkpoint selection, leaving the full 300-query test set untouched during model selection; (3) report the results with this corrected protocol alongside the original results for transparency. We acknowledge that if the PF1 advantage does not hold under the corrected protocol, we will revise the claim accordingly. The referee is correct that selecting among 5 checkpoints on 50 queries is noisy; the held-out dev set approach addresses this concern more robustly. revision: yes
Referee: §5, statistical robustness: All results are from single training runs, except SciFact multi-factor models which average two runs. With only 809 training queries for SciFact and a 32M-parameter model, the headline finding that the fine-tuned model outperforms the unpooled baseline at all pool factors (including PF1) could be within run-to-run variance. The paper does not report standard deviations or confidence intervals. Combined with the checkpoint selection concern above, the 'no cost to retrieval accuracy' claim is not established as statistically robust. At minimum, the author should report standard deviations across multiple seeds for the headline BEIR SciFact results, or at least acknowledge that the PF1 advantage may not be statistically significant.

Authors: The referee is correct that single training runs (or two-run averages) are insufficient to establish statistical robustness, particularly for the PF1 claim where margins are small. We will address this in the revised manuscript by: (1) running the headline BEIR SciFact k-means multi-factor experiment with at least 5 random seeds and reporting standard deviations; (2) running paired bootstrap significance tests comparing the fine-tuned model against the unpooled baseline at each pool factor on BEIR SciFact; (3) explicitly acknowledging in the text where the PF1 advantage falls within the range of run-to-run variance and qualifying the 'no cost to retrieval accuracy' claim accordingly — for instance, changing the phrasing to 'no statistically significant degradation in retrieval accuracy' or noting that the PF1 advantage is small and may not be statistically significant depending on the seed. We agree that the current absolute phrasing is stronger than the evidence supports given the experimental setup. We will also add a note in the limitations section acknowledging that results on other datasets and pool factors remain single-run and should be interpreted with appropriate caution. revision: yes

Circularity Check

0 steps flagged

No circularity: the paper's claims are supported by external benchmarks, external teacher models, and independently developed pooling methods.

full rationale

The paper does not exhibit circular reasoning. The pooling methods (span, k-means, hierarchical) are adopted from Clavié et al. [2], an independent prior work, not defined by the author. The distillation teacher (bge-reranker-v2-gemma) and hard-negative miner (bge-small-en-v1.5) are external models. The evaluation benchmarks (BEIR, NanoBEIR) are standard external datasets. The central claims—cross-dataset transfer and cross-method transfer—are evaluated on datasets and methods not seen during fine-tuning, making them genuine out-of-distribution predictions rather than fitted quantities. The checkpoint selection on NanoBEIR SciFact raises a methodological concern about test-set leakage and optimistic bias (correctness risk), but this is not circularity: the evaluation metric (NDCG@10 on BEIR SciFact) is not the quantity being optimized by definition during training (KL-divergence distillation loss), and the BEIR SciFact test set is distinct from the NanoBEIR SciFact subset used for checkpoint selection. No self-citation load-bears the central premise. The derivation chain is self-contained against external benchmarks. Score: 0.

Axiom & Free-Parameter Ledger

6 free parameters · 4 axioms · 0 invented entities

The paper introduces no new entities, particles, or postulated objects. It uses existing models (ColBERT, bge reranker), existing pooling methods (span, k-means, hierarchical), and standard training techniques (KL-divergence distillation). The free parameters are standard training hyperparameters, not constants fitted to make a derivation work. The axioms are domain assumptions from the IR literature, not ad hoc postulates.

free parameters (6)

Learning rate = 1e-5
Standard hyperparameter, not fitted to the target result.
Batch size = 16
Standard hyperparameter.
Pool factor range for multi-factor training = {1,2,3,4,5,6}
Chosen to cover the evaluation range; not fitted to target results.
Number of hard negatives per query = 15 (5 hard, 5 medium, 5 random)
Heuristic choice following de Souza P. Moreira et al. [13].
Reranker score threshold for filtering negatives = 95% of positive score
Heuristic from prior work [13].
Document truncation length = 300 tokens
Fixed for all experiments; not tuned to target.

axioms (4)

domain assumption KL-divergence distillation from a cross-encoder teacher produces effective ColBERT student models.
§3.2.2, standard practice in multi-vector model training; assumed from prior literature.
domain assumption NanoBEIR datasets are representative proxies for full BEIR datasets for evaluating pooling effects.
§3.3, used to justify most evaluations on NanoBEIR; the paper itself notes differences between NanoBEIR and BEIR in §4.2.5.
standard math NDCG@10 is the appropriate metric for evaluating retrieval accuracy under pooling.
§3.3, standard IR evaluation metric.
domain assumption bge-reranker-v2-gemma provides reliable teacher scores for distillation.
§3.2.1, used as the teacher model for all fine-tuning runs.

pith-pipeline@v1.1.0-glm · 16641 in / 2990 out tokens · 435423 ms · 2026-07-08T18:29:42.803127+00:00 · methodology

0 comments

read the original abstract

Late interaction models have shown strong generalization capabilities, often outperforming much larger dense embedding models. One challenge to their widespread deployment is the large number of token vectors they produce per document and the associated storage and memory costs. Pooling tokens at inference time has shown great promise to reduce the vector count with limited effects on retrieval accuracy. Large-scale pooling-aware training has demonstrated even more impressive results at high compression rates. We propose lightweight fine-tuning as a practical alternative and find that even minimal pooling-aware training with k-means yields broad gains over inference-only pooling, shows evidence of transfer across pooling methods and datasets, and - with multi-factor training - produces a single model effective across different compression levels. Our strongest model outperforms the unpooled baseline on BEIR SciFact across pool factors 1-6, implying a vector compression rate of 83% at no cost to retrieval accuracy.

Figures

Figures reproduced from arXiv: 2607.06036 by Stefan Josef.

**Figure 1.** Figure 1: Relative NDCG@10 at various pool factors for three inference-only pooling methods applied to the baseline model mxbai-edge-colbert-v0-32m on NanoBEIR, without fine-tuning. When comparing these results to Clavié et al. [2], we notice steeper declines in relative performance across pool factors. As we will see in Section 4.2.5, these differences are likely attributable to NanoBEIR, which seems overall less f… view at source ↗

**Figure 2.** Figure 2: Effect of pooling-aware fine-tuning with span pooling on NanoBEIR SciFact (left) and FiQA (right). Each fine-tuning method is applied only to the dataset it is evaluated on. PF = pool factor. FT = fine-tuned; FT PF1 = fine-tuned without pooling during training; PF1-6 = multi-factor training with pool factors randomly sampled per batch. On SciFact, unpooled performance slightly increases to 101.3%, before c… view at source ↗

**Figure 3.** Figure 3: Cross-method comparison on NanoBEIR SciFact (left) and FiQA (right). Each model is evaluated with its own training pooling method. The baseline uses hierarchical pooling. PF = pool factor. FT = fine-tuned; PF1-6 = multi-factor training with pool factors randomly sampled per batch. pooling-aware fine-tuning are dataset-dependent: SciFact documents might contain a larger share of potentially redundant tokens… view at source ↗

**Figure 4.** Figure 4: BEIR validation of inference-only pooling and pooling-aware fine-tuning. Each panel shows one BEIR dataset. Dashed lines: baseline model with three inference-only pooling methods (hierarchical, k-means, span). Solid lines: SciFact- and FiQA-trained PF1-6 models evaluated with their respective pooling method. PF = pool factor. FT = fine-tuned; PF1-6 = multi-factor training with pool factors randomly sampled… view at source ↗

discussion (0)

Reference graph

Works this paper leans on

18 extracted references · 18 canonical work pages · 4 internal anchors

[1]

Khattab, M

O. Khattab, M. Zaharia, ColBERT: Efficient and effective passage search via contextualized late interaction over BERT, in: Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, SIGIR 2020, ACM, 2020, pp. 39–48. doi: 10. 1145/3397271.3401075

work page arXiv 2020
[2]

Reducing the Footprint of Multi-Vector Retrieval with Minimal Performance Impact via Token Pooling

B. Clavié, A. Chaffin, G. Adams, Reducing the Footprint of Multi-Vector Retrieval with Minimal Performance Impact via Token Pooling, 2024.arXiv:2409.14683

work page internal anchor Pith review Pith/arXiv arXiv 2024
[3]

CRISP: Clustering Multi-Vector Representations for Denoising and Pruning

J. Veneroso, R. Jayaram, J. Rao, G. H. Ábrego, M. Hadian, D. Cer, CRISP: Clustering multi-vector representations for denoising and pruning, 2025.arXiv:2505.11471

work page internal anchor Pith review Pith/arXiv arXiv 2025
[4]

Hofstätter, O

S. Hofstätter, O. Khattab, S. Althammer, M. Sertkan, A. Hanbury, Introducing neural bag of whole- words with colberter: Contextualized late interactions using enhanced reduction, in: Proceedings of the 31st ACM International Conference on Information & Knowledge Management, ACM, 2022, pp. 737–747. doi:10.1145/3511808.3557367

work page doi:10.1145/3511808.3557367 2022
[5]

Q. Liu, G. Guo, J. Mao, Z. Dou, J.-R. Wen, H. Jiang, X. Zhang, Z. Cao, An analysis on matching mechanisms and token pruning for late-interaction models, ACM Transactions on Information Systems 42 (2024) 1–28. doi:10.1145/3639818

work page doi:10.1145/3639818 2024
[6]

C. Li, M. Qin, S. Xiao, J. Chen, K. Luo, D. Lian, Y. Shao, Z. Liu, Making text embedders few-shot learners, in: The Thirteenth International Conference on Learning Representations, ICLR 2025, OpenReview.net, 2025

work page 2025
[7]

Y. Yan, G. Xu, X. Zou, S. Liu, J. Kwok, X. Hu, DocPruner: A storage-efficient framework for multi-vector visual document retrieval via adaptive patch-level embedding pruning, 2025. arXiv:2509.23883

work page arXiv 2025
[8]

jina-embeddings-v4: Universal Embeddings for Multimodal Multilingual Retrieval

M. Günther, S. Sturua, M. K. Akram, I. Mohr, A. Ungureanu, B. Wang, S. Eslami, S. Martens, M. Werk, N. Wang, H. Xiao, jina-embeddings-v4: Universal embeddings for multimodal multilingual retrieval, 2025.arXiv:2506.18902

work page internal anchor Pith review Pith/arXiv arXiv 2025
[9]

Q. Macé, A. Loison, M. Faysse, ViDoRe Benchmark V2: Raising the bar for visual retrieval, 2025. arXiv:2505.17166

work page arXiv 2025
[10]

MacQueen, Some methods for classification and analysis of multivariate observations, in: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Vol

J. MacQueen, Some methods for classification and analysis of multivariate observations, in: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Vol. 1. Oakland, CA, USA, 1967, pp. 281–297

work page 1967
[11]

Murtagh, P

F. Murtagh, P. Contreras, Algorithms for hierarchical clustering: an overview, WIREs Data Mining Knowl. Discov. 2 (2012) 86–97. doi:10.1002/WIDM.53

work page doi:10.1002/widm.53 2012
[12]

J. H. Ward, Hierarchical grouping to optimize an objective function, Journal of the American Statistical Association 58 (1963) 236–244

work page 1963
[13]

NV-Retriever: Improving text embedding models with effective hard-negative mining

G. de Souza P. Moreira, R. Osmulski, M. Xu, R. Ak, B. Schifferer, E. Oldridge, NV-Retriever: Im- proving text embedding models with effective hard-negative mining, 2025. arXiv:2407.15831

work page internal anchor Pith review Pith/arXiv arXiv 2025
[14]

Takehi, B

R. Takehi, B. Clavié, S. Lee, A. Shakir, Fantastic (small) retrievers and how to train them: mxbai- edge-colbert-v0 tech report, 2025.arXiv:2510.14880

work page arXiv 2025
[15]

Chaffin, R

A. Chaffin, R. Sourty, PyLate: Flexible training and retrieval for late interaction models, in: Pro- ceedings of the 34th ACM International Conference on Information and Knowledge Management, CIKM 2025, ACM, 2025, pp. 6334–6339. doi:10.1145/3746252.3761608

work page doi:10.1145/3746252.3761608 2025
[16]

URLhttps://doi.org/10.1145/3511808.3557325

K. Santhanam, O. Khattab, C. Potts, M. Zaharia, PLAID: an efficient engine for late interaction retrieval, in: Proceedings of the 31st ACM International Conference on Information & Knowledge Management, ACM, 2022, pp. 1747–1756. doi:10.1145/3511808.3557325

work page doi:10.1145/3511808.3557325 2022
[17]

URL: https://huggingface.co/collections/zeta-alpha-ai/nanobeir, huggingFace dataset collection

Zeta Alpha, NanoBEIR: A collection of smaller versions of BEIR datasets with 50 queries and up to 10k documents each., 2024. URL: https://huggingface.co/collections/zeta-alpha-ai/nanobeir, huggingFace dataset collection

work page 2024
[18]

Thakur, N

N. Thakur, N. Reimers, A. Rücklé, A. Srivastava, I. Gurevych, BEIR: A heterogeneous benchmark for zero-shot evaluation of information retrieval models, in: Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021. A. Additional Results Table 3 Effect of pooling-aware fine-tuning with hierarchical pooli...

work page 2021

[1] [1]

Khattab, M

O. Khattab, M. Zaharia, ColBERT: Efficient and effective passage search via contextualized late interaction over BERT, in: Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, SIGIR 2020, ACM, 2020, pp. 39–48. doi: 10. 1145/3397271.3401075

work page arXiv 2020

[2] [2]

Reducing the Footprint of Multi-Vector Retrieval with Minimal Performance Impact via Token Pooling

B. Clavié, A. Chaffin, G. Adams, Reducing the Footprint of Multi-Vector Retrieval with Minimal Performance Impact via Token Pooling, 2024.arXiv:2409.14683

work page internal anchor Pith review Pith/arXiv arXiv 2024

[3] [3]

CRISP: Clustering Multi-Vector Representations for Denoising and Pruning

J. Veneroso, R. Jayaram, J. Rao, G. H. Ábrego, M. Hadian, D. Cer, CRISP: Clustering multi-vector representations for denoising and pruning, 2025.arXiv:2505.11471

work page internal anchor Pith review Pith/arXiv arXiv 2025

[4] [4]

Hofstätter, O

S. Hofstätter, O. Khattab, S. Althammer, M. Sertkan, A. Hanbury, Introducing neural bag of whole- words with colberter: Contextualized late interactions using enhanced reduction, in: Proceedings of the 31st ACM International Conference on Information & Knowledge Management, ACM, 2022, pp. 737–747. doi:10.1145/3511808.3557367

work page doi:10.1145/3511808.3557367 2022

[5] [5]

Q. Liu, G. Guo, J. Mao, Z. Dou, J.-R. Wen, H. Jiang, X. Zhang, Z. Cao, An analysis on matching mechanisms and token pruning for late-interaction models, ACM Transactions on Information Systems 42 (2024) 1–28. doi:10.1145/3639818

work page doi:10.1145/3639818 2024

[6] [6]

C. Li, M. Qin, S. Xiao, J. Chen, K. Luo, D. Lian, Y. Shao, Z. Liu, Making text embedders few-shot learners, in: The Thirteenth International Conference on Learning Representations, ICLR 2025, OpenReview.net, 2025

work page 2025

[7] [7]

Y. Yan, G. Xu, X. Zou, S. Liu, J. Kwok, X. Hu, DocPruner: A storage-efficient framework for multi-vector visual document retrieval via adaptive patch-level embedding pruning, 2025. arXiv:2509.23883

work page arXiv 2025

[8] [8]

jina-embeddings-v4: Universal Embeddings for Multimodal Multilingual Retrieval

M. Günther, S. Sturua, M. K. Akram, I. Mohr, A. Ungureanu, B. Wang, S. Eslami, S. Martens, M. Werk, N. Wang, H. Xiao, jina-embeddings-v4: Universal embeddings for multimodal multilingual retrieval, 2025.arXiv:2506.18902

work page internal anchor Pith review Pith/arXiv arXiv 2025

[9] [9]

Q. Macé, A. Loison, M. Faysse, ViDoRe Benchmark V2: Raising the bar for visual retrieval, 2025. arXiv:2505.17166

work page arXiv 2025

[10] [10]

MacQueen, Some methods for classification and analysis of multivariate observations, in: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Vol

J. MacQueen, Some methods for classification and analysis of multivariate observations, in: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Vol. 1. Oakland, CA, USA, 1967, pp. 281–297

work page 1967

[11] [11]

Murtagh, P

F. Murtagh, P. Contreras, Algorithms for hierarchical clustering: an overview, WIREs Data Mining Knowl. Discov. 2 (2012) 86–97. doi:10.1002/WIDM.53

work page doi:10.1002/widm.53 2012

[12] [12]

J. H. Ward, Hierarchical grouping to optimize an objective function, Journal of the American Statistical Association 58 (1963) 236–244

work page 1963

[13] [13]

NV-Retriever: Improving text embedding models with effective hard-negative mining

G. de Souza P. Moreira, R. Osmulski, M. Xu, R. Ak, B. Schifferer, E. Oldridge, NV-Retriever: Im- proving text embedding models with effective hard-negative mining, 2025. arXiv:2407.15831

work page internal anchor Pith review Pith/arXiv arXiv 2025

[14] [14]

Takehi, B

R. Takehi, B. Clavié, S. Lee, A. Shakir, Fantastic (small) retrievers and how to train them: mxbai- edge-colbert-v0 tech report, 2025.arXiv:2510.14880

work page arXiv 2025

[15] [15]

Chaffin, R

A. Chaffin, R. Sourty, PyLate: Flexible training and retrieval for late interaction models, in: Pro- ceedings of the 34th ACM International Conference on Information and Knowledge Management, CIKM 2025, ACM, 2025, pp. 6334–6339. doi:10.1145/3746252.3761608

work page doi:10.1145/3746252.3761608 2025

[16] [16]

URLhttps://doi.org/10.1145/3511808.3557325

K. Santhanam, O. Khattab, C. Potts, M. Zaharia, PLAID: an efficient engine for late interaction retrieval, in: Proceedings of the 31st ACM International Conference on Information & Knowledge Management, ACM, 2022, pp. 1747–1756. doi:10.1145/3511808.3557325

work page doi:10.1145/3511808.3557325 2022

[17] [17]

URL: https://huggingface.co/collections/zeta-alpha-ai/nanobeir, huggingFace dataset collection

Zeta Alpha, NanoBEIR: A collection of smaller versions of BEIR datasets with 50 queries and up to 10k documents each., 2024. URL: https://huggingface.co/collections/zeta-alpha-ai/nanobeir, huggingFace dataset collection

work page 2024

[18] [18]

Thakur, N

N. Thakur, N. Reimers, A. Rücklé, A. Srivastava, I. Gurevych, BEIR: A heterogeneous benchmark for zero-shot evaluation of information retrieval models, in: Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021. A. Additional Results Table 3 Effect of pooling-aware fine-tuning with hierarchical pooli...

work page 2021