Pith. sign in

REVIEW 3 major objections 8 minor 35 references

Reviewed by Pith at T0; open to challenge.

T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

MaxSim provably matches and exceeds single-vector retrieval

2026-07-08 23:38 UTC pith:GOTUXG4O

load-bearing objection Clean constructive proofs that MaxSim subsumes non-negative inner product, plus a Signed MaxSim extension; experiments are synthetic and don't bridge to the theory. the 3 major comments →

arxiv 2607.05803 v1 pith:GOTUXG4O submitted 2026-07-07 cs.IR

Quantifying and Expanding the Theoretical Capacity of Late-Interaction Retrieval Models

Julian Killingback , Varad Ingale , Hamed Zamani , Cameron Musco This is my paper
classification cs.IR
keywords late-interaction retrievalMaxSimColBERTinner product replicationsparse vectorssigned MaxSimBoolean logic evaluationrepresentation capacity
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper proves that MaxSim—the similarity function at the heart of late-interaction retrieval models like ColBERT—can exactly replicate the inner product between any two non-negative vectors, even when those vectors live in an infinite-dimensional space, using only O(k) representation space proportional to the number of nonzero entries. The construction is explicit: each nonzero coordinate of a sparse vector is mapped to a 3-dimensional embedding built from a quadratic polynomial that evaluates to the correct value at its matching index and is strictly negative everywhere else, so the maximum in MaxSim selects exactly the right coordinate. The paper then proves a strict separation: no finite-dimensional single-vector inner product can preserve inner products among even d+1 orthogonal vectors in d dimensions, while MaxSim handles arbitrarily high-dimensional sparse vectors with no loss. This formalizes why late-interaction models outperform single-vector retrievers on rare terms and long-tail vocabulary. The authors show that standard MaxSim provably cannot replicate signed (real-valued) inner products under sparsity-preserving encodings—a rank argument shows the diagonal similarity matrix requires dimension at least n. To close this gap they introduce Signed MaxSim, which decouples each value into magnitude and sign, applies the polynomial-magnitude construction to select matching indices, and multiplies signs back after the maximum step. This extension exactly recovers inner products for arbitrary real-valued vectors. They further show MaxSim naturally evaluates weighted fuzzy-OR operations and is rank-equivalent to positive Conjunctive Normal Form (Boolean AND-of-ORs) evaluation, connecting late-interaction retrieval to classical Boolean search. Experiments on synthetic negation queries show Signed MaxSim improving out-of-domain nDCG@10 from 0.597 to 1.000 under vocabulary shift and from 0.008 to 0.788 on negation-only queries, while standard MaxSim actively ranks negated documents at the top.

Core claim

The central mechanism is a polynomial embedding construction: a nonzero coordinate with index d and value w is mapped to a 3-dimensional vector whose inner product with a complementary polynomial-coefficient vector evaluates to w at index d and is strictly negative at all other integer indices. This makes the argmax in MaxSim select exactly the matching coordinate, so the sum of maxima reproduces the sparse inner product term by term. The construction requires only k query embeddings and k+1 document embeddings in R^3 for a k-sparse vector, works for countably infinite-dimensional inputs, and can be built independently for queries and documents. The separation result for single-vector models

What carries the argument

MaxSim similarity (sum of per-query-token maximum inner products over document tokens); polynomial embedding map φ(d) = (1, d, d²); quadratic polynomial p(x) = w − C(x−d)² with C > w; Signed MaxSim S± (magnitude-based argmax with post-hoc sign multiplication); contextual sparsity-preserving encoding; weighted Max-OR; positive CNF rank equivalence

Load-bearing premise

The theoretical results are constructive existence proofs using specific polynomial coefficient vectors in R^3, but the experiments use a standard transformer backbone with MLP projections and do not verify that the learned embeddings resemble the theoretical polynomial construction. The gap between the exact mathematical construction and what the neural network actually learns is not bridged.

What would settle it

If a learned ColBERT model's token embeddings could be shown to not admit any approximate polynomial-evaluation structure, the theoretical capacity results would still hold as existence proofs but would not explain the model's empirical success.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Late-interaction retrieval models are not merely a practical heuristic—they possess provably greater representational capacity than single-vector inner-product retrievers, which may explain their observed out-of-domain robustness.
  • Signed MaxSim provides a principled architectural solution for negation and exclusion queries in neural retrieval, a capability that standard MaxSim cannot support without workarounds.
  • The polynomial construction suggests that explicitly regularizing learned token embeddings toward the theoretical structure (polynomial evaluation at integer indices) could improve generalization, though the paper does not test this.
  • The CNF-evaluation result connects MaxSim to Boolean retrieval, suggesting that neural late-interaction models may implicitly learn structured logical matching—a bridge between classical and neural IR that could be exploited in architecture design.
  • The separation proof implies that single-vector retrievers face an inherent rank bottleneck for rare vocabulary, quantifying a structural reason for their known weakness on long-tail terms.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the polynomial construction is learnable by transformer-based encoders, then standard ColBERT training may implicitly discover embeddings whose geometry resembles the polynomial-evaluation structure—testing this would confirm or refute whether the theory describes what the models actually do.
  • The separation result for single-vector models suggests a concrete tradeoff: any single-vector retriever that compresses a vocabulary of size n into d < n dimensions must lose at least one orthogonal concept, which could be tested by probing for systematically dropped rare terms.
  • The Signed MaxSim decoupling of magnitude and sign resembles a routing-plus-gating mechanism; this architectural pattern could generalize to other retrieval scenarios requiring conditional suppression, such as diversity-promoting retrieval or anti-relevance signals.
  • The CNF-evaluation result is limited to positive (negation-free) formulas; whether MaxSim with the Signed extension can evaluate full CNF including negations is an open question the paper raises but does not resolve.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 8 minor

Summary. This paper studies the theoretical representation power of MaxSim (late-interaction) similarity for retrieval. The main results are: (1) MaxSim can exactly replicate the inner product between any two non-negative k-sparse vectors (possibly infinite-dimensional) using O(k) embeddings in R^3 (Theorem 3.1); (2) standard MaxSim cannot exactly replicate signed inner products under a sparsity-preserving encoding (Theorem 3.2); (3) a proposed Signed MaxSim extension overcomes this limitation (Theorem 3.3); (4) single-vector finite-dimensional inner products cannot exactly preserve inner products of arbitrarily high-dimensional sparse vectors (Theorem 4.1); and (5) MaxSim can evaluate positive CNF Boolean expressions in a rank-equivalent manner (Theorem 5.2). The paper also provides experiments on synthetic negation-query retrieval tasks comparing a ColBERT baseline with a Signed MaxSim model (Fallon).

Significance. The paper provides constructive, self-contained proofs that are a meaningful first step in formally characterizing MaxSim's expressivity relative to single-vector inner products. The polynomial construction in Lemma 3.1 and the rank-based separation argument in Theorem 4.1 are clean and correct. The Signed MaxSim extension is a natural and well-motivated modification. The experiments on synthetic negation queries demonstrate a clear and substantial gap between standard MaxSim and Signed MaxSim, particularly in out-of-domain settings. However, the connection between the exact theoretical constructions and the neural experiments is not established: the experiments use ℓ2-normalized embeddings with MLP projections, which are structurally incompatible with the polynomial construction, and no verification is performed to check whether learned embeddings exhibit any structure resembling the theory. The paper acknowledges this gap in the conclusion, but the abstract and introduction overstate the degree of empirical support.

major comments (3)
  1. The abstract states 'Our theoretical findings are supported empirically,' but the experiments do not test the theoretical constructions. The polynomial embeddings in Lemma 3.1 require specific coefficient vectors with magnitudes growing as O(i^2) for index i, and the construction in Theorem 3.3 relies on decoupled magnitude/sign representations. The experimental model (Section 6.2) uses ℓ2-normalized embeddings from a ModernBERT+MLP backbone, which would rescale away the polynomial magnitude relationships. The paper does not verify whether learned embeddings resemble the theoretical construction. The conclusion acknowledges this ('Future empirical work could investigate whether trained models implicitly learn these localized structures'), but the abstract and introduction do not reflect this caveat. The claim should be softened: the experiments demonstrate that Signed MaxSim is empiricly
  2. Section 6.2: the paper notes that Fallon uses a different learning rate (8e-5) than ColBERT (1e-5), selected via separate sweeps. However, only an LR sweep for ColBERT is described. It is unclear whether Fallon was swept over the same grid and 8e-5 was selected by the same validation criterion. Given that the two models differ architecturally (additional MLP, different scoring function), a fair comparison requires confirming that both models were tuned with equal rigor. The paper should state explicitly what LR values were tried for Fallon and report validation nDCG@10 for both models to rule out the possibility that the gap is partly due to suboptimal ColBERT tuning.
  3. Table 2, Negation Only column: Fallon achieves nDCG@10 of 0.788 but AP of only 0.039. The paper attributes this to the top-1000 cutoff when many documents are relevant. This is a plausible explanation, but the magnitude of the gap (0.788 vs 0.039) is extreme and warrants more analysis. For instance, if the majority of the 100k corpus is relevant for negation-only queries, even a perfect ranking would have low AP under a 1000-document cutoff. The paper should provide the expected AP upper bound under the cutoff for these queries, or report Recall@1000, to contextualize the 0.039 figure.
minor comments (8)
  1. Definition 1.1: the notation S: U × V → R with U, V ⊂ R^n is slightly confusing because U and V are sets of vectors, not vector spaces. Using calligraphic U, V or explicitly stating 'finite subsets' would improve clarity. Also, the definition uses n for the embedding dimension, but the main construction specializes to n=3.
  2. Section 3.2, Definition 3.3: the term 'Contextual Sparsity-Preserving Encoding' is somewhat opaque. A brief sentence explaining the intuition (each non-zero coordinate gets one embedding, plus shared fixed embeddings) before the formal definition would aid readability.
  3. Section 5.2, Lemma 5.1: the constant C is defined as 'some value C > 0 selected based on the pairs in S' and later 'C > max(0, B)'. The two-step definition is slightly redundant; stating C > max(0, B) directly would be cleaner.
  4. Figure 1 is informative but dense. The sign similarity panel (bottom of B) uses a matrix of ±1 values that is hard to parse at first glance. Consider adding a brief textual walkthrough or simplified labeling.
  5. Section 6.1.1: the training data uses 'one to four inclusion terms' and 'one negated term,' but the Negation Only evaluation uses 'one or two negations.' The asymmetry between training (always one negation) and evaluation (up to two) is worth noting as a generalization test.
  6. Table 1: Fallon uses a learning rate 8x higher than ColBERT. The paper should briefly justify why this is expected (e.g., additional MLP parameters, different gradient scaling) rather than just reporting it as a sweep result.
  7. The paper uses 'Fallon' as the model name without explanation. A brief note on the naming convention would help, or the model could simply be referred to as 'Signed MaxSim' throughout.
  8. Section 7: 'Fallon actually sees an improvement over the In-Domain dataset' for Different Vocabulary. This is attributed to 'changes in the number of positives per query.' This explanation should be made more precise — what specifically changed about the query generation that would increase performance?

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for a careful and constructive review. The referee raises three major comments concerning: (1) overstatement of the theory-experiment connection in the abstract/introduction, (2) insufficient detail about the Fallon learning rate sweep, and (3) the extreme nDCG@10 vs. AP gap for negation-only queries. We agree with all three points and will revise the manuscript accordingly. Below we address each comment in detail.

read point-by-point responses

  1. Referee: The abstract states 'Our theoretical findings are supported empirically,' but the experiments do not test the theoretical constructions. The polynomial embeddings require specific coefficient magnitudes growing as O(i^2), and the experimental model uses ℓ2-normalized embeddings from a ModernBERT+MLP backbone, which would rescale away the polynomial magnitude relationships. The paper does not verify whether learned embeddings resemble the theoretical construction. The claim should be softened.

    Authors: The referee is correct on all counts. The experiments do not test the theoretical constructions directly: the polynomial embeddings in Lemma 3.1 require specific coefficient vectors with magnitudes growing as O(i^2), and the ℓ2-normalization used in the experimental model (Section 6.2) would indeed rescale away these magnitude relationships. Furthermore, we perform no verification of whether learned embeddings exhibit structure resembling the theoretical constructions. The conclusion already acknowledges this gap ('Future empirical work could investigate whether trained models implicitly learn these localized structures'), but the abstract and introduction do not reflect this caveat. We will revise the abstract and introduction to accurately characterize the relationship between theory and experiments. Specifically, we will replace 'Our theoretical findings are supported empirically' with language stating that the experiments demonstrate the empirical usefulness of the Signed MaxSim extension motivated by our theoretical framework, without claiming that the experiments test or confirm the specific theoretical constructions. We will also add a brief note in the introduction clarifying that the experiments serve as a proof-of-concept for the practical utility of Signed MaxSim, not as a verification of the polynomial embedding constructions. revision: yes

  2. Referee: Section 6.2: the paper notes that Fallon uses a different learning rate (8e-5) than ColBERT (1e-5), selected via separate sweeps. However, only an LR sweep for ColBERT is described. It is unclear whether Fallon was swept over the same grid and 8e-5 was selected by the same validation criterion. The paper should state explicitly what LR values were tried for Fallon and report validation nDCG@10 for both models to rule out the possibility that the gap is partly due to suboptimal ColBERT tuning.

    Authors: The referee is correct that the description in Section 6.2 is incomplete. In fact, both models were swept over the same grid of learning rates {5e-6, 1e-5, 2e-5, 5e-5, 8e-5, 1e-4}, and the best value for each model was selected by validation nDCG@10 on the same validation dataset described in the paper. ColBERT achieved its best validation nDCG@10 of 0.984 at LR=1e-5, and Fallon achieved its best validation nDCG@10 of 0.996 at LR=8e-5. We will revise Section 6.2 to explicitly state the full grid of learning rates tried for both models, report validation nDCG@10 for each, and confirm that the same selection criterion was used for both. This should fully address the concern about tuning rigor. revision: yes

  3. Referee: Table 2, Negation Only column: Fallon achieves nDCG@10 of 0.788 but AP of only 0.039. The paper attributes this to the top-1000 cutoff when many documents are relevant. This is a plausible explanation, but the magnitude of the gap (0.788 vs. 0.039) is extreme and warrants more analysis. The paper should provide the expected AP upper bound under the cutoff for these queries, or report Recall@1000, to contextualize the 0.039 figure.

    Authors: The referee is correct that the gap between nDCG@10 (0.788) and AP (0.039) is extreme and warrants further analysis. The explanation we give in the paper is correct in direction—negation-only queries have very many relevant documents (often the majority of the 100k corpus), so the top-1000 cutoff severely limits AP—but we agree that without quantifying this effect, the reader cannot assess whether 0.039 is reasonable or surprisingly low. We will compute and report the AP upper bound under the top-1000 cutoff for negation-only queries. Concretely: for a query with R relevant documents out of 100k, the maximum achievable AP@1000 is obtained when all top-1000 retrieved documents are relevant, yielding AP = (1000/R) * (average precision over the top 1000 positions) = (1000/R) * (H_1000 / 1000) = H_1000 / R, where H_1000 is the 1000th harmonic number (~7.49). For negation-only queries where R is typically 50,000-90,000, this upper bound is approximately 0.00008-0.00015, which is far below the 0.039 Fallon achieves—suggesting our AP computation may use a different convention or that the relevant document counts are lower than we assumed. We will verify the exact relevant document counts per query, compute the precise AP upper bound, and report Recall@1000 in the revised manuscript to properly contextualize the AP values for both models. revision: yes

Circularity Check

0 steps flagged

No circularity found: constructive proofs are self-contained, self-citations are non-load-bearing context.

full rationale

The paper's theoretical results (Theorems 3.1, 3.2, 3.3, 4.1, 5.1, 5.2) are self-contained constructive or impossibility proofs. The central construction in Lemma 3.1 defines a polynomial p(x) = w - C(x-d)^2 with C = w+1 and proves from scratch that it satisfies the required properties (p(d)=w, p(d')<0 for d'≠d). Theorem 3.1 then constructs sets U and V from input vectors u, v and proves S(U,V) = ⟨u,v⟩ by case analysis — the output equality is derived, not assumed. Theorem 3.3 extends this by decomposing into magnitude/sign, again with a full proof that does not assume its conclusion. Theorem 3.2 is a rank-based impossibility argument. Theorem 4.1 is a standard rank bottleneck argument. Theorem 5.1/5.2 uses Lagrange interpolation (Lemma 5.1, fully proven) to show MaxSim evaluates Weighted Max-OR and positive CNF. No theorem's conclusion appears as its own premise. Self-citations ([15], [16], [17] by overlapping authors) appear only in Related Work for context and are not invoked as premises in any proof. The citation to [3, 4] (Chanpuriya, Musco) notes a 'similar approach' but the paper provides its own complete proof. The experiments (Sections 6-7) compare Signed MaxSim against standard ColBERT on synthetic negation tasks — these are independent empirical evaluations, not fitted-parameter predictions renamed as results. The paper explicitly acknowledges the gap between theoretical constructions and learned neural representations in the conclusion, so it does not overclaim that experiments verify the polynomial structure. No circularity is present.

Axiom & Free-Parameter Ledger

1 free parameters · 3 axioms · 1 invented entities

The axiom ledger is minimal. The theoretical results rely on standard linear algebra and polynomial interpolation. The only free parameter (C = w+1) is a constructive choice, not a fitted constant. The main domain assumption is that neural networks can learn the theoretical embedding structures, which is not proven but is tested empirically. The invented entity (Signed MaxSim) has independent theoretical and empirical support.

free parameters (1)
  • C (polynomial constant) = w+1
    Used in Lemma 3.1 to ensure p(d') < 0 for non-matching indices. This is a constructive choice, not fitted to data.
axioms (3)
  • standard math Standard linear algebra rank bound: rank(UV^T) <= min(rank(U), rank(V)) <= inner dimension
    Used in Theorem 3.2 and Theorem 4.1 to establish impossibility results.
  • standard math Lagrange interpolation exists for distinct points
    Used in Lemma 5.1 to construct OR-clause polynomials.
  • domain assumption Neural network training with contrastive loss can approximate the theoretical embedding constructions
    The experiments assume that a ModernBERT backbone with MLP projections can learn representations that achieve the theoretical capabilities of MaxSim/ Signed MaxSim. This is not proven and is the gap between theory and practice.
invented entities (1)
  • Signed MaxSim similarity (S±) independent evidence
    purpose: Extension of MaxSim that decouples magnitude and sign to enable exact replication of real-valued inner products
    The paper provides a formal definition (Definition 3.4), proves it can replicate arbitrary inner products (Theorem 3.3), and demonstrates empirical improvements on negation queries (Table 2). The falsifiable prediction is that Signed MaxSim outperforms standard MaxSim on tasks requiring negative values.

pith-pipeline@v1.1.0-glm · 23924 in / 2235 out tokens · 461858 ms · 2026-07-08T23:38:37.009111+00:00 · methodology

0 comments
read the original abstract

Late-interaction retrieval models that use the MaxSim similarity function have shown strong empirical performance, often outperforming single-vector dense and sparse retrieval models. Despite these empirical findings, little is known about the theoretical representation power of MaxSim and how it compares to other retrieval approaches. This paper shows by construction that MaxSim similarity can exactly replicate the inner product between any two non-negative k-sparse vectors with possibly infinite dimension, requiring only O(k) representation space. Moreover, there exist similarities that MaxSim can express while standard vector inner products with the same representation space cannot. Leveraging our theoretical framework, we introduce Signed MaxSim which allows late-interaction models to exactly replicate any real-valued inner product, something we prove standard MaxSim is not capable of. We also show that MaxSim can act as an aggregation of soft-OR operations and as an evaluator of logical expressions in positive Conjunctive Normal Form. Our findings show that MaxSim is at least as capable as standard vector inner products for any non-negative vectors and our extension, Signed MaxSim, is as capable for any vectors. Both similarities possess additional capabilities that inner product cannot replicate, marking one of the first theoretical justifications and quantifications of late-interaction methods. Our theoretical findings are supported empirically: on a retrieval task featuring queries with negations, Signed MaxSim improves out-of-domain performance significantly over a standard ColBERT/MaxSim baseline with nDCG@10 increasing from 0.597 to 1.000 under a vocabulary shift and from 0.008 to 0.788 on negation-only queries.

Figures

Figures reproduced from arXiv: 2607.05803 by Cameron Musco, Hamed Zamani, Julian Killingback, Varad Ingale.

Figure 1
Figure 1. Figure 1: Comparison of scoring for the query “Open-source AI tools that do not mention Google.” Both documents satisfy the positive request for an open-source AI tool, but only Doc 2 satisfies the exclusion constraint. (A) Standard MaxSim selects the largest raw similarity for each query token. Although the similarity to the explicit “Google” token can be negative, the maximum operation instead selects the incident… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

35 extracted references · 35 canonical work pages · 7 internal anchors

  1. [1]

    N. Alon, S. Moran, and A. Yehudayoff. Sign rank versus VC dimension. In V . Feldman, A. Rakhlin, and O. Shamir, editors,Proceedings of the 29th Conference on Learning Theory, COLT 2016, New York, USA, June 23-26, 2016, volume 49 ofJMLR Workshop and Conference Proceedings, pages 47–80. JMLR.org,

    work page 2016

  2. [2]

    URLhttp://proceedings.mlr.press/v49/alon16.html

    work page

  3. [3]

    Chaffin, L

    A. Chaffin, L. Arnaboldi, A. Chatelain, and F. Krzakala. Colbert-zero: To pre-train or not to pre-train colbert models, 2026. URLhttps://arxiv.org/abs/2602.16609. 18

    work page arXiv 2026

  4. [4]

    Chanpuriya, C

    S. Chanpuriya, C. Musco, K. Sotiropoulos, and C. E. Tsourakakis. Node embeddings and ex- act low-rank representations of complex networks. In H. Larochelle, M. Ranzato, R. Had- sell, M. Balcan, and H. Lin, editors,Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, Decem- ber...

    work page 2020

  5. [5]

    Chanpuriya, R

    S. Chanpuriya, R. A. Rossi, A. B. Rao, T. Mai, N. Lipka, Z. Song, and C. Musco. Exact representation of sparse networks with symmetric nonnegative embeddings. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors,Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023,...

    work page 2023

  6. [7]

    Formal, B

    T. Formal, B. Piwowarski, and S. Clinchant. Splade: Sparse lexical and expansion model for first stage ranking. InProceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’21, page 2288–2292, New York, NY , USA, 2021. Association for Computing Machinery. ISBN 9781450380379. doi: 10.1145/3404835...

    work page doi:10.1145/3404835.3463098 2021

  7. [8]

    Gaussian Error Linear Units (GELUs)

    D. Hendrycks and K. Gimpel. Gaussian error linear units (gelus).arXiv preprint arXiv:1606.08415, 2016

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  8. [9]

    Hofstätter, O

    S. Hofstätter, O. Khattab, S. Althammer, M. Sertkan, and A. Hanbury. Introducing neural bag of whole- words with colberter: Contextualized late interactions using enhanced reduction. In M. A. Hasan and L. Xiong, editors,Proceedings of the 31st ACM International Conference on Information & Knowledge Management, Atlanta, GA, USA, October 17-21, 2022, pages ...

    work page doi:10.1145/3511808 2022

  9. [10]

    LEMUR: Learned Multi-Vector Retrieval

    E. Jääsaari, V . Hyvönen, and T. Roos. LEMUR: learned multi-vector retrieval.CoRR, abs/2601.21853,

    work page internal anchor Pith review Pith/arXiv arXiv

  10. [12]

    Järvelin and J

    K. Järvelin and J. Kekäläinen. Cumulated gain-based evaluation of IR techniques.ACM Trans. Inf. Syst., 20(4):422–446, 2002. doi: 10.1145/582415.582418. URL http://doi.acm.org/10.1145/582415. 582418

    work page doi:10.1145/582415.582418 2002

  11. [13]

    R. Jayaram. Multi-vector embeddings are provably more expressive than single vector embeddings, 2026. URLhttps://arxiv.org/abs/2606.23475

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  12. [14]

    Z. Ji, H. Jain, A. Veit, S. J. Reddi, S. Jayasumana, A. S. Rawat, A. K. Menon, F. X. Yu, and S. Kumar. Efficient document ranking with learnable late interactions.CoRR, abs/2406.17968, 2024. doi: 10.48550/ ARXIV .2406.17968. URLhttps://doi.org/10.48550/arXiv.2406.17968

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2406.17968 2024

  13. [15]

    Compressing Pre-trained Language Models by Matrix Decomposition

    V . Karpukhin, B. Oguz, S. Min, P. Lewis, L. Wu, S. Edunov, D. Chen, and W. Yih. Dense passage retrieval for open-domain question answering. In B. Webber, T. Cohn, Y . He, and Y . Liu, editors,Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 6769–6781. Association for C...

    work page doi:10.18653/v1/2020 2020

  14. [16]

    Khattab and M

    O. Khattab and M. Zaharia. Colbert: Efficient and effective passage search via contextualized late interaction over BERT. In J. X. Huang, Y . Chang, X. Cheng, J. Kamps, V . Murdock, J. Wen, and Y . Liu, editors,Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, SIGIR 2020, Virtual Event, China,...

    work page doi:10.1145/3397271.3401075 2020

  15. [17]

    Killingback, H

    J. Killingback, H. Zeng, and H. Zamani. Hypencoder: Hypernetworks for information retrieval. In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’25, pages 2372–2383, New York, NY , USA, 2025. Association for Computing Machinery. ISBN 9798400715921. doi: 10.1145/3726302.3729983. URL htt...

    work page doi:10.1145/3726302.3729983 2025

  16. [18]

    A Unified Model and Document Representation for On-Device Retrieval-Augmented Generation

    J. Killingback, O. Meshi, H. Li, H. Zamani, and M. Karimzadehgan. A unified model and document representation for on-device retrieval-augmented generation, 2026. URL https://arxiv.org/abs/ 2604.14403. 19

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  17. [19]

    Killingback, M

    J. Killingback, M. Rafiee, M. Manas, and H. Zamani. Scaling laws for embedding dimension in information retrieval, 2026. URLhttps://arxiv.org/abs/2602.05062

    work page arXiv 2026

  18. [20]

    Loshchilov and F

    I. Loshchilov and F. Hutter. Decoupled weight decay regularization. InInternational Conference on Learning Representations, 2017. URLhttps://api.semanticscholar.org/CorpusID:53592270

    work page 2017

  19. [21]

    Y . Luan, J. Eisenstein, K. Toutanova, and M. Collins. Sparse, dense, and attentional representations for text retrieval.Transactions of the Association for Computational Linguistics, 9:329–345, 2021. doi: 10.1162/TACL_A_00369. URLhttps://doi.org/10.1162/tacl_a_00369

    work page doi:10.1162/tacl_a_00369 2021

  20. [22]

    MacAvaney, A

    S. MacAvaney, A. Mallia, and N. Tonellotto. Efficient constant-space multi-vector retrieval. In C. Hauff, C. Macdonald, D. Jannach, G. Kazai, F. M. Nardini, F. Pinelli, F. Silvestri, and N. Tonellotto, editors, Advances in Information Retrieval - 47th European Conference on Information Retrieval, ECIR 2025, Lucca, Italy, April 6-10, 2025, Proceedings, Par...

    work page doi:10.1007/978-3-031-88714-7 2025

  21. [23]

    Menon, S

    A. Menon, S. Jayasumana, A. S. Rawat, S. Kim, S. Reddi, and S. Kumar. In defense of dual-encoders for neural ranking. In K. Chaudhuri, S. Jegelka, L. Song, C. Szepesvari, G. Niu, and S. Sabato, editors, Proceedings of the 39th International Conference on Machine Learning, volume 162 ofProceedings of Machine Learning Research, pages 15376–15400. PMLR, 17–2...

    work page 2022

  22. [24]

    Paszke et al

    A. Paszke et al. Pytorch: An imperative style, high-performance deep learning library. InAdvances in Neural Information Processing Systems 32, pages 8024–8035, 2019. URL https://proceedings. neurips.cc/paper/2019/hash/bdbca288fee7f92f2bfa9f7012727740-Abstract.html

    work page 2019

  23. [26]

    S. E. Robertson and S. Walker. Some simple effective approximations to the 2-poisson model for probabilistic weighted retrieval. InProceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’94, page 232–241, Berlin, Heidelberg,

    work page

  24. [27]

    ISBN 038719889X

    Springer-Verlag. ISBN 038719889X

    work page

  25. [28]

    A. S, M. Agarwal, A. Garg, N. Kayal, and K. Shiragur. On strengths and limitations of single-vector embeddings, 2026. URLhttps://arxiv.org/abs/2603.29519

    work page arXiv 2026

  26. [29]

    Santhanam, O

    K. Santhanam, O. Khattab, C. Potts, and M. Zaharia. PLAID: an efficient engine for late interaction retrieval. In M. A. Hasan and L. Xiong, editors,Proceedings of the 31st ACM International Conference on Information & Knowledge Management, Atlanta, GA, USA, October 17-21, 2022, pages 1747–1756. ACM,

    work page 2022

  27. [30]

    URLhttps://doi.org/10.1145/3511808.3557325

    doi: 10.1145/3511808.3557325. URLhttps://doi.org/10.1145/3511808.3557325

    work page doi:10.1145/3511808.3557325

  28. [31]

    Santhanam, O

    K. Santhanam, O. Khattab, J. Saad-Falcon, C. Potts, and M. Zaharia. Colbertv2: Effective and efficient retrieval via lightweight late interaction. In M. Carpuat, M. de Marneffe, and I. V . M. Ruíz, editors,Pro- ceedings of the 2022 Conference of the North American Chapter of the Association for Computational Lin- guistics: Human Language Technologies, NAA...

    work page doi:10.18653/v1/2022.naacl-main.272 2022

  29. [32]

    Sourty, A

    R. Sourty, A. Chaffin, O. Weller, P. R. Moura Junior, and A. Chatelain. Denseon with the lateon: Open state-of-the-art single and multi-vector models. https://huggingface.co/blog/lightonai/ denseon-lateon, 2026

    work page 2026

  30. [33]

    CRISP: Clustering Multi-Vector Representations for Denoising and Pruning

    J. Veneroso, R. Jayaram, J. Rao, G. H. Ábrego, M. Hadian, and D. Cer. CRISP: clustering multi-vector representations for denoising and pruning.CoRR, abs/2505.11471, 2025. doi: 10.48550/ARXIV .2505. 11471. URLhttps://doi.org/10.48550/arXiv.2505.11471

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv 2025

  31. [34]

    Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference

    B. Warner, A. Chaffin, B. Clavié, O. Weller, O. Hallström, S. Taghadouini, A. Gallagher, R. Biswas, F. Ladhak, T. Aarsen, N. Cooper, G. Adams, J. Howard, and I. Poli. Smarter, better, faster, longer: A modern bidirectional encoder for fast, memory efficient, and long context finetuning and inference, 2024. URLhttps://arxiv.org/abs/2412.13663

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  32. [35]

    Weller, M

    O. Weller, M. Boratko, I. Naim, and J. Lee. On the theoretical limitations of embedding-based retrieval. CoRR, abs/2508.21038, 2025. doi: 10.48550/arXiv.2508.21038. URL https://doi.org/10.48550/ arXiv.2508.21038. 20

    work page doi:10.48550/arxiv.2508.21038 2025

  33. [36]

    T. Wolf, L. Debut, V . Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, J. Davison, S. Shleifer, P. von Platen, C. Ma, Y . Jernite, J. Plu, C. Xu, T. Le Scao, S. Gugger, M. Drame, Q. Lhoest, and A. Rush. Transformers: State-of-the-art natural language processing. In Q. Liu and D. Schlangen, editors,Proceedings of the 202...

    work page 2020

  34. [37]

    Transformers: State-of-the-Art Natural Language Processing

    Association for Computational Linguistics. doi: 10.18653/v1/2020.emnlp-demos.6. URL https://aclanthology.org/2020.emnlp-demos.6/

    work page doi:10.18653/v1/2020.emnlp-demos.6 2020

  35. [38]

    Zamani, M

    H. Zamani, M. Dehghani, W. B. Croft, E. Learned-Miller, and J. Kamps. From neural re-ranking to neural ranking: Learning a sparse representation for inverted indexing. InProceedings of the 27th ACM International Conference on Information and Knowledge Management, CIKM ’18, page 497–506, New York, NY , USA, 2018. Association for Computing Machinery. ISBN 9...

    work page doi:10.1145/3269206.3271800 2018