REVIEW 3 minor 2 cited by
Is Dimensionality a Barrier for Retrieval Models?
T0 review · 0 major / 3 minor · reviewed 2026-05-25 · grok-4.3
Pith's one-line read The infinite-dimension maximal margin for any relevance matrix A is nearly achieved already in dimension O(m^{-2} log n).
desk verdict The paper tightens the dimension needed for near-optimal margins in the classical inner-product retrieval model and pins down the exact requirement for the all-k-sparse case. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The margin m^rd(d, A) is the largest value m such that there exist unit-norm query vectors U_j and document vectors V_i satisfying signed inner-product separation exactly according to the entries of A.
What would settle it
An explicit matrix A for which every embedding achieving margin within a constant factor of m^rd(+∞, A) requires dimension larger than C m^{-2} log n for any fixed C would falsify the main upper bound.
Extended reading notes
Core claim
For any relevance matrix A the quantity m^rd(+∞, A) can be nearly recovered by unit-norm embeddings whose dimension is only O(m^rd(+∞, A)^{-2} log n). In the special case where A contains every possible k-sparse row exactly once, dimension O(k log(n/k)) is necessary and sufficient to attain the optimal margin Θ(k^{-1/2}).
Load-bearing premise
The retrieval model requires exact signed inner-product separation with a uniform margin between unit-norm vectors.
Editorial extensions
If this is right
- For all-k-sparse queries the dimension O(k log(n/k)) is both necessary and sufficient to reach margin Θ(k^{-1/2}).
- Modern embedding sizes around 1000 already suffice for near-optimal margins on data sets with trillions of documents under the inner-product model.
- Explicit constructions exist that produce large margins even when dimension is o(k log(n/k)).
- Sigmoid loss yields larger empirical margins than InfoNCE on the tested synthetic instances.
Reading between the lines
- If margin governs robustness and compositional generalization, then training objectives that directly target signed inner-product separation should remain effective at moderate dimensions.
- The dimension lower bound for the k-sparse case may be used to derive concrete sample-complexity requirements for learning retrieval representations.
- The communication-complexity origin of the model suggests analogous dimension bounds could apply to other separation tasks such as nearest-neighbor search under Hamming or Euclidean distance.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies maximal-margin retrieval embeddings for a 0-1 relevance matrix A, where unit-norm query and document vectors must satisfy signed inner-product separation of margin m. It proves that the infinite-dimensional optimum m^rd(+∞, A) can be recovered up to (1-o(1)) factors in dimension d = O(m^{-2} log n), improving the dimension bound of [BDES02]. For the all-k-sparse query matrix it supplies a matching lower bound showing that d = O(k log(n/k)) is necessary and sufficient to achieve the optimal margin Θ(k^{-1/2}). Additional constructions are given for d = o(k log(n/k)) and an empirical comparison of InfoNCE versus sigmoid loss is reported.
Significance. If the stated theorems hold, the work supplies a tight, parameter-free characterization of the dimension needed for optimal-margin retrieval embeddings and resolves the open question posed in [WBNL26]. The dimension-reduction result is obtained via an explicit construction together with a matching lower bound for the sparse-query case; the empirical section provides a concrete, falsifiable comparison between two standard losses. These elements together give a self-contained theoretical and practical account of why modest embedding dimensions suffice at scale.
minor comments (3)
- [Abstract] Abstract: the statement that the sigmoid loss shows 'a clear advantage' is not accompanied by any numerical values or statistical test; the claim should be supported by the specific margins or loss curves reported in the experimental section.
- [Abstract] The notation m^rd(d, A) is introduced without an explicit reference to the section containing its formal definition; a forward pointer would improve readability.
- [§1] The improvement over [BDES02] is stated only in the abstract; the introduction or related-work section should contain a one-sentence comparison of the new dimension exponent versus the prior bound.
Simulated Author's Rebuttal
We thank the referee for the positive assessment of our manuscript, the accurate summary of our results on dimension bounds for maximal-margin retrieval embeddings, and the recommendation for minor revision. The report correctly identifies the resolution of the open question from [WBNL26] via matching upper and lower bounds. No specific major comments were raised in the report.
Circularity Check
No significant circularity; mathematical theorems are self-contained
full rationale
The paper's core claims consist of mathematical theorems establishing dimension bounds for achieving near-optimal margins m^rd(∞, A) via constructions that improve on the cited result of [BDES02], together with a matching lower bound for the k-sparse case. These rest on the classical inner-product retrieval model from [PS86] and [WBNL26] but do not reduce any claimed margin or dimension to a fitted quantity, self-definition, or load-bearing self-citation chain. The derivation is independent of the present paper's own inputs and is externally verifiable as a margin-preserving embedding result.
Assumptions & free parameters
assumptions (1)
- domain assumption Unit-norm query and document embeddings exist that realize the optimal margin m^rd(+∞, A) for any fixed relevance matrix A
Cite this review
Pith. "Pith review of Is Dimensionality a Barrier for Retrieval Models?." pith.science (2026). https://pith.science/paper/GQOTS5VH
@misc{pith2026260523556,
author = {Pith},
title = {Pith review of: Is Dimensionality a Barrier for Retrieval Models?},
year = {2026},
howpublished = {\url{https://pith.science/paper/GQOTS5VH}},
note = {Machine review of arXiv:2605.23556}
}
abstract
Why does the low dimensionality of representations, typically $d\approx 1000$, not prevent modern embedding-based retrieval models from scaling to billions, or even trillions, of data points? To answer this question, we study maximal-margin embeddings in the following retrieval model, classically studied in communication complexity [PS86] and more recently in embedding-based retrieval [WBNL26]. Let $A\in \{0,1\}^{N\times n}$ be a matrix indicating whether each of $N$ queries is relevant to each of $n$ documents. We are interested in the largest margin $m>0,$ denoted by $\mathsf{m}^{\mathsf{rd}}(d, A),$ for which there exist unit norm embeddings of the queries and documents $\{U_j\}_{j = 1}^N, \{V_i\}_{i = 1}^n$ with the following property. $\langle U_j, V_i\rangle \ge m$ whenever $A_{ji} = 1$ and $\langle U_j, V_i\rangle \le -m$ otherwise. A large margin is a key proxy for representation quality: it controls both robustness to perturbations and compositional generalization across queries. Our main theorem establishes that the best possible margin without a restriction on the dimension, $\mathsf{m}^{\mathsf{rd}}(+\infty, A),$ can be nearly achieved in dimension $d = O(\mathsf{m}^{\mathsf{rd}}(+\infty, A)^{-2}\log n)$ which improves a theorem of [BDES02]. Together with a matching lower bound in Theorem 1.5, we conclude that when $A\in \{0,1\}^{\binom{n}{k}\times n}$ is the matrix containing all possible $k$-sparse rows once, dimension $d = O(k\log (n/k))$ is necessary and sufficient for the maximal possible margin $\mathsf{m}^{\mathsf{rd}}(+\infty, A) = \Theta(k^{-1/2})$ in this setting. This fully resolves the setup of [WBNL26]. We also give several constructions for large margins when $d = o(k\log (n/k)).$ Finally, we empirically test the InfoNCE and sigmoid losses for producing large margin embeddings and demonstrate a clear advantage of the sigmoid loss.
Figures
Lean theorems connected to this paper
-
IndisputableMonolith/Cost/FunctionalEquation.leanwashburn_uniqueness_aczel unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
Our main theorem establishes that the best possible margin without a restriction on the dimension, m^rd(+∞, A), can be nearly achieved in dimension d = O(m^rd(+∞, A)^{-2} log n) which improves a theorem of [BDES02].
-
IndisputableMonolith/Foundation/DimensionForcing.leanalexander_duality_circle_linking unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
Our proofs are based on connections between this problem and the literature on compressed sensing and the restricted isometry property.
What do these tags mean?
- matches
- The paper's claim is directly supported by a theorem in the formal canon.
- supports
- The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
- extends
- The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
- uses
- The paper appears to rely on the theorem as machinery.
- contradicts
- The paper's claim conflicts with a theorem or certificate in the canon.
- unclear
- Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.
Forward citations
Cited by 2 Pith papers
-
The Limits of Binding in Dual Encoders
Binding failure in dual encoders is a conditional limit: an exponential depth law for recursive codes, a vanishing contrastive reward for swap negatives, and a tight smoothness ceiling, with deployed models far inside...
-
Non-negative Elastic Net Decoding for Information Retrieval
NNN decoding selects documents via non-negative elastic net reconstruction of the query embedding, with a theorem showing it strictly dominates dense retrieval on correlated corpora and experiments showing gains over ...
Reference graph
Works this paper leans on
-
[1]
Combinatorics, Probability and Computing , volume=
Learning complexity vs communication complexity , author=. Combinatorics, Probability and Computing , volume=. 2009 , publisher=
work page 2009
-
[2]
International conference on machine learning , pages=
A simple framework for contrastive learning of visual representations , author=. International conference on machine learning , pages=. 2020 , organization=
work page 2020
-
[3]
Proceedings of the IEEE/CVF international conference on computer vision , pages=
With a little help from my friends: Nearest-neighbor contrastive learning of visual representations , author=. Proceedings of the IEEE/CVF international conference on computer vision , pages=
-
[4]
An algorithmic theory of learning: Robust concepts and random projection , author=. Machine Learning , volume=. 2006 , publisher=
work page 2006
-
[5]
Proceedings of the Symposium on the Mathematical Theory of Automata , volume=
On convergence proofs on perceptrons , author=. Proceedings of the Symposium on the Mathematical Theory of Automata , volume=. 1962 , organization=
work page 1962
-
[6]
Kernels as features: On kernels, margins, and low-dimensional mappings , author=. Machine Learning , volume=. 2006 , publisher=
work page 2006
-
[7]
IEEE Transactions on Information Theory , volume=
Compressed sensing , author=. IEEE Transactions on Information Theory , volume=. 2006 , publisher=
work page 2006
-
[8]
IEEE Transactions on Big Data , volume=
Billion-scale similarity search with GPUs , author=. IEEE Transactions on Big Data , volume=. 2019 , publisher=
work page 2019
Show all 182 references
-
[9]
IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=
Product Quantization for Nearest Neighbor Search , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=. 2011 , publisher=
2011
-
[10]
IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=
Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=. 2020 , publisher=
2020
-
[11]
arXiv preprint arXiv:2508.10104 , year=
Dinov3 , author=. arXiv preprint arXiv:2508.10104 , year=
-
[12]
Laurent and P
B. Laurent and P. Massart , title =. The Annals of Statistics , number =. 2000 , doi =
2000
-
[13]
Lower bounds in communication complexity and learning theory via analytic methods , author=
-
[14]
Geometrical realization of set systems and probabilistic communication complexity , booktitle =
Noga Alon and Peter Frankl and Vojt. Geometrical realization of set systems and probabilistic communication complexity , booktitle =. 1985 , publisher =
1985
-
[15]
Estimating the Optimal Margins of Embeddings in Euclidean Half Spaces , journal =
J. Estimating the Optimal Margins of Embeddings in Euclidean Half Spaces , journal =. 2003 , doi =
2003
-
[16]
Machine Learning , volume =
Corinna Cortes and Vladimir Vapnik , title =. Machine Learning , volume =. 1995 , doi =
1995
-
[17]
Bartlett , title =
Peter L. Bartlett , title =. IEEE Transactions on Information Theory , volume =
-
[18]
2018 , issue_date =
Barman, Siddharth , title =. 2018 , issue_date =. doi:10.1137/15M1050574 , journal =
2018 doi
-
[19]
Conference on Learning Theory , pages=
Approximate is good enough: Probabilistic variants of dimensional and margin complexity , author=. Conference on Learning Theory , pages=. 2020 , organization=
2020
-
[20]
Schapire and Yoav Freund and Peter Bartlett and Wee Sun Lee , title =
Robert E. Schapire and Yoav Freund and Peter Bartlett and Wee Sun Lee , title =. The Annals of Statistics , volume =. 1998 , doi =
1998
-
[21]
, title =
Freund, Yoav and Schapire, Robert E. , title =. Machine Learning , volume =. 1999 , doi =
1999
-
[22]
Annals of the Institute of Statistical Mathematics , volume=
Some geometric applications of the beta distribution , author=. Annals of the Institute of Statistical Mathematics , volume=. 1990 , publisher=
1990
-
[23]
Bartlett and Dylan J
Peter L. Bartlett and Dylan J. Foster and Matus J. Telgarsky , title =. Advances in Neural Information Processing Systems 30 , editor =
-
[24]
Contemporary mathematics , volume=
Extensions of Lipschitz mappings into a Hilbert space , author=. Contemporary mathematics , volume=
-
[25]
Journal of Machine Learning Research , volume=
Limitations of learning via embeddings in Euclidean half spaces , author=. Journal of Machine Learning Research , volume=
-
[26]
Conference on Learning Theory , pages=
Sign rank versus VC dimension , author=. Conference on Learning Theory , pages=. 2016 , organization=
2016
-
[27]
1996 , isbn =
Kushilevitz, Eyal and Nisan, Noam , title =. 1996 , isbn =
1996
-
[28]
Journal of Machine Learning Research , volume =
Daniel Soudry and Elad Hoffer and Mor Shpigel Nacson and Suriya Gunasekar and Nathan Srebro , title =. Journal of Machine Learning Research , volume =. 2018 , url =
2018
-
[29]
Robust Large Margin Deep Neural Networks , journal =
Jure Sokoli. Robust Large Margin Deep Neural Networks , journal =. 2017 , doi =
2017
-
[30]
Proceedings of the 55th Annual ACM Symposium on Theory of Computing , pages=
Local and global expansion in random geometric graphs , author=. Proceedings of the 55th Annual ACM Symposium on Theory of Computing , pages=
-
[31]
arXiv preprint arXiv:2507.12926 , year=
An exponential improvement for Ramsey lower bounds , author=. arXiv preprint arXiv:2507.12926 , year=
-
[32]
Communications on Pure and Applied Mathematics , volume=
Explicit Construction of RIP Matrices Is Ramsey-Hard , author=. Communications on Pure and Applied Mathematics , volume=. 2020 , publisher=
2020
-
[33]
Elsayed and Dilip Krishnan and Hossein Mobahi and Kevin Regan and Samy Bengio , title =
Gamaleldin F. Elsayed and Dilip Krishnan and Hossein Mobahi and Kevin Regan and Samy Bengio , title =. Advances in Neural Information Processing Systems 31 , editor =
-
[34]
Machine Learning , volume =
Maria-Florina Balcan and Avrim Blum and Nathan Srebro , title =. Machine Learning , volume =. 2008 , doi =
2008
-
[35]
Weinberger and Lawrence K
Kilian Q. Weinberger and Lawrence K. Saul , title =. Journal of Machine Learning Research , volume =. 2009 , url =
2009
-
[36]
2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06) , volume =
Raia Hadsell and Sumit Chopra and Yann LeCun , title =. 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06) , volume =. 2006 , doi =
2006
-
[37]
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages =
Florian Schroff and Dmitry Kalenichenko and James Philbin , title =. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages =. 2015 , doi =
2015
-
[38]
Proceedings of the 33rd International Conference on Machine Learning , series =
Weiyang Liu and Yandong Wen and Zhiding Yu and Meng Yang , title =. Proceedings of the 33rd International Conference on Machine Learning , series =
-
[39]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =
Jiankang Deng and Jia Guo and Niannan Xue and Stefanos Zafeiriou , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =
-
[40]
arXiv preprint arXiv:2408.00995 , year =
Kiril Bangachev and Guy Bresler , title =. arXiv preprint arXiv:2408.00995 , year =
-
[41]
J. H. Conway and N. J. A. Sloane , title =. 1999 , edition =
1999
-
[42]
Emmanuel J. Cand. Decoding by Linear Programming , journal =
-
[43]
2013 , publisher =
Simon Foucart and Holger Rauhut , title =. 2013 , publisher =
2013
-
[44]
Fiduccia and Edward R
Charles M. Fiduccia and Edward R. Scheinerman and Ann N. Trenk and Jennifer S. Zito , title =. Discrete Mathematics , volume =
-
[45]
A Linear Lower Bound on the Unbounded Error Probabilistic Communication Complexity , journal =
J. A Linear Lower Bound on the Unbounded Error Probabilistic Communication Complexity , journal =
-
[46]
Dense Passage Retrieval for Open-Domain Question Answering , booktitle =
Vladimir Karpukhin and Barlas O. Dense Passage Retrieval for Open-Domain Question Answering , booktitle =
-
[47]
Journal of Functional Analysis , volume =
Bo'az Klartag and Shahar Mendelson , title =. Journal of Functional Analysis , volume =
-
[48]
arXiv preprint arXiv:1609.05034 , year =
Stefan Neumann and Rainer Gemulla and Pauli Miettinen , title =. arXiv preprint arXiv:1609.05034 , year =
-
[50]
Transactions on Machine Learning Research , year =
Maxime Oquab and others , title =. Transactions on Machine Learning Research , year =
-
[51]
Proceedings of the 41st International Conference on Machine Learning (ICML) , pages =
Kiho Park and Yo Joong Choe and Victor Veitch , title =. Proceedings of the 41st International Conference on Machine Learning (ICML) , pages =
-
[52]
Journal of Computer and System Sciences , volume =
Ramamohan Paturi and Janos Simon , title =. Journal of Computer and System Sciences , volume =
-
[53]
Proceedings of the 38th International Conference on Machine Learning (ICML) , pages =
Alec Radford and Jong Wook Kim and Chris Hallacy and Aditya Ramesh and Gabriel Goh and Sandhini Agarwal and Girish Sastry and Amanda Askell and Pamela Mishkin and Jack Clark and Gretchen Krueger and Ilya Sutskever , title =. Proceedings of the 38th International Conference on ...
-
[54]
Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP-IJCNLP) , pages =
Nils Reimers and Iryna Gurevych , title =. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP-IJCNLP) , pages =
2019
-
[55]
Sherstov , title =
Alexander A. Sherstov , title =. Computational Complexity , volume =
-
[56]
Constructive approximation , volume=
A simple proof of the restricted isometry property for random matrices , author=. Constructive approximation , volume=. 2008 , publisher=
2008
-
[57]
arXiv preprint arXiv:1807.03748 , year=
Representation learning with contrastive predictive coding , author=. arXiv preprint arXiv:1807.03748 , year=
-
[58]
Roman Vershynin , title =
-
[59]
arXiv preprint arXiv:2601.20844 , year =
Zihao Wang and Hang Yin and Lihui Liu and Hanghang Tong and Yangqiu Song and Ginny Wong and Simon See , title =. arXiv preprint arXiv:2601.20844 , year =
-
[60]
Young and Edward R
Stephen J. Young and Edward R. Scheinerman , title =. Lecture Notes in Computer Science , volume =
-
[61]
arXiv preprint arXiv:2303.15343 , year =
Xiaohua Zhai and Basil Mustafa and Alexander Kolesnikov and Lucas Beyer , title =. arXiv preprint arXiv:2303.15343 , year =
-
[62]
Proceedings of the 57th annual meeting of the association for computational linguistics , pages=
Latent retrieval for weakly supervised open domain question answering , author=. Proceedings of the 57th annual meeting of the association for computational linguistics , pages=
-
[63]
Proceedings of the 37th International Conference on Machine Learning , articleno =
Wang, Tongzhou and Isola, Phillip , title =. Proceedings of the 37th International Conference on Machine Learning , articleno =. 2020 , publisher =
2020
-
[64]
Journal of Approximation Theory , volume=
Optimal recovery and n-widths for convex classes of functions , author=. Journal of Approximation Theory , volume=. 1995 , publisher=
1995
-
[65]
Xi Chen and Xiao Wang and Soravit Changpinyo and AJ Piergiovanni and Piotr Padlewski and Daniel Salz and Sebastian Goodman and Adam Grycner and Basil Mustafa and Lucas Beyer and Alexander Kolesnikov and Joan Puigcerver and Nan Ding and Keran Rong and Hassan Akbari and Gaurav M...
2023
-
[66]
Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles
Noroozi, Mehdi and Favaro, Paolo. Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles. Computer Vision -- ECCV 2016. 2016
2016
-
[67]
Wyner, A. D. , title =. Bell System Technical Journal , volume =. doi:https://doi.org/10.1002/j.1538-7305.1968.tb00062.x , year =
1968 doi
-
[68]
Proceedings of the Glasgow Mathematical Association , author=
The Closest Packing of Spherical Caps in n Dimensions , volume=. Proceedings of the Glasgow Mathematical Association , author=. 1955 , pages=. doi:10.1017/S2040618500033219 , number=
1955 doi
-
[69]
, title =
Shannon, Claude E. , title =. Bell System Technical Journal , volume =. doi:https://doi.org/10.1002/j.1538-7305.1959.tb03905.x , year =
1959 doi
-
[70]
On the origin of number and arrangement of the places of exit on the surface of pollen-grains
Tammes, Pieter Merkus Lambertus. On the origin of number and arrangement of the places of exit on the surface of pollen-grains. 1930
1930
-
[71]
Sphere packings, lattices and groups , volume =
Conway, J.H and Sloane, N.J.A and Bannai, E and Borcherds, R.E and Leech, J and Norton, S.P and Odlyzko, A.M and Parker, R.A and Queen, L and Venkov, B.B , edition =. Sphere packings, lattices and groups , volume =
-
[72]
Zhai, Xiaohua and Mustafa, Basil and Kolesnikov, Alexander and Beyer, Lucas , year =
-
[73]
A Theoretical Analysis of Contrastive Unsupervised Representation Learning , booktitle =
Nikunj Saunshi and Orestis Plevrakis and Sanjeev Arora and Mikhail Khodak and Hrishikesh Khandeparkar , editor =. A Theoretical Analysis of Contrastive Unsupervised Representation Learning , booktitle =. 2019 , url =
2019
-
[74]
CoRR , volume =
Anna Van Elst and Debarghya Ghoshdastidar , title =. CoRR , volume =. 2024 , url =. doi:10.48550/ARXIV.2412.03486 , eprinttype =. 2412.03486 , timestamp =
2024 doi
-
[75]
Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages =
Srinivasan, Krishna and Raman, Karthik and Chen, Jiecao and Bendersky, Michael and Najork, Marc , title =. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages =. 2021 , isbn =. doi:10.1145/3404835.3463257 , abstract =
2021 doi
-
[76]
International Conference on Learning Representations , year=
Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval , author=. International Conference on Learning Representations , year=
-
[77]
2019 , eprint=
Representation Learning with Contrastive Predictive Coding , author=. 2019 , eprint=
2019
-
[78]
Yiwei Lu and Guojun Zhang and Sun Sun and Hongyu Guo and Yaoliang Yu , journal=. \ f\ -. 2023 , url=
2023
-
[79]
2020 , isbn =
Khattab, Omar and Zaharia, Matei , title =. 2020 , isbn =. doi:10.1145/3397271.3401075 , booktitle =
2020 doi
-
[80]
Journal of Computer and System Sciences , volume=
Probabilistic communication complexity , author=. Journal of Computer and System Sciences , volume=. 1986 , publisher=
1986
-
[81]
Random Structures & Algorithms , volume=
Testing for high-dimensional geometry in random graphs , author=. Random Structures & Algorithms , volume=. 2016 , publisher=
2016
-
[82]
2021 , eprint=
De Finetti-Style Results for Wishart Matrices: Combinatorial Structure and Phase Transitions , author=. 2021 , eprint=
2021
-
[83]
, booktitle=
Yao, Andrew C. , booktitle=. Lower bounds by probabilistic arguments , year=
-
[84]
Complexity classes in communication complexity theory , year=
Babai, Laszlo and Frankl, Peter and Simon, Janos , booktitle=. Complexity classes in communication complexity theory , year=
-
[85]
1979 , isbn =
Yao, Andrew Chi-Chih , title =. 1979 , isbn =. doi:10.1145/800135.804414 , booktitle =
1979 doi
-
[86]
, journal=
Khanna, Saurabh and Murthy, Chandra R. , journal=. On the Restricted Isometry of the Columnwise Khatri–Rao Product , year=
-
[87]
Solutions to some functional equations and their applications to characterization of probability distributions , author=. Sankhy. 1968 , publisher=
1968
-
[88]
arXiv preprint arXiv:1905.09245 , year=
On the restricted isometry property of centered self Khatri-Rao products , author=. arXiv preprint arXiv:1905.09245 , year=
1905 arXiv
-
[89]
International Conference on Learning Representations , year=
Learning deep representations by mutual information estimation and maximization , author=. International Conference on Learning Representations , year=
-
[90]
2025 , eprint=
A Statistical Theory of Contrastive Learning via Approximate Sufficient Statistics , author=. 2025 , eprint=
2025
-
[91]
2025 , eprint=
A Statistical Theory of Contrastive Pre-training and Multimodal Generative AI , author=. 2025 , eprint=
2025
-
[92]
2024 , eprint=
CROME: Cross-Modal Adapters for Efficient Multimodal LLM , author=. 2024 , eprint=
2024
-
[93]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =
Yang, Lingxiao and Zhang, Ru-Yuan and Wang, Yanchen and Xie, Xiaohua , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =. 2024 , pages =
2024
-
[94]
The Twelfth International Conference on Learning Representations , year=
UniAdapter: Unified Parameter-Efficient Transfer Learning for Cross-modal Modeling , author=. The Twelfth International Conference on Learning Representations , year=
-
[95]
2022 , url=
Elan Rosenfeld and Preetum Nakkiran and Hadi Pouransari and Oncel Tuzel and Fartash Faghri , booktitle=. 2022 , url=
2022
-
[96]
2021 , isbn =
Macdonald, Craig and Tonellotto, Nicola , title =. 2021 , isbn =. doi:10.1145/3459637.3482156 , booktitle =
2021 doi
-
[97]
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts , year=
Changpinyo, Soravit and Sharma, Piyush and Ding, Nan and Soricut, Radu , booktitle=. Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts , year=
-
[98]
Sigmoid Loss for Language Image Pre-Training , year=
Zhai, Xiaohua and Mustafa, Basil and Kolesnikov, Alexander and Beyer, Lucas , booktitle=. Sigmoid Loss for Language Image Pre-Training , year=
-
[99]
Proceedings of the 40th International Conference on Machine Learning , articleno =
Li, Junnan and Li, Dongxu and Savarese, Silvio and Hoi, Steven , title =. Proceedings of the 40th International Conference on Machine Learning , articleno =. 2023 , publisher =
2023
-
[100]
Kingma and Jimmy Ba , editor =
Diederik P. Kingma and Jimmy Ba , editor =. Adam:. 3rd International Conference on Learning Representations,. 2015 , url =
2015
-
[101]
Proceedings of the 37th International Conference on Machine Learning , pages =
A Simple Framework for Contrastive Learning of Visual Representations , author =. Proceedings of the 37th International Conference on Machine Learning , pages =. 2020 , editor =
2020
-
[102]
Discriminative Unsupervised Feature Learning with Convolutional Neural Networks , url =
Dosovitskiy, Alexey and Springenberg, Jost Tobias and Riedmiller, Martin and Brox, Thomas , booktitle =. Discriminative Unsupervised Feature Learning with Convolutional Neural Networks , url =
-
[103]
Neural collapse under cross-entropy loss , journal =
Jianfeng Lu and Stefan Steinerberger , keywords =. Neural collapse under cross-entropy loss , journal =. 2022 , note =. doi:https://doi.org/10.1016/j.acha.2021.12.011 , url =
2022 doi
-
[104]
ImageBind One Embedding Space to Bind Them All , year=
Girdhar, Rohit and El-Nouby, Alaaeldin and Liu, Zhuang and Singh, Mannat and Alwala, Kalyan Vasudev and Joulin, Armand and Misra, Ishan , booktitle=. ImageBind One Embedding Space to Bind Them All , year=
-
[105]
arXiv preprint arXiv:2509.20354 , year=
Embeddinggemma: Powerful and lightweight text representations , author=. arXiv preprint arXiv:2509.20354 , year=
-
[106]
International journal of computer vision , volume=
Imagenet large scale visual recognition challenge , author=. International journal of computer vision , volume=. 2015 , publisher=
2015
-
[107]
Emerging Properties in Self-Supervised Vision Transformers , year=
Caron, Mathilde and Touvron, Hugo and Misra, Ishan and Jegou, Hervé and Mairal, Julien and Bojanowski, Piotr and Joulin, Armand , booktitle=. Emerging Properties in Self-Supervised Vision Transformers , year=
-
[108]
Proceedings of The 27th International Conference on Artificial Intelligence and Statistics , pages =
Analysis of Using Sigmoid Loss for Contrastive Learning , author =. Proceedings of The 27th International Conference on Artificial Intelligence and Statistics , pages =. 2024 , editor =
2024
-
[109]
2025 , eprint=
SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features , author=. 2025 , eprint=
2025
-
[110]
Attention is All you Need , url =
Vaswani, Ashish and Shazeer, Noam and Parmar, Niki and Uszkoreit, Jakob and Jones, Llion and Gomez, Aidan N and Kaiser, ukasz and Polosukhin, Illia , booktitle =. Attention is All you Need , url =
-
[111]
Proceedings of the 38th International Conference on Machine Learning , pages =
Learning Transferable Visual Models From Natural Language Supervision , author =. Proceedings of the 38th International Conference on Machine Learning , pages =. 2021 , editor =
2021
-
[112]
International Conference on Learning Representations , year=
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale , author=. International Conference on Learning Representations , year=
- [113]
-
[114]
Scaling Up Vision-Language Pretraining for Image Captioning , year=
Hu, Xiaowei and Gan, Zhe and Wang, Jianfeng and Yang, Zhengyuan and Liu, Zicheng and Lu, Yumao and Wang, Lijuan , booktitle=. Scaling Up Vision-Language Pretraining for Image Captioning , year=
-
[115]
Proceedings of the 38th International Conference on Machine Learning , pages =
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision , author =. Proceedings of the 38th International Conference on Machine Learning , pages =. 2021 , editor =
2021
-
[116]
Proceedings of the 41st International Conference on Machine Learning , articleno =
Park, Kiho and Choe, Yo Joong and Veitch, Victor , title =. Proceedings of the 41st International Conference on Machine Learning , articleno =. 2024 , publisher =
2024
-
[117]
(Hermann) , address =
Minkowski, H. (Hermann) , address =. Geometrie der Zahlen , year =. Geometrie der Zahlen , keywords =
-
[118]
Bedingt konvergente Reihen und konvexe Systeme
Steinitz, Ernst , journal =. Bedingt konvergente Reihen und konvexe Systeme. , url =
-
[119]
Rendiconti del Circolo Matematico di Palermo (1884-1940) , year=
Constantin Carath. Rendiconti del Circolo Matematico di Palermo (1884-1940) , year=
1940
-
[120]
, journal =
Helly, Ed. , journal =. Über Mengen konvexer Körper mit gemeinschaftlichen Punkte. , url =
-
[121]
Excursions into Combinatorial Geometry , year =
Boltyanski, Vladimir and Martini, Horst and Soltan, P.S , address =. Excursions into Combinatorial Geometry , year =
-
[122]
2025 , eprint=
Discriminative and Consistent Representation Distillation , author=. 2025 , eprint=
2025
-
[123]
International Conference on Learning Representations , year=
Contrastive Representation Distillation , author=. International Conference on Learning Representations , year=
-
[124]
Proceedings of the 7th Machine Learning for Healthcare Conference , pages =
Contrastive Learning of Medical Visual Representations from Paired Images and Text , author =. Proceedings of the 7th Machine Learning for Healthcare Conference , pages =. 2022 , editor =
2022
-
[125]
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , month =
Schroff, Florian and Kalenichenko, Dmitry and Philbin, James , title =. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , month =
-
[126]
Momentum Contrast for Unsupervised Visual Representation Learning , year=
He, Kaiming and Fan, Haoqi and Wu, Yuxin and Xie, Saining and Girshick, Ross , booktitle=. Momentum Contrast for Unsupervised Visual Representation Learning , year=
-
[127]
Proceedings of the 37th International Conference on Machine Learning , articleno =
Chen, Ting and Kornblith, Simon and Norouzi, Mohammad and Hinton, Geoffrey , title =. Proceedings of the 37th International Conference on Machine Learning , articleno =. 2020 , publisher =
2020
-
[128]
2017 , isbn =
Wang, Feng and Xiang, Xiang and Cheng, Jian and Yuille, Alan Loddon , title =. 2017 , isbn =. doi:10.1145/3123266.3123359 , booktitle =
2017 doi
-
[129]
SphereFace: Deep Hypersphere Embedding for Face Recognition , year=
Liu, Weiyang and Wen, Yandong and Yu, Zhiding and Li, Ming and Raj, Bhiksha and Song, Le , booktitle=. SphereFace: Deep Hypersphere Embedding for Face Recognition , year=
-
[130]
Parkhi and Andrea Vedaldi and Andrew Zisserman
Omkar M. Parkhi and Andrea Vedaldi and Andrew Zisserman. Deep Face Recognition. British Machine Vision Conference. 2015
2015
-
[131]
Computer Vision – ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXXV , pages =
Zhang, Renrui and Zhang, Wei and Fang, Rongyao and Gao, Peng and Li, Kunchang and Dai, Jifeng and Qiao, Yu and Li, Hongsheng , title =. Computer Vision – ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXXV , pages =. 2022 , isbn =...
2022 doi
-
[132]
Proceedings of the 31st ACM International Conference on Multimedia , pages =
Liu, Hongye and Xie, Xianhai and Gao, Yang and Yu, Zhou , title =. Proceedings of the 31st ACM International Conference on Multimedia , pages =. 2023 , isbn =. doi:10.1145/3581783.3611939 , abstract =
2023 doi
-
[133]
International journal of computer vision , keywords =
CLIP-Adapter: Better Vision-Language Models with Feature Adapters , volume =. International journal of computer vision , keywords =. 2024 , author =
2024
-
[134]
2025 , eprint=
Gemma 3 Technical Report , author=. 2025 , eprint=
2025
-
[135]
Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1) , year=
RedCaps: Web-curated image-text data created by the people, for the people , author=. Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1) , year=
-
[136]
2022 , howpublished =
COYO-700M: Image-Text Pair Dataset , author =. 2022 , howpublished =
2022
-
[137]
2022 , url=
Christoph Schuhmann and Romain Beaumont and Richard Vencu and Cade W Gordon and Ross Wightman and Mehdi Cherti and Theo Coombes and Aarush Katta and Clayton Mullis and Mitchell Wortsman and Patrick Schramowski and Srivatsa R Kundurthy and Katherine Crowson and Ludwig Schmidt a...
2022
-
[138]
Advances in Neural Information Processing Systems , editor=
Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning , author=. Advances in Neural Information Processing Systems , editor=. 2022 , url=
2022
-
[139]
2025 , url=
It's Not a Modality Gap: Characterizing and Addressing the Contrastive Gap , author=. 2025 , url=
2025
-
[140]
Contrastive Multiview Coding
Tian, Yonglong and Krishnan, Dilip and Isola, Phillip. Contrastive Multiview Coding. Computer Vision -- ECCV 2020. 2020
2020
-
[141]
Proceedings of the 41st International Conference on Machine Learning , pages =
Position: The Platonic Representation Hypothesis , author =. Proceedings of the 41st International Conference on Machine Learning , pages =. 2024 , editor =
2024
-
[142]
2022 , editor =
Li, Junnan and Li, Dongxu and Xiong, Caiming and Hoi, Steven , booktitle =. 2022 , editor =
2022
-
[143]
2018 , eprint=
Unsupervised Representation Learning by Predicting Image Rotations , author=. 2018 , eprint=
2018
-
[144]
LiT: Zero-Shot Transfer with Locked-image text Tuning , year=
Zhai, Xiaohua and Wang, Xiao and Mustafa, Basil and Steiner, Andreas and Keysers, Daniel and Kolesnikov, Alexander and Beyer, Lucas , booktitle=. LiT: Zero-Shot Transfer with Locked-image text Tuning , year=
-
[145]
and Sebe, Nicu and Matas, Jiri and Welling, Max and Leibe, Bastian , address =
Zhang, Richard and Isola, Phillip and Efros, Alexei A. and Sebe, Nicu and Matas, Jiri and Welling, Max and Leibe, Bastian , address =. Colorful Image Colorization , volume =. Computer Vision - ECCV 2016 , copyright =
2016
-
[146]
Proceedings of the 2nd Mathematical and Scientific Machine Learning Conference , pages =
On the emergence of simplex symmetry in the final and penultimate layers of neural network classifiers , author =. Proceedings of the 2nd Mathematical and Scientific Machine Learning Conference , pages =. 2022 , editor =
2022
-
[147]
Vardan Papyan and X. Y. Han and David L. Donoho , title =. Proceedings of the National Academy of Sciences , volume =. 2020 , doi =
2020
-
[148]
arXiv preprint arXiv:2502.14786 , year=
Siglip 2: Multilingual vision-language encoders with improved semantic understanding, localization, and dense features , author=. arXiv preprint arXiv:2502.14786 , year=
-
[149]
arXiv preprint arXiv:2311.03658 , year=
The linear representation hypothesis and the geometry of large language models , author=. arXiv preprint arXiv:2311.03658 , year=
-
[150]
2013 , eprint=
Efficient Estimation of Word Representations in Vector Space , author=. 2013 , eprint=
2013
-
[151]
2024 , journal=
Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet , author=. 2024 , journal=
2024
-
[152]
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year=
Towards Backward-Compatible Representation Learning , author=. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year=
2020
-
[153]
The Twelfth International Conference on Learning Representations , year=
Structuring Representation Geometry with Rotationally Equivariant Contrastive Learning , author=. The Twelfth International Conference on Learning Representations , year=
-
[154]
International Conference on Learning Representations , year=
Contrastive Learning with Hard Negative Samples , author=. International Conference on Learning Representations , year=
-
[155]
Advances in neural information processing systems , volume=
Debiased contrastive learning , author=. Advances in neural information processing systems , volume=
-
[156]
2025 , eprint=
Compressibility Barriers to Neighborhood-Preserving Data Visualizations , author=. 2025 , eprint=
2025
-
[157]
The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=
Generalized Contrastive Learning for Universal Multimodal Retrieval , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=
-
[158]
arXiv preprint arXiv:2505.12473 , year=
Multi-modal contrastive learning adapts to intrinsic dimensions of shared latent variables , author=. arXiv preprint arXiv:2505.12473 , year=
-
[159]
Remarques sur un r
Pisier, Gilles , journal=. Remarques sur un r
-
[160]
2025 , eprint=
Global Minimizers of Sigmoid Contrastive Loss , author=. 2025 , eprint=
2025
-
[161]
2025 , eprint=
Understanding Self-Supervised Learning via Gaussian Mixture Models , author=. 2025 , eprint=
2025
-
[162]
The Fourteenth International Conference on Learning Representations , year=
On the Theoretical Limitations of Embedding-Based Retrieval , author=. The Fourteenth International Conference on Learning Representations , year=
-
[163]
CoReS: Compatible Representations via Stationarity , year =
Biondi, Niccol\`. CoReS: Compatible Representations via Stationarity , year =. doi:10.1109/TPAMI.2023.3259542 , journal =
2023 doi
-
[164]
Forward Compatible Training for Large-Scale Embedding Retrieval Systems , year=
Ramanujan, Vivek and Anasosalu Vasu, Pavan Kumar and Farhadi, Ali and Tuzel, Oncel and Pouransari, Hadi , booktitle=. Forward Compatible Training for Large-Scale Embedding Retrieval Systems , year=
-
[165]
The Eleventh International Conference on Learning Representations , year =
FastFill: Efficient Compatible Model Update , author=. The Eleventh International Conference on Learning Representations , year =
-
[166]
2022 , issue_date =
Deng, Jiankang and Guo, Jia and Yang, Jing and Xue, Niannan and Kotsia, Irene and Zafeiriou, Stefanos , title =. 2022 , issue_date =. doi:10.1109/TPAMI.2021f.3087709 , journal =
2022 doi
-
[167]
arXiv preprint arXiv:2212.03533 , year =
Text Embeddings by Weakly-Supervised Contrastive Pre-training , author =. arXiv preprint arXiv:2212.03533 , year =
-
[168]
arXiv preprint arXiv:2308.03281 , year =
Towards General Text Embeddings with Multi-stage Contrastive Learning , author =. arXiv preprint arXiv:2308.03281 , year =
-
[169]
Findings of the Association for Computational Linguistics: ACL 2024 , pages =
M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation , author =. Findings of the Association for Computational Linguistics: ACL 2024 , pages =
2024
-
[170]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =
Masked Autoencoders Are Scalable Vision Learners , author =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =
-
[171]
Computer Vision -- ECCV 2020 , year =
Unifying Deep Local and Global Features for Image Search , author =. Computer Vision -- ECCV 2020 , year =
2020
-
[172]
Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , year =
DOLG: Single-Stage Image Retrieval With Deep Orthogonal Fusion of Local and Global Features , author =. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , year =
-
[173]
arXiv preprint arXiv:2504.13181 , year=
Perception encoder: The best visual embeddings are not at the output of the network , author=. arXiv preprint arXiv:2504.13181 , year=
-
[174]
Crammer, Koby and Singer, Yoram , title =. J. Mach. Learn. Res. , month = mar, pages =. 2002 , issue_date =
2002
-
[175]
and Bartlett, Peter J
Smola, Alexander J. and Bartlett, Peter J. , title =. 2000 , isbn =
2000
-
[176]
Proceedings of the 30th International Conference on International Conference on Machine Learning - Volume 28 , pages =
Cotter, Andrew and Shalev-Shwartz, Shai and Srebro, Nathan , title =. Proceedings of the 30th International Conference on International Conference on Machine Learning - Volume 28 , pages =. 2013 , publisher =
2013
-
[177]
Proceedings of the 17th International Conference on Neural Information Processing Systems , pages =
Zhu, Ji and Rosset, Saharon and Hastie, Trevor and Tibshirani, Rob , title =. Proceedings of the 17th International Conference on Neural Information Processing Systems , pages =. 2003 , publisher =
2003
-
[178]
Tsochantaridis, Ioannis and Joachims, Thorsten and Hofmann, Thomas and Altun, Yasemin , title =. J. Mach. Learn. Res. , month = dec, pages =. 2005 , issue_date =
2005
-
[179]
Sentence-bert: Sentence embeddings using siamese bert-networks , author=. Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP) , pages=
2019
-
[180]
Journal of Computer and System Sciences , volume=
A linear lower bound on the unbounded error probabilistic communication complexity , author=. Journal of Computer and System Sciences , volume=. 2002 , publisher=
2002
-
[181]
Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48 , pages =
Liu, Weiyang and Wen, Yandong and Yu, Zhiding and Yang, Meng , title =. Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48 , pages =. 2016 , publisher =
2016
-
[182]
Journal of Complexity , volume=
The Gelfand widths of Lp-balls for 0<p<1 , author=. Journal of Complexity , volume=. 2010 , publisher=
2010
-
[183]
1967 , publisher=
Convex polytopes , author=. 1967 , publisher=
1967
Reviewed May 25, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.