Pith. sign in

REVIEW 3 major objections 4 minor 29 references

The "Law" of the Unconscious Contrastive Learner: Probabilistic Alignment of Unpaired Modalities

T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper proves that, under three stated assumptions, the probability ratio between two modalities never seen together in training is a deterministic monotone function of the dot product of their contrastive representations.

desk verdict The practical Monte Carlo method is solid, but the paper's central 'Law' as stated is not actually proven—the stated g(x) is not the function the derivation yields, and the paper's own experiments show the needed uniform-marginal assumption fails for the dot-product critic. read the letter →

arxiv 2501.11326 v1 pith:R573GZOB submitted 2025-01-20 cs.LG stat.ML

Yongwei Che , Benjamin Eysenbach This is my paper · ORCID
classification cs.LGstat.ML MSC 68T0762H30
keywords contrastivelearningmultimodalalignmentdensityratioestimationInfoNCErepresentationgeometryBayesianmarginalizationlanguage-conditionedreinforcementzero-shotretrieval
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Contrastive models are often used as if a representation from one modality can stand in for another, even when the two modalities never appeared together in training. This paper asks when that 'plug-and-play' swap is actually doing correct probabilistic inference. Under three assumptions, it proves the probability ratio $p(C\mid A)/p(C)$ between unpaired modalities is a deterministic, monotonically increasing function of the dot product $\phi_A(A)^\top \phi_C(C)$, justifying the common heuristic. It also gives a Monte Carlo estimator that marginalizes over the intermediate modality under fewer assumptions, and shows this estimator composes pretrained models and improves language-conditioned navigation under ambiguous instructions.

What carries the argument

The machinery is a chain of three objects. First, Assumption 2 treats each contrastive critic as a density-ratio estimator, $e^{f(\phi_A,\phi_B)} \propto p(B|A)/p(B)$. Second, Lemma 1 rewrites the target ratio as an expectation over the intermediate representation, $p(C|A)/p(C) = K_1 K_2 \, \mathbb{E}_{\phi_B}[\exp\{f(\phi_A,\phi_B)+f(\phi_B,\phi_C)\}]$, which is exact message passing on the graphical model $A-B-C$ when Assumption 1 holds. Third, under Assumption 3 the uniform marginal turns that expectation into an integral of a von Mises-Fisher density, whose normalizing constant yields the monotone Bessel function $g$. When uniformity fails, the same expectation can be approximated by Monte Carlo samples, giving the LogSumExp algorithm.

What would settle it

Estimate $p(C\mid A)/p(C)$ directly from data, for example by counting matched examples in a large synthetic triple satisfying conditional independence, then compare it with $g(\phi_A(A)^\top \phi_C(C))$ for the same inputs. If the two disagree beyond sampling error, or if the empirical relation is not monotone in the dot product, the law as stated fails. The paper's Figure 2b already points toward this test: under the dot-product critic the direct method underperforms the Monte Carlo baseline, which is evidence that the uniformity assumption fails in that setting.

Watch

Extended reading notes

Core claim

The central result is Lemma 2: when A and C are conditionally independent given B, when the two contrastive critics encode density ratios, and when all representation marginals are uniform on the unit hypersphere, the ratio $p(C\mid A)/p(C)$ equals $g(\phi_A(A)^\top \phi_C(C))$, where $g(x) = (2\pi)^{p/2} I_{p/2-1}(x)/\|x\|^{p/2-1}$ is monotonically increasing in $x$. In words, a direct dot-product comparison of embeddings from modalities never trained together is not just a similarity score; it is a fixed monotone transform of the true likelihood ratio. The paper also proves a Gaussian analogue for unnormalized representations under an L2-distance critic, and derives a Monte Carlo LogSumExp estimator from Lemma 1 that replaces the uniformity assumption with explicit sampling over the intermediate modality.

Load-bearing premise

The load-bearing premise is that contrastive representations are spread uniformly over the unit sphere; the paper's own experiments show this is violated for the dot-product critic in which the closed-form law is stated.

Editorial extensions

If this is right

  • When the assumptions hold, cosine similarity between unpaired modality embeddings is a monotone transform of the log-likelihood ratio, so retrieval by nearest-neighbor search is implicitly Bayesian.
  • The Monte Carlo LogSumExp method gives a way to compose pretrained contrastive models across a shared intermediate modality without needing paired data from the target modalities.
  • For language-conditioned reinforcement learning, marginalizing over future states rather than using a mean language embedding avoids the failure mode of navigating to the centroid of an ambiguous description.
  • With an L2-distance critic and Gaussian marginals, the same reasoning yields a closed form in which the log ratio is essentially the negative squared distance between unpaired representations.
  • Because $g$ is monotone, ranking candidates by raw dot product agrees with ranking by the exact probability ratio, so the method validates common practice even when users never compute the Bessel transform explicitly.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A practical diagnostic suggested by the two estimators: run both direct comparison and the LogSumExp estimator on a held-out set; where their rankings diverge, the uniformity assumption is suspect and the Monte Carlo answer is safer, since it requires fewer assumptions.
  • The framework points toward graph-structured modality composition beyond a single chain: chaining through several intermediate modalities should work whenever each step satisfies conditional independence, though violations would accumulate, and the Monte Carlo estimator degrades gracefully rather than failing outright.
  • The language-conditioned RL results suggest that any policy using a single averaged language embedding for an ambiguous instruction is discarding a multimodal goal distribution; replacing the mean with an explicit sum over candidate goal states is a drop-in improvement.
  • Because the law's closed form is monotone, the paper's result also explains why uncalibrated cosine-similarity thresholds in existing systems can still produce correct rankings: the threshold scale is wrong, but the ordering is preserved.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper studies whether contrastive representations learned from paired data (A,B) and (B,C) can support inferences about unpaired modalities A and C. Lemma 1 expresses the marginal likelihood ratio p(C|A)/p(C) as an expectation over representations of the bridge modality B under Assumptions 1 (conditional independence) and 2 (density-ratio critics). Lemma 2 aims to show that, under an additional uniformity assumption on the representation marginal, this ratio is a monotone function of the dot product phi_A(A)^T phi_C(C), thereby justifying the common 'direct comparison' heuristic. Lemma 3 gives an analogous closed form for unnormalized Gaussian representations with an L2 critic. Section 5 turns Lemma 1 into a Monte Carlo LogSumExp algorithm that avoids Assumption 3. Experiments on synthetic data, on CLIP/CLAP/LanguageBind, and in language-conditioned reinforcement learning compare direct comparison with the Monte Carlo method and with ground-truth oracles.

Significance. If Lemma 2 were correct, it would provide a theoretical grounding for a widely used heuristic. The Monte Carlo method based on Lemma 1 is a genuinely useful and assumption-light contribution: it connects pre-trained models without additional training and provides a principled treatment of ambiguity in language-conditioned tasks. The derivations are not circular: they start from Bayes' rule and assumptions adapted from prior work on InfoNCE and hyperspherical uniformity, and the experiments compare against ground-truth oracles. However, the central 'Law' is not established as stated. Lemma 2 contains a normalization error, an incorrect substitution into the Bessel function, and the printed function is not monotone. The paper's own experiments show that the key uniformity assumption is violated for the dot-product setting of Lemma 2. These issues affect the main theoretical claim and require substantial correction.

major comments (3)
  1. [4.3 (Lemma 2 and its proof)] The claimed equality and the stated monotonicity are not correct as written. The proof writes the expectation under U(S^{d-1}) as an unnormalized surface integral; the uniform density contributes a factor 1/Vol(S^{d-1}), and the von Mises-Fisher normalization gives E[exp(phi_B^T(phi_A+phi_C))] = 1 / (Vol(S^{d-1}) C_p(||phi_A+phi_C||)), not 1/C_p(...). More importantly, the last step substitutes the inner product x = phi_A^T phi_C directly into the Bessel-function formula. The correct argument is kappa = ||phi_A+phi_C|| = sqrt(2+2x). For p=2 the printed g(x) = (2*pi)^(p/2) I_{p/2-1}(x)/||x||^{p/2-1} reduces to 2*pi*I_0(x), an even function, so it is not monotonically increasing on [-1,1]; for odd p it is not real for negative x. Thus the central 'Law' as stated is false even under Assumptions 1-3. The fix is to set g(x) = (K1*K2 / Vol(S^{d-1})) * (2*pi)^(p/2) I_{p/2-1}(sqrt(2+2x)) / (sqrt(2+2x))^{p/2-1}, to include the previously omitted constants, and to prove monotonicity of I_{p/2-1}(kappa)/kappa^{p/2-1} in kappa. The proof's final line also contains a typo: 'phi_A(a)^T phi_B(b)' should read 'phi_A(a)^T phi_C(c)'.
  2. [4.4 / Appendix B (Lemma 3)] The value of delta is inconsistent between the main text and the appendix: Lemma 3 states delta = 1/(c+1), whereas Eq. (8) and the appendix's final line correspond to delta = 2/(c+1). The sentence after Lemma 3 that the log probability ratio 'asymptotically approaches the negative l2 distance' as c increases is also inaccurate: with gamma -> 1/4 and delta -> 0 as c -> infinity, the limiting log ratio is -1/4 ||phi_A - phi_C||^2 (up to an additive constant), not the negative l2 distance or the negative squared l2 distance used by the critic. Please correct the constant in the lemma statement and revise the asymptotic claim.
  3. [6.1.1 (Fig. 2b, Fig. 7)] The paper's own experiments show that Assumption 3 is violated for the dot-product critic, which is precisely the setting of Lemma 2. Figure 2b shows that the Direct method underperforms the Monte Carlo method, and Fig. 7 shows that the unnormalized representation distribution is not uniform. The paper therefore does not empirically validate the 'Law' in its main claimed form; it validates only the conditional statement 'if Assumption 3 holds, then the Law holds.' Moreover, Section 6.2.2 tests uniformity only for language representations over the AudioSet ontology, not for the marginal distributions of all three encoders used in the dot-product experiments. Please either provide experiments in which Assumption 3 is verified for the dot-product critic, or explicitly restrict the claim of the 'Law' to settings where uniformity can be established, presenting the synthetic dot-product results as a negative test of the assumption.
minor comments (4)
  1. [4.2] The displayed lower bound log E[exp(phi_A^T phi_B + phi_B^T phi_C)] >= phi_A^T phi_C is not a consequence of the triangle inequality and is false for a pointwise realization. For example, in d=2 take phi_A=(1,0), phi_C=(0,1), and phi_B=(-1,-1)/sqrt(2); then phi_B^T(phi_A+phi_C) = -sqrt(2) < 0 = phi_A^T phi_C. The passage should be corrected or removed.
  2. [3.3 (Assumption 3)] Assumption 3 is stated with an integral over an indicator p(phi) = integral p(x) 1{phi_M(x)=phi} dx, which is not well-defined for continuous representations; the assumption should be stated as a pushforward measure on the hypersphere.
  3. [6.2.2] The two-sample Kolmogorov-Smirnov results (CLIP p=0.0877, CLAP p=0.1788) are described as showing that Assumption 3 'fares well,' but a failure to reject uniformity is not positive evidence for uniformity, especially with a small ontology. Reporting an effect size or a power analysis would make the claim more appropriate.
  4. [Figure 2 caption] The caption lists 'Center' twice; the second center should be 'Right'.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the derivation is conditional on explicit assumptions, benchmarked externally, and the only self-citation is not load-bearing.

full rationale

The derivation chain is not circular. Lemma 1 is obtained by applying Bayes' rule to Assumptions 1 and 2: the paper writes p(C|A)/p(C) = ∫_B p(C|B)/p(C) · p(B|A)/p(B) · p(B) dB = K1·K2·E_ϕB[exp{f(ϕC,ϕB)+f(ϕB,ϕA)}]. Lemma 2 is then a closed-form evaluation of that expectation under Assumption 3, which is stated as an assumption and empirically tested in Sec. 6.2.2 and Fig. 7. The Monte Carlo method in Sec. 5 is an unbiased estimator of Lemma 1, not a fitted quantity renamed as a prediction; no parameter is fit to the (A,C) data whose retrieval accuracy is later reported. The experiments compare Direct and Monte Carlo methods against a ground-truth oracle trained on (A,C) pairs, providing an external benchmark. The only relevant self-citation is Eysenbach et al. (2024), cited alongside Wang & Isola (2020) to motivate the Gaussian marginal used in Lemma 3; because that marginal is explicitly assumed and the central 'Law' relies on Assumption 3, which is supported by the external Wang & Isola citation and by the paper's own uniformity tests, this self-citation is not load-bearing. I therefore find no circular step. Separately, and not as a circularity finding: the Lemma 2 proof as written appears to drop the 1/Vol(S^{d-1}) prefactor of the uniform density and writes sqrt(2+ϕ(A)^T ϕ(B)) where the substitution requires sqrt(2+2ϕ(A)^T ϕ(C)); the stated g(x) also does not match the vMF normalization after that substitution. This is a mathematical correctness concern, not a circularity.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical or mathematical entities; it relies on standard probability and known properties of contrastive representations. No free parameters are fitted to data; the constants K, K1, K2 are approximation errors from contrastive learning.

assumptions (5)
  • domain assumption Assumption 1: A and C are conditionally independent given B (A ⊥ C | B).
    Needed for Lemma 1 to factor p(C|A,B) as p(C|B); the paper proves this is necessary for identifiability (Lemma 4).
  • domain assumption Assumption 2: Contrastive representations model the density ratio e^{f(phi_A, phi_B)} = p(B|A) / (K p(B)).
    Borrowed from Poole et al. 2019 and Ma & Collins 2018; the paper states it as an assumption because it can be violated with finite data.
  • domain assumption Assumption 3: The marginal distribution of representations is uniform over the hypersphere.
    Borrowed from Wang & Isola 2020; used to evaluate the integral in Lemma 2; empirically violated for the dot-product critic in the paper's own experiments.
  • domain assumption For Lemma 3, the representation marginal is an isotropic Gaussian N(0, cI).
    Used to evaluate the Gaussian integral in Lemma 3; justified by prior work but not verified in experiments.
  • standard math Standard probabilistic identities: Bayes' rule, law of total probability, and the von Mises-Fisher integral.
    Basic probability facts used throughout the proofs; no issues.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The "Law" of the Unconscious Contrastive Learner: Probabilistic Alignment of Unpaired Modalities." pith.science (2026). https://pith.science/paper/R573GZOB

@misc{pith2026250111326,
  author       = {Pith},
  title        = {Pith review of: The "Law" of the Unconscious Contrastive Learner: Probabilistic Alignment of Unpaired Modalities},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/R573GZOB}},
  note         = {Machine review of arXiv:2501.11326}
}
read the original abstract

While internet-scale data often comes in pairs (e.g., audio/image, image/text), we often want to perform inferences over modalities unseen together in the training data (e.g., audio/text). Empirically, this can often be addressed by learning multiple contrastive embedding spaces between existing modality pairs, implicitly hoping that unseen modality pairs will end up being aligned. This theoretical paper proves that this hope is well founded, under certain assumptions. Starting with the proper Bayesian approach of integrating out intermediate modalities, we show that directly comparing the representations of data from unpaired modalities can recover the same likelihood ratio. Our analysis builds on prior work on the geometry and probabilistic interpretation of contrastive representations, showing how these representations can answer many of the same inferences as probabilistic graphical models. Our analysis suggests two new ways of using contrastive representations: in settings with pre-trained contrastive models, and for handling language ambiguity in reinforcement learning. Our numerical experiments study the importance of our assumptions and demonstrate these new applications.

Figures

Figures reproduced from arXiv: 2501.11326 by the authors.

Figure 1
Figure 1. Aligning vision and audio by integrating over the intermediate ‘language’ modality. While prior work has shown that aligning modalities A ↔ B and B ↔ C results in representations that can compare modalities A and C, it remains unclear when and why this is guaranteed to work. This paper provides the assumptions under which this approach is principled, and our analysis unlocks new ways of comparing unpaired modalities… view at source ↗
Figure 2
Figure 2. Testing the “Law” of the Unconscious Contrastive Learning with three parametrizations of the critic function. We assess whether the “Law” holds by comparing the success of the “Direct” method to an oracle that is trained on (A, C) examples. We also include the Monte Carlo method based on Lemma 1 to understand the assumptions belying our analysis. (Left) For the L2 critic, all methods perform well, suggesting that al… view at source ↗
Figure 3
Figure 3. A principled way of combining pre￾trained models. Given pre-trained models that compute the similarities A ↔ B and B ↔ C, we use Lemma 1 to infer the similarity between A ↔ C. In many practical applications, users only have access to the similarity scores between pairs of modalities, but nonetheless want to reason about the similarities of unpaired modalities. For example, let’s say a user wants to compute the simil… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: (Left) Direct evaluation of the CLIP Image Encoder with the CLAP Audio Encoder for audio-visual inference versus using our LogSum￾Exp algorithm with those same encoders. (Right) Direct evaluation with LanguageBind encoders vs our LogSumExp algorithm with the those same…
Figure 5
Figure 5. Figure 5: The Accuracy of our LogSumExp (Monte Carlo) approximation scales with the number of [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: LogSumExp performs well with other intermediate modality types (audio and vision). The [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Embedding Distributions under different critic functions [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Performance degrades when Assumption 1 is violated. The x-axis corresponds to the [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: Fork Maze Environment with multiple possible paths to the "first column" [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: Direct evaluation chooses a subopti￾mal rightward action when given eight possible directions (unit vectors), failing to find the short￾est path to the "first column." [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]
Figure 12
Figure 12. Figure 12: Monte Carlo approximation visualization over the intermediate state modality, sampling [PITH_FULL_IMAGE:figures/full_fig_p019_12.png]
Figure 13
Figure 13. Figure 13: Success Rates of Direct Comparison vs LSE on different Grid Environments [PITH_FULL_IMAGE:figures/full_fig_p020_13.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

29 extracted references · 11 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Do as i can, not as i say: Grounding language in robotic affordances, 2022

    Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Daniel Ho, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Eric Jang, Rosario Jauregui Ruano, Kyle Jeffrey, Sally Jesmonth, Nikhil J Joshi, Ryan Julian, Dmitry Kalashnikov, Yuheng Kuang, K...

    arXiv 2022

  3. [3]

    Bridging environments and language with rendering functions and vision-language models

    Theo Cachet, Christopher R Dance, and Olivier Sigaud. Bridging environments and language with rendering functions and vision-language models. In Forty-first International Conference on Machine Learning, 2024. URL https://openreview.net/forum?id=ZrM67ZZ5vj

    work page 2024

  4. [4]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pp.\ 1597--1607. PMLR, 2020

    2020

  5. [5]

    An empirical study of training self-supervised vision transformers

    Xinlei Chen, Saining Xie, and Kaiming He. An empirical study of training self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp.\ 9640--9649, 2021

    2021

  6. [6]

    Clap learning audio concepts from natural language supervision

    Benjamin Elizalde, Soham Deshmukh, Mahmoud Al Ismail, and Huaming Wang. Clap learning audio concepts from natural language supervision. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.\ 1--5. IEEE, 2023

    2023

  7. [7]

    Contrastive learning as goal-conditioned reinforcement learning, 2023

    Benjamin Eysenbach, Tianjun Zhang, Ruslan Salakhutdinov, and Sergey Levine. Contrastive learning as goal-conditioned reinforcement learning, 2023

    work page 2023

  8. [8]

    Inference via interpolation: Contrastive representations provably enable planning and inference

    Benjamin Eysenbach, Vivek Myers, Ruslan Salakhutdinov, and Sergey Levine. Inference via interpolation: Contrastive representations provably enable planning and inference. arXiv preprint arXiv:2403.04082, 2024

    arXiv 2024

Show all 29 references
  1. [9]

    Contrastive learning for cross-modal artist retrieval, 08 2023

    Andres Ferraro, Jaehun Kim, Sergio Oramas, Andreas Ehmann, and Fabien Gouyon. Contrastive learning for cross-modal artist retrieval, 08 2023

    2023

  2. [10]

    Gemmeke, Daniel P

    Jort F. Gemmeke, Daniel P. W. Ellis, Dylan Freedman, Aren Jansen, Wade Lawrence, R. Channing Moore, Manoj Plakal, and Marvin Ritter. Audio set: An ontology and human-labeled dataset for audio events. In 2017 IEEE International Conference on Acoustics, Speech and Signal Process...

    2017

  3. [11]

    Imagebind: One embedding space to bind them all

    Rohit Girdhar, Alaaeldin El-Nouby, Zhuang Liu, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, and Ishan Misra. Imagebind: One embedding space to bind them all. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 15180--15190, 2023

    2023

  4. [12]

    Bootstrap your own latent-a new approach to self-supervised learning

    Jean-Bastien Grill, Florian Strub, Florent Altch \'e , Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to self-supervised learning. Advances in neur...

    2020

  5. [13]

    Audioclip: Extending clip to image, text and audio, 2021

    Andrey Guzhov, Federico Raue, Jörn Hees, and Andreas Dengel. Audioclip: Extending clip to image, text and audio, 2021

    2021

  6. [14]

    Dimensionality reduction by learning an invariant mapping

    Raia Hadsell, Sumit Chopra, and Yann LeCun. Dimensionality reduction by learning an invariant mapping. In 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR'06), volume 2, pp.\ 1735--1742. IEEE, 2006

    2006

  7. [15]

    Interactive language: Talking to robots in real time

    Corey Lynch, Ayzaan Wahid, Jonathan Tompson, Tianli Ding, James Betker, Robert Baruch, Travis Armstrong, and Pete Florence. Interactive language: Talking to robots in real time. IEEE Robotics and Automation Letters, 2023

    2023

  8. [16]

    Noise contrastive estimation and negative sampling for conditional models: Consistency and statistical efficiency

    Zhuang Ma and Michael Collins. Noise contrastive estimation and negative sampling for conditional models: Consistency and statistical efficiency. arXiv preprint arXiv:1809.01812, 2018

    2018 arXiv

  9. [17]

    Efficient estimation of word representations in vector space

    Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781, 2013

    2013 arXiv

  10. [18]

    Representation learning with contrastive predictive coding

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018

    2018 arXiv

  11. [19]

    On variational bounds of mutual information

    Ben Poole, Sherjil Ozair, Aaron Van Den Oord, Alex Alemi, and George Tucker. On variational bounds of mutual information. In International Conference on Machine Learning, pp.\ 5171--5180. PMLR, 2019

    2019

  12. [20]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp...

    2021

  13. [21]

    Vint: A foundation model for visual navigation, 2023

    Dhruv Shah, Ajay Sridhar, Nitish Dashora, Kyle Stachowicz, Kevin Black, Noriaki Hirose, and Sergey Levine. Vint: A foundation model for visual navigation, 2023

    2023

  14. [22]

    Roboclip: One demonstration is enough to learn robot policies

    Sumedh Sontakke, Jesse Zhang, S \'e b Arnold, Karl Pertsch, Erdem B y k, Dorsa Sadigh, Chelsea Finn, and Laurent Itti. Roboclip: One demonstration is enough to learn robot policies. Advances in Neural Information Processing Systems, 36, 2024

    2024

  15. [23]

    Language-guided robot grasping: Clip-based referring grasp synthesis in clutter

    Georgios Tziafas, Yucheng Xu, Arushi Goel, Mohammadreza Kasaei, Zhibin Li, and Hamidreza Kasaei. Language-guided robot grasping: Clip-based referring grasp synthesis in clutter. arXiv preprint arXiv:2311.05779, 2023

    2023 arXiv

  16. [24]

    Understanding contrastive representation learning through alignment and uniformity on the hypersphere

    Tongzhou Wang and Phillip Isola. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In International conference on machine learning, pp.\ 9929--9939. PMLR, 2020

    2020

  17. [25]

    Connecting multi-modal contrastive representations, 2023

    Zehan Wang, Yang Zhao, Xize Cheng, Haifeng Huang, Jiageng Liu, Li Tang, Linjun Li, Yongqi Wang, Aoxiong Yin, Ziang Zhang, and Zhou Zhao. Connecting multi-modal contrastive representations, 2023

    2023

  18. [26]

    Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding, 2023

    Le Xue, Mingfei Gao, Chen Xing, Roberto Martín-Martín, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, and Silvio Savarese. Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding, 2023

    2023

  19. [27]

    Understanding belief propagation and its generalizations

    Jonathan S Yedidia, William T Freeman, Yair Weiss, et al. Understanding belief propagation and its generalizations. Exploring artificial intelligence in the new millennium, 8 0 (236-239): 0 0018--9448, 2003

    2003

  20. [28]

    Muse: Multi-view contrastive learning for heterophilic graphs, 2023

    Mengyi Yuan, Minjie Chen, and Xiang Li. Muse: Multi-view contrastive learning for heterophilic graphs, 2023

    2023

  21. [29]

    Languagebind: Extending video-language pretraining to n-modality by language-based semantic alignment

    Bin Zhu, Bin Lin, Munan Ning, Yang Yan, Jiaxi Cui, HongFa Wang, Yatian Pang, Wenhao Jiang, Junwu Zhang, Zongwei Li, et al. Languagebind: Extending video-language pretraining to n-modality by language-based semantic alignment. arXiv preprint arXiv:2310.01852, 2023

    arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.