Pith. sign in

REVIEW 3 major objections 6 minor 41 references

A Mathematical Perspective On Contrastive Learning

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Standard contrastive learning, read probabilistically, matches conditional means but inflates conditional variances; a one-sided quadratic tilt matches a full conditional and a joint loss improves marginals.

desk verdict A useful tilting framework with correct Gaussian minimizers, but the practical CLIP claims outrun the unnormalized-linear model, and the proofs need fixing. read the letter →

arxiv 2505.24134 v1 pith:7D6RLREG submitted 2025-05-30 stat.ML cs.CVcs.LG

Ricardo Baptista , Andrew M. Stuart , Son Tran This is my paper
classification stat.MLcs.CVcs.LG
keywords contrastivelearningmultimodalconditionaldistributionsGaussiananalysislow-rankmatrixapproximationexponentialtiltingcrossmodalretrievalprobabilisticlossfunctions
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's aim is to give contrastive learning a probabilistic meaning: training encoders is equivalent to picking an exponential tilting of the product of the two marginal distributions so that the resulting joint distribution approximates the true joint. In the Gaussian case with linear encoders this can be solved in closed form, turning latent-space identification into low-rank matrix approximation. The central finding is that the standard two-sided cosine loss matches conditional means but systematically inflates conditional variances, while a one-sided quadratic tilt can match one full conditional exactly and a joint loss brings the marginals closer to the truth. For a practitioner the upshot is that the choice of loss and alignment metric should depend on the task: retrieval wants correct conditional means, generation wants correct conditional spread.

What carries the argument

The machinery is the exponential tilting of the product of the marginals, $\nu(du,dv)\propto \exp(\langle Gu,Hv\rangle)\mu_u(du)\mu_v(dv)$ for cosine alignment and $\nu(du,dv)\propto \exp(-\tfrac12|Gu-Hv|^2)\mu_u(du)\mu_v(dv)$ for the quadratic tilt, with $G,H$ linear encoders. These tiltings make the model joint Gaussian when the marginals are Gaussian, and all learnable statistics are carried by $A=G^\top H$ and $B=G^\top G$. The Gaussian analysis then reduces each learning problem to a low-rank matrix approximation in the Frobenius norm, solved by singular-value truncation; for the joint loss the per-singular-value function $h(\sigma)$ encodes the optimal shrinkage of the cross-covariance.

What would settle it

Fit linear encoders to a two-dimensional centred Gaussian with $C_{uu}=C_{vv}=1$ and $C_{uv}=\rho>0$ using the population two-sided cosine conditional loss, and inspect the learned conditional $\nu_{u|v}$. The theory predicts variance exactly $1$ and mean $\rho v$, so the variance exceeds the true conditional variance $1-\rho^2$; observing a conditional variance smaller than $1$ under dependence would refute the central corollary.

Watch

Extended reading notes

Core claim

Within the Gaussian model class, the paper proves three precise statements. Under the cosine alignment with the two-sided conditional loss, the unique minimizer over unconstrained matrices is $A^*=C_{uu}^{-1}C_{uv}C_{vv}^{-1}$, and the induced conditionals $\mathcal{N}(C_{uv}C_{vv}^{-1}v, C_{uu})$ and $\mathcal{N}(C_{vu}C_{uu}^{-1}u, C_{vv})$ have the true conditional means but covariances strictly larger than the data conditionals unless $u$ and $v$ are independent. Under the one-sided quadratic tilt, minimizing the conditional KL divergence for $u|v$ alone yields parameters $(A^*,B^*)$ for which $\nu_{u|v}$ is exactly $\mathcal{N}(C_{uv}C_{vv}^{-1}v, C_{u|v})$. Under the cosine alignment with the joint KL loss, the optimal parameter applies the shrinkage function $h(\sigma)=\sigma^{-1}\left(\tfrac12\sqrt{1+4\sigma^2}-\tfrac12\right)$ to the singular values of the normalized cross-covariance, giving marginal covariances closer to the true marginals than those from the conditional loss. All three results have rank-constrained versions in which the best rank-$r$ encoder is obtained by singular-value truncation.

Load-bearing premise

The closed-form conclusions hold only inside the model class of exponential tiltings of the product of the marginals built from linear encoders; if the true conditionals are not representable by such a tilting, or if the encoders are nonlinear, the specific matched-statistics results in Corollaries 5.2, 5.4, and 5.7 do not apply.

Editorial extensions

If this is right

  • For retrieval and mode-seeking, the standard two-sided cosine loss is well matched to the task: the optimal encoder recovers the true conditional means, so the mode of the learned conditional is the true conditional mode.
  • For generative sampling, the standard loss is systematically over-dispersed: the learned conditionals have the right centre but inflated spread, so samples are too scattered.
  • A one-sided quadratic tilt gives an exact conditional model for the single direction it targets, at the cost of leaving the other conditional misspecified; this suits applications with one dominant retrieval direction.
  • The joint loss is preferable when marginal structure matters, because it produces marginal covariances closer to the data, but it introduces bias in the conditional means.
  • The embedding dimension acts as a rank constraint: when $n_e<\min(n_u,n_v)$, the optimal encoders are the best rank-$r$ approximations of the unconstrained solution, so capacity directly controls which conditional statistics are captured.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: Corollary 5.2 gives a diagnostic that can be run on any trained contrastive model: generate samples from the learned conditional and compare their spread with the spread of observed pairs; persistent over-dispersion would indicate the model is effectively solving the two-sided cosine conditional loss.
  • Editorial extension: the one-sided quadratic result suggests that in asymmetric modality pairs, such as text-to-image, the loss can be deliberately oriented to model the harder conditional exactly, accepting that the reverse conditional will be poorly calibrated; a symmetric two-sided quadratic objective appears not to be covered by the paper's closed-form theory.
  • Editorial extension: because the joint loss needs only one batch from the joint distribution and one from the product of marginals, it is computationally cheaper at scale; if the conditional-mean bias it introduces is tolerable, it may be a better default for generative fine-tuning, a conjecture that could be tested on non-Gaussian data.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper develops a probabilistic reformulation of bimodal contrastive learning in which the learned representation is viewed as an exponential tilting of the product of the two marginal distributions, parameterized by encoders. It introduces generalized conditional and joint loss functions, as well as unnormalized and quadratic tiltings, and analyzes the resulting optimization problems in the multivariate Gaussian setting with linear encoders. The main theoretical results give closed-form minimizers: under the cosine-alignment conditional loss the conditional means are matched but conditional variances are inflated (Theorem 5.1, Corollary 5.2); under a one-sided positive-quadratic tilting the full conditional distribution for one modality is matched exactly (Theorem 5.3, Corollary 5.4); and under the joint loss the marginals are closer to the truth than under the conditional loss (Theorem 5.6, Corollary 5.7). The paper also includes numerical experiments on Gaussians, MNIST classification, and a Lagrangian data assimilation problem, and it connects the framework to retrieval and classification tasks.

Significance. If the results hold, the paper provides a useful mathematical lens on contrastive learning: the reformulation as a change of measure is elegant, and the closed-form Gaussian analysis gives concrete, falsifiable statements about which conditional statistics are matched by different losses and alignment metrics. The low-rank interpretation is a genuine contribution, and the numerical experiments, especially the data assimilation application, support the claimed relevance beyond a purely academic setting. The paper is not circular: the 'predictions' are theorems derived from the loss definitions, not fitted quantities. A notable strength is that the authors explicitly flag several limitations, including the restriction to unnormalized linear encoders and the lack of general guarantees for normalized encoders. The main weaknesses are algebraic errors in the appendix proofs and a scope mismatch between the unnormalized-encoder theory and the broader practical framing in the abstract and discussion.

major comments (3)
  1. [Appendix D, proof of Theorem 5.1, Eq. (68)] The completion-of-the-square identity used to solve for A* is algebraically incorrect as printed. Expanding the displayed product Tr((A^T C_uu - C_vv^{-1}C_vu)(A C_vv - C_uu^{-1}C_uv)) gives Tr(A^T C_uu A C_vv) - 2Tr(A C_vu) plus a constant, whereas Lcond(A) is -Tr(A C_vu) + Tr(A^T C_uu A C_vv); in the scalar case the printed identity has cross term -2αc instead of -αc. The correct identity is Lcond(A) + (1/2)Tr(C_uu^{-1}C_uv C_vv^{-1}C_vu) = (1/2)||C_uu^{1/2}(A-A*)C_vv^{1/2}||_F^2 with A* = C_uu^{-1}C_uv C_vv^{-1}. The stated minimizer and the rank-r formula (44) are nevertheless correct, so this is a proof-repair issue rather than a wrong result.
  2. [Appendix D, proof of Theorem 5.3, Eq. (70)] The KL divergence formula has a sign error and a missing prefactor. The log-determinant term should be -log|(B+C_uu^{-1})C_{u|v}|, not +log|...|, and the quadratic term Δ(A,B) should carry the 1/2 factor from Lemma E.1. With the printed plus sign the objective is not bounded below in B, so the displayed derivation cannot identify B*; with the corrected sign, minimizing over S = B+C_uu^{-1} the function Tr(S C_{u|v}) - log|S C_{u|v}| gives S = C_{u|v}^{-1}, i.e. B* as in (49b). The missing 1/2 propagates into the rank-constrained objective (50b), which as printed does not define the minimizer of the actual loss. The central Corollary 5.4 uses the unconstrained minimizers and remains correct, but (50) needs to be rederived.
  3. [Scope of practical claims, Sections 5 and 7] Corollaries 5.2, 5.4 and 5.7 are proved only for the unnormalized linear-encoder model classes (39) and (46). Section 5 states this restriction, and the normalized-encoder toy in Section 5.4 is explicitly described as not guaranteed to generalize to arbitrary dimensions. Nevertheless, the abstract, Section 1, and Section 7 present the Gaussian results as informing the choice of loss and alignment metric for contrastive learning generally, and the standard CLIP setting in (2) uses normalized encoders. Because normalization changes the learned measure qualitatively (the toy already shows non-Gaussian conditionals for Gaussian data), the paper should either extend the theory to normalized encoders or prominently restrict the practical recommendations to the unnormalized variants studied in the theorems.
minor comments (6)
  1. [Corollary 5.2, Eq. (45b)] The term 'Cvuv' is a typo and should read 'Cvu'.
  2. [Figure 3 caption] The caption says 'cosine distance with the joint loss' but the described matching of the u|v conditional mean and variance is the positive quadratic form with the one-sided conditional loss of Theorem 5.3.
  3. [Equation (35)] The classification rule is written as an arg min of the log-weights; since wi is proportional to exp(⟨G_i, gu(u)⟩ + F_i), the mode should be an arg max.
  4. [Appendix D, proof of Theorem 5.1] The line defining A* as C_vv^{-1}C_vu C_uu^{-1} is dimensionally the transpose of the minimizer stated in Theorem 5.1; the intended expression is C_uu^{-1}C_uv C_vv^{-1}.
  5. [Section 1.3] The phrase 'supplementary meterial' should read 'supplementary material'.
  6. [Section 5 headings] The labels 'Cosine Distance' are used for the unnormalized inner-product model (39); since cosine distance normally presupposes normalization, a brief clarification that these are unnormalized tiltings would help the reader map the theory to practice.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the Gaussian characterizations are closed-form optimizers of explicitly defined losses, not fitted quantities renamed as predictions.

full rationale

The paper's central results (Corollaries 5.2, 5.4, and 5.7) are theorems obtained by minimizing explicitly defined population-level loss functions, such as Lcond in (42), Lcond(A,B;2,0) in Section 5.2, and Ljoint in (52). The derivations in Appendix D use Gaussian moment identities, the von Neumann trace inequality, and rank-constrained matrix approximation; no parameter is fitted to a subset of data and then reported as a prediction of a closely related quantity. The model classes in (39) and (46) are stated modeling choices rather than conclusions forced by self-citation, and the one-sided loss is deliberately constructed to target a specific conditional, so exact recovery of that conditional is the objective being optimized, not a circularly imported result. The identity between the CLIP loss and the conditional loss in Theorem 2.8 is proved directly from the definitions. The paper explicitly limits its Gaussian theory to unnormalized linear encoders and notes that normalized encoders are treated only in a one-dimensional toy example; this is a scope and generalizability caveat, not a circularity. The only author-overlapping citation ([4]) appears in the literature review and is not load-bearing for any theorem or corollary.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The framework introduces no new entities. The 'tilting' is a mathematical construction, an exponential change of measure, not a physical object. The analysis is self-contained given the model class; the main burden is the linear-Gaussian model class assumption.

assumptions (4)
  • domain assumption The paired data are drawn i.i.d. from a joint distribution µ on U x V (Data Assumption 2.1).
    All population-level losses and theorems are defined with respect to this µ; if the data are not i.i.d., the population limit is not justified.
  • domain assumption In the Gaussian setting, the joint distribution µ is centered Gaussian with strictly positive definite covariance C.
    Needed for the closed-form conditional distributions (37) and for the matrix computations in Section 5.
  • ad hoc to paper The model family is restricted to exponential tiltings of the product of marginals with linear encoders gu=Gu, gv=Hv (equations (38), (39), and (46)).
    This model class is the paper's proposed generalization; the closed-form minimizers (Theorems 5.1, 5.3, 5.6) only hold inside this class. Nonlinear encoders or non-Gaussian references are outside the theory.
  • domain assumption The analysis is performed in the population limit, ignoring finite-sample effects except in experiments.
    Theorem 2.8 connects the finite-sample CLIP loss to L_cond only up to a constant and requires the full dataset as the negative set; practical batch training is an approximation studied only numerically in Section 6.1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Mathematical Perspective On Contrastive Learning." pith.science (2026). https://pith.science/paper/7D6RLREG

@misc{pith2026250524134,
  author       = {Pith},
  title        = {Pith review of: A Mathematical Perspective On Contrastive Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7D6RLREG}},
  note         = {Machine review of arXiv:2505.24134}
}
read the original abstract

Multimodal contrastive learning is a methodology for linking different data modalities; the canonical example is linking image and text data. The methodology is typically framed as the identification of a set of encoders, one for each modality, that align representations within a common latent space. In this work, we focus on the bimodal setting and interpret contrastive learning as the optimization of (parameterized) encoders that define conditional probability distributions, for each modality conditioned on the other, consistent with the available data. This provides a framework for multimodal algorithms such as crossmodal retrieval, which identifies the mode of one of these conditional distributions, and crossmodal classification, which is similar to retrieval but includes a fine-tuning step to make it task specific. The framework we adopt also gives rise to crossmodal generative models. This probabilistic perspective suggests two natural generalizations of contrastive learning: the introduction of novel probabilistic loss functions, and the use of alternative metrics for measuring alignment in the common latent space. We study these generalizations of the classical approach in the multivariate Gaussian setting. In this context we view the latent space identification as a low-rank matrix approximation problem. This allows us to characterize the capabilities of loss functions and alignment metrics to approximate natural statistics, such as conditional means and covariances; doing so yields novel variants on contrastive learning algorithms for specific mode-seeking and for generative tasks. The framework we introduce is also studied through numerical experiments on multivariate Gaussians, the labeled MNIST dataset, and on a data assimilation application arising in oceanography.

Figures

Figures reproduced from arXiv: 2505.24134 by the authors.

Figure 1
Figure 1. Two-dimensional densities for the reference Gaussian distribution given by the product of [PITH_FULL_IMAGE:figures/full_fig_p023_1.png] view at source ↗
Figure 2
Figure 2. Densities for the conditional distributions [PITH_FULL_IMAGE:figures/full_fig_p024_2.png] view at source ↗
Figure 3
Figure 3. Densities for the conditional distributions [PITH_FULL_IMAGE:figures/full_fig_p024_3.png] view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Densities for the marginals νu, νv and the conditional distributions νu|v , νv|u resulting from the cosine distance with the joint loss. The marginal variances are better approximated, but a bias is introduced in the conditional means; see Theorem 5.6. marginal distrib…
Figure 5
Figure 5. Figure 5: Two-dimensional densities for the distributions [PITH_FULL_IMAGE:figures/full_fig_p025_5.png]
Figure 6
Figure 6. Figure 6: One-dimensional densities for the conditional distributions [PITH_FULL_IMAGE:figures/full_fig_p025_6.png]
Figure 7
Figure 7. Figure 7: Conditional distributions for the Gaussian process [PITH_FULL_IMAGE:figures/full_fig_p026_7.png]
Figure 8
Figure 8. Figure 8: Left: Conditional expectations E[u|v] and E[v|u] with increasing embedding dimension for a fixed realization of v and u, respectively. Right: Squared errors in the conditional expectations in expectation over 104 realizations of the conditioning variables. We observe c…
Figure 9
Figure 9. Figure 9: Conditional expectations with increasing batch size. We observe that the conditional expecta [PITH_FULL_IMAGE:figures/full_fig_p027_9.png]
Figure 10
Figure 10. Figure 10: Left: Conditional expectations E[u|v] and E[v|u] with increasing sample size for a fixed realization of v and u, respectively. Conditional expectations with increasing sample size. Right: Squared errors in the conditional expectations in expectation over 104 realizati…
Figure 11
Figure 11. Figure 11: Predicted probabilities of the image encoder [PITH_FULL_IMAGE:figures/full_fig_p029_11.png]
Figure 12
Figure 12. Figure 12: Sixteen images sampled from the MNIST test dataset according to the weights given by the [PITH_FULL_IMAGE:figures/full_fig_p030_12.png]
Figure 13
Figure 13. Figure 13: Visualization of two paired samples of the Eulerian potential field and Lagrangian trajectory. [PITH_FULL_IMAGE:figures/full_fig_p031_13.png]
Figure 14
Figure 14. Figure 14: Improvement in retrieval accuracy during training of the encoders for the Lagrangian data [PITH_FULL_IMAGE:figures/full_fig_p032_14.png]
Figure 15
Figure 15. Figure 15: Retrieval of a Eulerian potential from a Lagrangian trajectory. [PITH_FULL_IMAGE:figures/full_fig_p032_15.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

41 extracted references · 35 canonical work pages

  1. [1]

    (2017).Deep Learning1

    Bengio, Y., Goodfellow, I., Cour ville, A.et al. (2017).Deep Learning1. MIT press Cambridge, MA, USA

    work page 2017

  2. [2]

    Bennett, A. F. (1992). Inverse methods in physical oceanography. Cambridge university press

    work page 1992

  3. [3]

    Carlsson, M. (2021). von Neumann’s trace inequality for Hilbert–Schmidt operators.Expositiones Mathematicae 39 149–157

    work page 2021

  4. [4]

    and Chilimbi, T

    Chen, C., Zhang, J., Xu, Y., Chen, L., Duan, J., Chen, Y., Tran, S., Zeng, B. and Chilimbi, T. (2022). Why do we need large batchsizes in contrastive learning? A gradient-bias perspective.Advances in Neural Information Processing Systems35 33860–33875

    work page 2022

  5. [5]

    and Hinton, G

    Chen, T., Kornblith, S., Norouzi, M. and Hinton, G. (2020). A simple framework for contrastive learning of visual representations. InInternational Conference on Machine Learning1597–1607. PMLR

    work page 2020

  6. [6]

    and He, K

    Chen, X. and He, K. (2021). Exploring simple Siamese representation learning. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition15750–15758

    work page 2021

  7. [7]

    and Jitsev, J

    Cherti, M., Beaumont, R., Wightman, R., Wortsman, M., Ilharco, G., Gordon, C., Schuh- mann, C., Schmidt, L. and Jitsev, J. (2023). Reproducible Scaling Laws for Contrastive Language- Image Learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion (CVPR)2818–2829

    work page 2023

  8. [8]

    and LeCun, Y

    Chopra, S., Hadsell, R. and LeCun, Y. (2005). Learning a similarity metric discriminatively, with application to face verification. In2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05)1 539–546. IEEE

    work page 2005

Show all 41 references
  1. [9]

    Cotter, S. L. , Dashti, M., Robinson, J. C. and Stuart, A. M. (2009). Bayesian inverse problems for functions and applications to fluid mechanics.Inverse Problems 25 115008

    2009

  2. [10]

    and W ang, H.(2023)

    Elizalde, B., Deshmukh, S., Al Ismail, M. and W ang, H.(2023). Clap learning audio concepts from natural language supervision. InICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)1–5. IEEE

    2023

  3. [11]

    and Chen, Y

    F ang, H., Xiong, P., Xu, L. and Chen, Y. (2021). Clip2video: Mastering video-text retrieval via image clip.arXiv:2106.11097

    2021 arXiv

  4. [12]

    and Torokhti, A

    Friedland, S. and Torokhti, A. (2007). Generalized rank-constrained matrix approximations.SIAM Journal on Matrix Analysis and Applications29 656–659

    2007

  5. [13]

    Gage, P. (1994). A new algorithm for data compression.The C Users Journal12 23–38

    1994

  6. [14]

    Giuseppe Carere, H. C. L. (2024). Optimal low-rank approximations of posteriors for linear Gaussian inverse problems on Hilbert spaces.arXiv:2411.01112. Baptista, Stuart, Tran/Contrastive Learning 34

    2024 arXiv

  7. [15]

    Grill, J.-B., Strub, F., Altché, F., Tallec, C., Richemond, P., Buchatskaya, E., Doersch, C., A vila Pires, B., Guo, Z., Gheshlaghi Azar, M. et al. (2020). Bootstrap your own latent-a new approach to self-supervised learning.Advances in Neural Information Processing Systems33 ...

    2020

  8. [16]

    HaoChen, J. Z. , Wei, C., Gaidon, A. and Ma, T. (2021). Provable guarantees for self-supervised deep learning with spectral contrastive loss.Advances in Neural Information Processing Systems34 5000–5011

    2021

  9. [17]

    and Jones, C

    Ide, K., Kuznetsov, L. and Jones, C. K. (2002). Lagrangian data assimilation for point vortex systems. Journal of Turbulence3 053

    2002

  10. [18]

    and Duerig, T

    Jia, C., Yang, Y., Xia, Y., Chen, Y.-T., Parekh, Z., Pham, H., Le, Q., Sung, Y.-H., Li, Z. and Duerig, T. (2021). Scaling up visual and vision-language representation learning with noisy text supervision. InInternational Conference on Machine Learning4904–4916. PMLR

    2021

  11. [19]

    Joseph, K. J. , Khan, S. , Khan, F. S. and Balasubramanian, V. N. (2021). Towards Open World Object Detection. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)5830–5840

    2021

  12. [20]

    Kirillov, A., Mintun, E., Ra vi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., White- head, S., Berg, A. C. , Lo, W.-Y. et al. (2023). Segment anything. InProceedings of the IEEE/CVF International Conference on Computer Vision4015–4026

    2023

  13. [21]

    and Jones, C

    Kuznetsov, L., Ide, K. and Jones, C. K. (2003). A method for assimilation of Lagrangian data. Monthly Weather Review131 2247–2260

    2003

  14. [22]

    and Haffner, P

    LeCun, Y., Bottou, L., Bengio, Y. and Haffner, P. (1998). Gradient-based learning applied to document recognition.Proceedings of the IEEE86 2278–2324

    1998

  15. [23]

    Liu, H. , Li, C. , Wu, Q. and Lee, Y. J. (2024). Visual instruction tuning. Advances in Neural Information Processing Systems36

    2024

  16. [24]

    and Li, T

    Luo, H., Ji, L., Zhong, M., Chen, Y., Lei, W., Duan, N. and Li, T. (2022). Clip4clip: An empirical study of clip for end to end video clip retrieval and captioning.Neurocomputing 508 293–304

    2022

  17. [25]

    and Zisserman, A

    Miech, A., Alayrac, J.-B., Smaira, L., Laptev, I., Sivic, J. and Zisserman, A. (2020). End-to-end learning of visual representations from uncurated instructional videos. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition9879–9889

    2020

  18. [26]

    and Chen, M

    Nichol, A., Dhariw al, P., Ramesh, A., Shyam, P., Mishkin, P., McGrew, B., Sutskever, I. and Chen, M. (2021). Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv:2112.10741

    2021 arXiv

  19. [27]

    Pardo, L. (2018). Statistical inference based on divergence measures. Chapman and Hall/CRC

    2018

  20. [28]

    , Kim, J

    Radford, A. , Kim, J. W. , Hallacy, C. , Ramesh, A. , Goh, G. , Agar w al, S., Sastry, G., Askell, A., Mishkin, P., Clark, J. et al. (2021). Learning transferable visual models from natural language supervision. InInternational Conference on Machine Learning8748–8763. PMLR

    2021

  21. [29]

    and Chen, M

    Ramesh, A., Dhariw al, P., Nichol, A., Chu, C. and Chen, M. (2022). Hierarchical text-conditional image generation with CLIP latents.arXiv:2204.06125 1 3

    2022 arXiv

  22. [30]

    and Ommer, B

    Rombach, R., Blattmann, A., Lorenz, D., Esser, P. and Ommer, B. (2022). High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition10684–10695

    2022

  23. [31]

    Shi, L., F an, J.and Yan, J. (2024). OT-CLIP: Understanding and Generalizing CLIP via Optimal Transport. InInternational Conference on Machine Learning. PMLR

    2024

  24. [32]

    and Marzouk, Y

    Spantini, A., Solonen, A., Cui, T., Martin, J., Tenorio, L. and Marzouk, Y. (2015). Optimal Low-rank Approximations of Bayesian Linear Inverse Problems.SIAM Journal on Scientific Computing 37 A2451-A2487

    2015

  25. [33]

    Tschannen, M., Djolonga, J., Rubenstein, P. K. , Gelly, S. and Lucic, M. (2019). On mutual information maximization for representation learning.arXiv:1907.13625

    2019 arXiv

  26. [34]

    Von Luxburg, U. (2007). A tutorial on spectral clustering.Statistics and computing17 395–416

    2007

  27. [35]

    and Liao, J

    W ang, C., Chai, M., He, M., Chen, D. and Liao, J. (2022). CLIP-NeRF: Text-and-image driven manipulation of neural radiance fields. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition3835–3844

    2022

  28. [36]

    and Goodman, N

    Wu, M., Zhuang, C., Mosse, M., Yamins, D. and Goodman, N. (2020). On mutual information in contrastive learning for visual representations. InInternational Conference on Learning Representations. Baptista, Stuart, Tran/Contrastive Learning 35

    2020

  29. [37]

    and Wu, Y

    Yu, J., W ang, Z., V asudev an, V., Yeung, L., Seyedhosseini, M. and Wu, Y. (2022). CoCa: Contrastive Captioners are Image-Text Foundation Models.Transactions of Machine Learning Research 2022

    2022

  30. [38]

    , Lin, F

    Yue, Y. , Lin, F. , Yamada, K. D. and Zhang, Z. (2023). Hyperbolic contrastive learning. arXiv:2302.01409

    2023 arXiv

  31. [39]

    and Deny, S

    Zbontar, J., Jing, L., Misra, I., LeCun, Y. and Deny, S. (2021). Barlow twins: Self-supervised learning via redundancy reduction. InInternational Conference on Machine Learning12310–12320. PMLR

    2021

  32. [40]

    and Beyer, L

    Zhai, X., Mustaf a, B., Kolesnikov, A. and Beyer, L. (2023). Sigmoid loss for language image pre- training. InProceedings of the IEEE/CVF International Conference on Computer Vision11975–11986

    2023

  33. [41]

    Ziyin, L., Lubana, E. S. , Ueda, M. and Tanaka, H. (2023). What shapes the loss landscape of self-supervised learning? InThe Eleventh International Conference on Learning Representations. Appendix A: Proofs of Optimization Results Proof of Theorem 3.2.Let ν and µ be measures, ...

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.