Pith. sign in

REVIEW 4 major objections 4 minor 52 references

A Principled Framework for Multi-View Contrastive Learning

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper proves that two new multi-view contrastive losses share InfoNCE's asymptotic optimum — perfect alignment plus uniform spread on the sphere — and shows one of them escapes dimension collapse at five or more views.

desk verdict Promising losses and solid experiments, but the main theorem is wrong and Table I oversells MV-DHEL; deserves review but needs major revision. read the letter →

arxiv 2507.06979 v1 pith:562ZNLCN submitted 2025-07-09 cs.LG cs.CV

Panagiotis Koromilas , Efthymios Georgiou , Giorgos Bouritsas , Theodoros Giannakopoulos , Mihalis A. Nicolaou , Yannis Panagakis This is my paper
classification cs.LGcs.CV
keywords multi-viewcontrastivelearningMV-InfoNCEMV-DHELalignmentanduniformitydimensionalitycollapseself-supervisedrepresentationmultimodalsentimentanalysishypersphere
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Multi-view contrastive learning is normally done by adding up pairwise losses, one for each pair of views, but the paper argues this stacks conflicting goals onto the same representation and misses interactions that only show up between three or more views at once. The authors propose two losses—MV-InfoNCE and MV-DHEL—that treat all views of a data point in a single optimization term, and they prove that both converge, as the number of negatives grows without bound, to the same asymptotic objective as standard two-view InfoNCE: perfect alignment of every view of an instance and a uniform spread of representations over the unit sphere. If the theorem and experiments hold, the practical payoff is that more views become an asset rather than a burden—more accurate linear probes and k-nearest-neighbor evaluations than pairwise baselines, stable training across batch sizes, and a representation space that uses all its dimensions once five or more views are supplied. The paper reports that MV-DHEL, the variant that separates alignment from uniformity, is the one that delivers this collapse resistance.

What carries the argument

The load-bearing construction is a framework that writes any contrastive loss as an alignment term plus a uniformity (energy) term in kernel notation, $K(u,v) = \kappa(\|u-v\|^2)$ with the Gaussian kernel, and pins down every multi-view variant by three choices: where the summation over the view of interest $l$ sits (inside or outside the logarithm), the positive set $P(l)$, and the negative set $N(i,l)$. Summing inside the logarithm gives one term per data instance (P3); a positive set containing all other views enforces simultaneous alignment (P1); a negative set whose interactions span the full point configuration yields the correct energy term (P2). MV-InfoNCE is obtained by summing inside the log with $P(l) = [N]\setminus\{l\}$ and $N(i,l) = \{(j,m): j \in [M], m \in [N]\setminus\{l\}\}$; MV-DHEL sums inside the log for alignment but outside the log for uniformity, with negatives restricted to the same view of other instances, $N(i,l) = \{(j,m): j \in [M]\setminus\{i\}, m = l\}$, which removes the alignment-uniformity coupling. Theorem IV.1 then shows, via Jensen's inequality on the alignment term and the law of large numbers plus dominated convergence on the uniformity terms, that both expected losses converge to the InfoNCE asymptotic formula $\mathbb{E}[-\langle u,v\rangle/\tau] + \mathbb{E}[\log \mathbb{E}[e^{\langle u,v\rangle/\tau}]]$ after subtracting $O(\log M)$ normalizing constants.

What would settle it

Run the paper's CIFAR-10 protocol with MV-DHEL at 3, 5, and 8 views and measure the rank of the 128-dimensional embedding matrix: the claim that five or more views fully utilize the embedding space predicts the rank climbing to near 128, so a rank that stalls well below that (or accuracy that drops from 4 to 8 views) refutes the collapse claim. Separately, in a huge-batch regime approximating the infinite-negative limit, the theorem predicts MV-InfoNCE and MV-DHEL can reach the same alignment-uniformity frontier; if coupling visibly caps MV-InfoNCE's uniformity there, the asymptotic equivalence is not describing the trained regime.

Watch

Extended reading notes

Core claim

Statement the authors would make to a fair reader: contrastive learning's goal is representable by two complementary forces—pulling the views of one instance together (alignment) and spreading different instances apart (uniformity)—and this goal can be preserved exactly when moving from two views to many, provided the loss is built with three design principles: P1, every view of an instance must be aligned within a single term; P2, the uniformity term must contain the complete set of pairwise interactions (a proper energy on the point configuration); P3, one optimization term per data point. The paper shows that pairwise aggregation (pwe, avg, PVC) violates all three—each representation is pulled by several competing alignment forces and separate uniformity terms—and constructs MV-InfoNCE and MV-DHEL to satisfy them. Theorem IV.1 states that after subtracting normalizing constants, the expectations of the two new losses have the same asymptotic behavior as the two-view InfoNCE objective, so the same minimizer—an encoder achieving perfect alignment and uniformity, if one exists in the function class—is the unique optimum. Empirically, both losses outperform the pairwise baselines on CIFAR-10, CIFAR-100, ImageNet-100, and ImageNet-1K with 2–4 views and on the three-modality sentiment datasets CMU-MOSEI and CH-SIMS, and MV-DHEL's embedding rank rises with view count to fill the full 128-dimensional space at five or more views, which the authors take as the multi-view benefit that supervised learning already enjoys.

Load-bearing premise

The optimality proof assumes an encoder exists in the practical model family that can simultaneously put every view of an instance at the same point and spread all instances uniformly over the unit sphere; if no trained network actually reaches both, the theorem describes a target that the learned representations only approximate.

Editorial extensions

If this is right

  • The two losses inherit InfoNCE's theoretical guarantee: if a perfectly aligned and uniform encoder exists in the model family, it is the unique minimizer, so adding more views does not move the target optimum—it only changes how reachable that optimum is during training.
  • MV-DHEL's decoupled uniformity means each added view contributes alignment constraints without multiplying conflicting uniformity gradients, which the paper ties to its stable performance across batch sizes and its growing embedding rank with view count.
  • Because both objectives take one term per data point and capture all view interactions, they extend naturally beyond two modalities; the paper reports gains on three-modality sentiment analysis where pairwise contrastive baselines degrade.
  • The batch-size robustness of MV-DHEL supports a practical recipe: hold the effective batch constant, reduce the number of unique instances, and spend the saved memory on more views per instance, gaining accuracy without extra memory cost.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The three principles (P1–P3) are stated for contrastive objectives, but they read as a general checklist for multi-view self-supervised design; a testable extension is that positive-only methods, which have no negatives at all, could adopt the outside-the-log uniformity structure and gain the same collapse resistance.
  • Since Theorem IV.1 gives both new losses the same global optimum, the meaningful difference between them is the optimization geometry; a concrete prediction is that MV-DHEL's margin over MV-InfoNCE widens precisely in regimes where alignment-uniformity coupling bites hardest (small batches, high temperature, many views).
  • MV-DHEL's per-view uniformity regularizes each modality's geometry separately, which may explain its large gains on text-dominated multimodal benchmarks; the paper reports the results but does not isolate this mechanism.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This paper proposes a framework for multi-view contrastive learning grounded in three principles: simultaneous alignment of all views (P1), an accurate energy/uniformity term with complete pairwise interactions (P2), and one optimization term per data instance (P3). Based on these principles the authors introduce two losses: MV-InfoNCE (Eq. 11), a single-term generalization of InfoNCE over all views, and MV-DHEL (Eq. 12), which decouples alignment from uniformity. Theorem IV.1 claims that both losses have the same asymptotic behavior as InfoNCE, converging to Eq. (15) after subtracting normalizing constants, and that perfect alignment plus uniformity is the only minimizer. The paper reports experiments on CIFAR-10/100, ImageNet-100/1K, and two multimodal sentiment datasets, with ablations on view multiplicity, batch size, dimensionality collapse, and memory overhead, and it releases code.

Significance. If the theoretical claims are corrected, this would be a useful contribution to multi-view contrastive learning: the three-principle framework is conceptually clean, the two losses are natural extensions of InfoNCE and DHEL, and the empirical study is broad, including ImageNet-1K, multimodal data, and rank-based collapse analysis. The release of code strengthens reproducibility. However, the central theoretical guarantee, Theorem IV.1, is not established as stated: the appendix contains an invalid log-of-sum step for MV-InfoNCE, a Jensen inequality applied in the wrong direction, and for MV-DHEL the derived asymptotic uniformity term carries an extra factor N that contradicts Eq. (15). The minimizer conclusion may still be salvageable with a corrected proof, but the theorem statement and its proof require substantial revision.

major comments (4)
  1. [Theorem IV.1 and Appendix proof of Γ] The claimed convergence of the MV-DHEL expectation to Eq. (15) is false. For the uniformity part of Eq. (14), the appendix correctly derives Γ = N E_{v~f#p_trans} log( (1/(M-1)) Σ_{j=1}^{M-1} e^{v^T u_j/τ} ), whose M→∞ limit is N times the InfoNCE uniformity term E_v log E_u e^{v^T u/τ}. No subtractive constant can remove this factor of N; for N=2 the loss reduces to DHEL, whose known asymptotic limit has uniformity weight 2, not 1. The theorem should be restated as a claim about the minimizers (perfect alignment and uniformity) rather than about convergence to Eq. (15).
  2. [Appendix, proof of Theorem IV.1, 'Regarding B'] The step replacing E_{U_1} log( (N-1) Σ_{l=1}^N E_u e^{U_{1,l}^T u/τ} ) with E_v log( N(N-1) E_u e^{v^T u/τ} ) is invalid: the logarithm of a sum of N random variables is not equal to the logarithm of a single variable plus a constant unless the variables are equal. Consequently, the asymptotic limit of E1 in Eq. (13) is not Eq. (15) in general; it depends on the joint distribution of the N views of an instance, not only on the marginal distribution.
  3. [Appendix, proof of Theorem IV.1, 'Regarding A'] The Jensen inequality used for the alignment term has the wrong direction: log-concavity gives E log( (1/n) Σ e^{x_i} ) ≥ E (1/n) Σ log e^{x_i} = E (1/n) Σ x_i, while the proof uses ≤. The intended bound -A ≤ 1/τ can be obtained directly from the fact that each exponent is at most 1/τ, but the written derivation is invalid.
  4. [Table I and Eq. (12)] The claim that MV-DHEL satisfies P3 (one term per data instance) is inconsistent with Eq. (12): the uniformity part is (1/M) Σ_{l=1}^N Σ_i log Σ_{j≠i} K(U_{i,l}, U_{j,l}), which contributes N separate uniformity terms per instance in addition to the alignment term. Under the same counting used for pwe and PVC in Table I, MV-DHEL should be marked as having more than one term per instance, and the paper should either modify the definition of P3 or the loss itself.
minor comments (4)
  1. [Appendix, Eqs. (18) and (20)] The MV-DHEL denominator in Eq. (18) contains e^{U^T_{i,l} U_{j,m}/τ} with an undefined index m; it should be l, i.e., U_{j,l}. The same issue appears in Eq. (20).
  2. [Section V-C6] The sentence 'Section V-C6 further confirms that MV-DHEL maintains stable performance...' is a self-reference; it should refer to Table V or to the preceding paragraph.
  3. [Throughout] There are several typographical errors, including 'adressing' (Section I), 'instroduces' (Section IV.A.2), 'alginment' (Section IV.C), and 'freezed' (Section V.A.1); a careful copyedit is needed.
  4. [Theorem IV.1] The theorem statement should explicitly state the normalizing constants that are subtracted (log(M-1) and log(N(N-1)) in Eqs. (24)-(25)), and it should discuss the existence assumption for an encoder that simultaneously achieves perfect alignment and uniformity, since the minimizer claim is conditional on that assumption.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the claimed multi-view asymptotic results are verified from the loss definitions, with an external result supplying the uniformity-minimizer step; the authors' prior DHEL work is used as a building block, not as a forced conclusion.

full rationale

The paper's derivation chain is: (i) decompose contrastive losses into alignment and uniformity energies following Wang and Isola [1]; (ii) construct MV-InfoNCE and MV-DHEL so that the placement of view-summation inside/outside the logarithm enforces principles P1-P3; (iii) prove in Theorem IV.1 that, after normalizing constants, the expected losses share InfoNCE's asymptotic alignment and uniformity behavior. Step (iii) is a direct mathematical verification of the constructed functional forms, not a relabeling of a fitted quantity as a prediction. The uniformity-minimizer conclusion is explicitly inherited from Wang and Isola [1], which is an external, independently published result, not from the authors' own prior work. Citations to the authors' prior DHEL paper [11] support the base two-view loss, kernel notation, decoupling intuition, and batch-size robustness; these are building blocks, while the multi-view objectives, the ImageNet/CIFAR/multimodal experiments, and the dimensionality-collapse rank measurements are all evaluated in this paper. The appendix's log-of-sum step, and the extra factor N in the MV-DHEL uniformity limit, are potential mathematical-correctness problems in the theorem as stated; however, a false or unproved theorem does not reduce the claimed result to its own inputs, so it is not a circularity under the specified patterns. No fitted input is disguised as a prediction, and no load-bearing argument reduces to a self-citation chain.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

No new physical or mathematical entities are postulated. The central derivation uses standard asymptotic tools and the conditional existence of a perfectly aligned and uniform encoder, inherited from Wang and Isola. The temperature is a standard hyperparameter, identical across compared methods and irrelevant to the asymptotic optimum. The main unstated assumption is the i.i.d. multi-view sampling model that the batch expectation is taken over.

free parameters (1)
  • temperature tau = 0.5 (CIFAR-10/100, ImageNet-100); 0.1 (ImageNet-1K)
    Standard contrastive temperature hyperparameter, identical for all compared methods; the asymptotic theory holds for any positive tau, so it is not fitted to the reported result.
assumptions (4)
  • standard math Law of large numbers, continuous mapping theorem, and dominated convergence theorem can be interchanged with expectations in the batch limit M to infinity.
    Used in the Appendix proof of Theorem IV.1 to derive the asymptotic formula for both proposed losses.
  • domain assumption A perfectly aligned and perfectly uniform encoder exists within the parameterized function class.
    Theorem IV.1's conclusion is conditional on this premise ('if perfectly uniform encoders exist, they form the exact minimizers'); the paper does not verify existence for ResNet architectures.
  • domain assumption Views of a data point are generated as independent samples from a fixed transformation distribution p_T, giving the product structure p(x_1,...,x_N) = integral p_init product p_T.
    Defined in Section III-B and Appendix D; the asymptotic derivation of the uniformity term relies on this factorization.
  • domain assumption Embeddings lie on the unit sphere and similarity is measured by the Gaussian kernel e^{u^T v / tau}.
    Standard InfoNCE setup from Section III-B and Appendix A; the kernel conditions are assumed to hold.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Principled Framework for Multi-View Contrastive Learning." pith.science (2026). https://pith.science/paper/562ZNLCN

@misc{pith2026250706979,
  author       = {Pith},
  title        = {Pith review of: A Principled Framework for Multi-View Contrastive Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/562ZNLCN}},
  note         = {Machine review of arXiv:2507.06979}
}
read the original abstract

Contrastive Learning (CL), a leading paradigm in Self-Supervised Learning (SSL), typically relies on pairs of data views generated through augmentation. While multiple augmentations per instance (more than two) improve generalization in supervised learning, current CL methods handle additional views suboptimally by simply aggregating different pairwise objectives. This approach suffers from four critical limitations: (L1) it utilizes multiple optimization terms per data point resulting to conflicting objectives, (L2) it fails to model all interactions across views and data points, (L3) it inherits fundamental limitations (e.g. alignment-uniformity coupling) from pairwise CL losses, and (L4) it prevents fully realizing the benefits of increased view multiplicity observed in supervised settings. We address these limitations through two novel loss functions: MV-InfoNCE, which extends InfoNCE to incorporate all possible view interactions simultaneously in one term per data point, and MV-DHEL, which decouples alignment from uniformity across views while scaling interaction complexity with view multiplicity. Both approaches are theoretically grounded - we prove they asymptotically optimize for alignment of all views and uniformity, providing principled extensions to multi-view contrastive learning. Our empirical results on ImageNet1K and three other datasets demonstrate that our methods consistently outperform existing multi-view approaches and effectively scale with increasing view multiplicity. We also apply our objectives to multimodal data and show that, in contrast to other contrastive objectives, they can scale beyond just two modalities. Most significantly, ablation studies reveal that MV-DHEL with five or more views effectively mitigates dimensionality collapse by fully utilizing the embedding space, thereby delivering multi-view benefits observed in supervised learning.

Figures

Figures reproduced from arXiv: 2507.06979 by the authors.

Figure 1
Figure 1. Alignment and uniformity optimisation in different objectives. This figure illustrates how three CL methods, NT￾Xent, our MV-InfoNCE, and our MV-DHEL, optimise data representations. Given a representation Ui with different views Ui,l, we visualize optimal interactions only for the loss term associated with Ui . Green lines indicate positive interactions (alignment), while red lines represent negative interactions (u… view at source ↗
Figure 2
Figure 2. , consider the representation Ui,1,: of instance i in view 1. Under pairwise aggregation, this single representation must align with both Ui,2,: (from the 1→2 loss term) and Ui,3,: (from the 1→3 loss term), while simultaneously maintaining uniformity with negative samples in view 2 and view 3 through their respective loss terms. These objectives generate competing gradient signals: ∇Ui,1,:Lpair([U:,1,: , U:,2,: ]) a… view at source ↗
Figure 3
Figure 3. Properties vs view multiplicity calculated on CIFAR10 (top) & CIFAR100 (bottom) dataset [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Top: Performance vs batch size for 4 views; Bottom: [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

52 extracted references · 43 canonical work pages

  1. [1]

    Understanding contrastive representation learning through alignment and uniformity on the hypersphere,

    T. Wang and P. Isola, “Understanding contrastive representation learning through alignment and uniformity on the hypersphere,” in International Conference on Machine Learning (ICML) . PMLR, 2020, pp. 9929– 9939

    work page 2020

  2. [2]

    Learning representations by maximizing mutual information across views,

    B. Philip, H. R. Devon, B. William et al. , “Learning representations by maximizing mutual information across views,” Advances in neural information processing systems , vol. 32, pp. 15 535–15 545, 2019

    work page 2019

  3. [3]

    Drawing multiple augmentation samples per image during training efficiently decreases test error,

    S. Fort, A. Brock, R. Pascanu, S. De, and S. L. Smith, “Drawing multiple augmentation samples per image during training efficiently decreases test error,” arXiv preprint arXiv:2105.13343 , 2021

    arXiv 2021

  4. [4]

    Augment your batch: Improving generalization through instance repeti- tion,

    E. Hoffer, T. Ben-Nun, I. Hubara, N. Giladi, T. Hoefler, and D. Soudry, “Augment your batch: Improving generalization through instance repeti- tion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 8129–8138

    work page 2020

  5. [5]

    The good, the bad and the ugly sides of data augmentation: An implicit spectral regularization perspective,

    C.-H. Lin, C. Kaushik, E. L. Dyer, and V . Muthukumar, “The good, the bad and the ugly sides of data augmentation: An implicit spectral regularization perspective,” Journal of Machine Learning Research , vol. 25, no. 91, pp. 1–85, 2024

    work page 2024

  6. [6]

    Unsupervised learning of visual features by contrasting cluster assign- ments,

    M. Caron, I. Misra, J. Mairal, P. Goyal, P. Bojanowski, and A. Joulin, “Unsupervised learning of visual features by contrasting cluster assign- ments,” Advances in neural information processing systems , vol. 33, pp. 9912–9924, 2020

    work page 2020

  7. [7]

    Emerging properties in self-supervised vision transformers,

    M. Caron, H. Touvron, I. Misra, H. J ´egou, J. Mairal, P. Bojanowski, and A. Joulin, “Emerging properties in self-supervised vision transformers,” in Proceedings of the International Conference on Computer Vision (ICCV), 2021

    2021

  8. [8]

    Vicregl: Self-supervised learning of local visual features,

    A. Bardes, J. Ponce, and Y . LeCun, “Vicregl: Self-supervised learning of local visual features,” Advances in Neural Information Processing Systems, vol. 35, pp. 8799–8810, 2022

    work page 2022

Show all 52 references
  1. [9]

    Contrastive multiview coding,

    Y . Tian, D. Krishnan, and P. Isola, “Contrastive multiview coding,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XI 16 . Springer, 2020, pp. 776–794

    2020

  2. [10]

    Poly-view contrastive learning,

    A. Shidani, R. D. Hjelm, J. Ramapuram, R. Webb, E. G. Dhekane, and D. Busbridge, “Poly-view contrastive learning,” in The Twelfth International Conference on Learning Representations , 2024. [Online]. Available: https://openreview.net/forum?id=iHcTLIor0m

    2024

  3. [11]

    Bridging mini-batch and asymptotic analysis in contrastive learning: From infoNCE to kernel-based losses,

    P. Koromilas, G. Bouritsas, T. Giannakopoulos, M. Nicolaou, and Y . Panagakis, “Bridging mini-batch and asymptotic analysis in contrastive learning: From infoNCE to kernel-based losses,” in Forty- first International Conference on Machine Learning , 2024. [Online]. Available: ...

    2024

  4. [12]

    Understanding dimensional collapse in contrastive self-supervised learning,

    L. Jing, P. Vincent, Y . LeCun, and Y . Tian, “Understanding dimensional collapse in contrastive self-supervised learning,” in International Conference on Learning Representations , 2022. [Online]. Available: https://openreview.net/forum?id=YevsQ05DEN7

    2022

  5. [13]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al., “Learning transferable visual models from natural language supervision,” in International conference on machine learning . PmLR, 2021, pp. 8748–8763

    2021

  6. [14]

    Tricolo: Trimodal contrastive loss for text to shape retrieval,

    Y . Ruan, H.-H. Lee, Y . Zhang, K. Zhang, and A. X. Chang, “Tricolo: Trimodal contrastive loss for text to shape retrieval,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , 2024, pp. 5815–5825

    2024

  7. [15]

    Contrastive multimodal fusion with tupleinfonce,

    Y . Liu, Q. Fan, S. Zhang, H. Dong, T. Funkhouser, and L. Yi, “Contrastive multimodal fusion with tupleinfonce,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2021, pp. 754–763

    2021

  8. [16]

    Contextual augmented global contrast for multimodal intent recognition,

    K. Sun, Z. Xie, M. Ye, and H. Zhang, “Contextual augmented global contrast for multimodal intent recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 26 963–26 973

    2024

  9. [17]

    Learning a similarity metric discriminatively, with application to face verification,

    S. Chopra, R. Hadsell, and Y . LeCun, “Learning a similarity metric discriminatively, with application to face verification,” in 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), vol. 1. IEEE, 2005, pp. 539–546

    2005

  10. [18]

    Improved deep metric learning with multi-class n-pair loss objective,

    K. Sohn, “Improved deep metric learning with multi-class n-pair loss objective,” Advances in neural information processing systems , vol. 29, 2016

    2016

  11. [19]

    Representation learning with contrastive predictive coding,

    A. v. d. Oord, Y . Li, and O. Vinyals, “Representation learning with contrastive predictive coding,” arXiv preprint arXiv:1807.03748 , 2018

    2018 arXiv

  12. [20]

    A simple framework for contrastive learning of visual representations,

    T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” in International conference on Machine Learning (ICML) . PMLR, 2020, pp. 1597– 1607

    2020

  13. [21]

    With a little help from my friends: Nearest-neighbor contrastive learning of visual representations,

    D. Dwibedi, Y . Aytar, J. Tompson, P. Sermanet, and A. Zisserman, “With a little help from my friends: Nearest-neighbor contrastive learning of visual representations,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2021, pp. 9588–9597

    2021

  14. [22]

    Decoupled contrastive learning,

    C.-H. Yeh, C.-Y . Hong, Y .-C. Hsu, T.-L. Liu, Y . Chen, and Y . LeCun, “Decoupled contrastive learning,” in European Conference on Computer Vision. Springer, 2022, pp. 668–684

    2022

  15. [23]

    Momentum contrast for unsupervised visual representation learning,

    K. He, H. Fan, Y . Wu, S. Xie, and R. Girshick, “Momentum contrast for unsupervised visual representation learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 9729–9738

    2020

  16. [24]

    Contrastive learning with hard negative samples,

    J. D. Robinson, C. Chuang, S. Sra, and S. Jegelka, “Contrastive learning with hard negative samples,” in 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. 12

    2021

  17. [25]

    On feature decorrelation in self-supervised learning,

    T. Hua, W. Wang, Z. Xue, S. Ren, Y . Wang, and H. Zhao, “On feature decorrelation in self-supervised learning,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2021, pp. 9598–9608

    2021

  18. [26]

    Understanding dimensional collapse in contrastive self-supervised learning,

    L. Jing, P. Vincent, Y . LeCun, and Y . Tian, “Understanding dimensional collapse in contrastive self-supervised learning,” in The Tenth Inter- national Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022 . OpenReview.net, 2022

    2022

  19. [27]

    How much data are augmentations worth? an investigation into scaling laws, invariance, and implicit regularization,

    J. Geiping, M. Goldblum, G. Somepalli, R. Shwartz-Ziv, T. Goldstein, and A. G. Wilson, “How much data are augmentations worth? an investigation into scaling laws, invariance, and implicit regularization,” in The Eleventh International Conference on Learning Representations , 2...

    2023

  20. [28]

    Grounding inductive biases in natural images: invariance stems from variations in data,

    D. Bouchacourt, M. Ibrahim, and A. Morcos, “Grounding inductive biases in natural images: invariance stems from variations in data,” Ad- vances in Neural Information Processing Systems , vol. 34, pp. 19 566– 19 579, 2021

    2021

  21. [29]

    Emp-ssl: Towards self-supervised learning in one training epoch,

    S. Tong, Y . Chen, Y . Ma, and Y . Lecun, “Emp-ssl: Towards self-supervised learning in one training epoch,” arXiv preprint arXiv:2304.03977, 2023

    2023 arXiv

  22. [30]

    Whitening for self-supervised representation learning,

    A. Ermolov, A. Siarohin, E. Sangineto, and N. Sebe, “Whitening for self-supervised representation learning,” in International Conference on Machine Learning. PMLR, 2021, pp. 3015–3024

    2021

  23. [31]

    Adaptive multi-head contrastive learning,

    L. Wang, P. Koniusz, T. Gedeon, and L. Zheng, “Adaptive multi-head contrastive learning,” in European Conference on Computer Vision . Springer, 2024, pp. 404–421

    2024

  24. [32]

    From global to local: Multi-patch and multi-scale contrastive similarity learning for unsupervised defocus blur detection,

    J. Li, B. Liang, X. Lu, M. Li, G. Lu, and Y . Xu, “From global to local: Multi-patch and multi-scale contrastive similarity learning for unsupervised defocus blur detection,” IEEE Transactions on Image Processing, vol. 32, pp. 1158–1169, 2023

    2023

  25. [33]

    Multi- view action recognition using contrastive learning,

    K. Shah, A. Shah, C. P. Lau, C. M. de Melo, and R. Chellappa, “Multi- view action recognition using contrastive learning,” in Proceedings of the ieee/cvf winter conference on applications of computer vision , 2023, pp. 3381–3391

    2023

  26. [34]

    Multi-level feature learning for contrastive multi-view clustering,

    J. Xu, H. Tang, Y . Ren, L. Peng, X. Zhu, and L. He, “Multi-level feature learning for contrastive multi-view clustering,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 16 051–16 060

    2022

  27. [35]

    Contrasting multiple representations with the multi-marginal matching gap,

    Z. Piran, M. Klein, J. Thornton, and marco cuturi, “Contrasting multiple representations with the multi-marginal matching gap,” in Forty-first International Conference on Machine Learning , 2024. [Online]. Available: https://openreview.net/forum?id=dV9B9qFeGi

    2024

  28. [36]

    Bootstrap your own latent-a new approach to self-supervised learning,

    J.-B. Grill, F. Strub, F. Altch ´e, C. Tallec, P. Richemond, E. Buchatskaya, C. Doersch, B. Avila Pires, Z. Guo, M. Gheshlaghi Azar et al. , “Bootstrap your own latent-a new approach to self-supervised learning,” Advances in neural information processing systems, vol. 33, pp. ...

    2020

  29. [37]

    Fastsiam: Resource- efficient self-supervised learning on a single gpu,

    D. Pototzky, A. Sultan, and L. Schmidt-Thieme, “Fastsiam: Resource- efficient self-supervised learning on a single gpu,” in DAGM German Conference on Pattern Recognition . Springer, 2022, pp. 53–67

    2022

  30. [38]

    Unsupervised feature learning by cross- level instance-group discrimination,

    X. Wang, Z. Liu, and S. X. Yu, “Unsupervised feature learning by cross- level instance-group discrimination,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2021, pp. 12 586–12 595

    2021

  31. [39]

    Dual temperature helps contrastive learning without many negative samples: Towards understanding and simplifying moco,

    C. Zhang, K. Zhang, T. X. Pham, A. Niu, Z. Qiao, C. D. Yoo, and I. S. Kweon, “Dual temperature helps contrastive learning without many negative samples: Towards understanding and simplifying moco,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogn...

    2022

  32. [40]

    Unsupervised feature learning via non-parametric instance discrimination,

    Z. Wu, Y . Xiong, S. X. Yu, and D. Lin, “Unsupervised feature learning via non-parametric instance discrimination,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 3733– 3742

    2018

  33. [41]

    Multizoo & multibench: A standardized toolkit for multimodal deep learning,

    P. P. Liang, Y . Lyu, X. Fan, A. Agarwal, Y . Cheng, L.-P. Morency, and R. Salakhutdinov, “Multizoo & multibench: A standardized toolkit for multimodal deep learning,” Journal of Machine Learning Research , vol. 24, pp. 1–7, 2023

    2023

  34. [42]

    M-sena: An integrated platform for multimodal sentiment analysis,

    H. Mao, Z. Yuan, H. Xu, W. Yu, Y . Liu, and K. Gao, “M-sena: An integrated platform for multimodal sentiment analysis,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: System Demonstrations , 2022, pp. 204–213

    2022

  35. [43]

    Multimodal language analysis in the wild: CMU-MOSEI dataset and interpretable dynamic fusion graph,

    A. Bagher Zadeh, P. P. Liang, S. Poria, E. Cambria, and L.-P. Morency, “Multimodal language analysis in the wild: CMU-MOSEI dataset and interpretable dynamic fusion graph,” in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Lo...

    2018

  36. [44]

    Ch- sims: A chinese multimodal sentiment analysis dataset with fine-grained annotation of modality,

    W. Yu, H. Xu, F. Meng, Y . Zhu, Y . Ma, J. Wu, J. Zou, and K. Yang, “Ch- sims: A chinese multimodal sentiment analysis dataset with fine-grained annotation of modality,” in Proceedings of the 58th annual meeting of the association for computational linguistics , 2020, pp. 3718–3727

    2020

  37. [45]

    Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition,

    T. M. Cover, “Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition,” IEEE transactions on electronic computers , no. 3, pp. 326–334, 1965

    1965

  38. [46]

    Rankme: Assessing the downstream performance of pretrained self-supervised representations by their rank,

    Q. Garrido, R. Balestriero, L. Najman, and Y . Lecun, “Rankme: Assessing the downstream performance of pretrained self-supervised representations by their rank,” in International Conference on Machine Learning. PMLR, 2023, pp. 10 929–10 974

    2023

  39. [47]

    Merbench: A unified evaluation benchmark for multimodal emotion recognition,

    Z. Lian, L. Sun, Y . Ren, H. Gu, H. Sun, L. Chen, B. Liu, and J. Tao, “Merbench: A unified evaluation benchmark for multimodal emotion recognition,” arXiv preprint arXiv:2401.03429 , 2024

    2024 arXiv

  40. [48]

    BERT: Pre- training of deep bidirectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre- training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolog...

    2019

  41. [49]

    Hubert: Self-supervised speech representation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “Hubert: Self-supervised speech representation learning by masked prediction of hidden units,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 3451–3460, 2021

    2021

  42. [50]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” in International Conference on Learning...

    2021

  43. [51]

    Decoupled weight decay regularization,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” in International Conference on Learning Representations , 2019. [Online]. Available: https://openreview.net/forum?id=Bkg6RiCqY7 13 APPENDIX ADDITIONAL PRELIMINARIES AND TECHNICAL DETAILS A. Kernels Minimizin...

    2019

  44. [52]

    for all experiments. f) CH-SIMS and CMU-MOSEI: A three layer transformer encoder is employed for each individual modality (unimodal encoder), followed by a late fusion concatenation operation, and a linear projection. This architecture is employed as the multimodal encoder arc...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.