Pith. sign in

REVIEW 2 major objections 6 minor 56 references

Clustering Properties of Self-Supervised Learning

T0 review · 2 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that the encoder's output is the most clusterable component of a self-supervised model, and that feeding that clustering back as a soft assignment target in the loss — ReSA — improves SSL across classification, transfer…

desk verdict ReSA is a genuinely new SSL objective with solid diagnostics, but the headline gains aren't matched and the self-clustering mechanism isn't isolated from augmentation and eval choices. read the letter →

arxiv 2501.18452 v2 pith:2J6GN73A submitted 2025-01-30 cs.LG cs.AI

Xi Weng , Jianing An , Xudong Ma , Binhang Qi , Jie Luo , Xi Yang , Jin Song Dong , Lei Huang This is my paper · ORCID
classification cs.LGcs.AI
keywords self-supervisedlearningjointembeddingarchitecturesrepresentationself-assignmentSinkhorn-Knoppclusteringpropertiescontrastivepositive-feedbackfine-grainedandcoarse-grainedrepresentations
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that self-supervised learning models already contain a hidden asset: the encoder output, or encoding, has better and more stable clustering structure than the embedding the loss usually sees. It establishes this with clustering metrics across several standard SSL models, then turns the observation into a training method. ReSA computes a doubly stochastic assignment matrix from the encodings of one view and uses it as a soft target for aligning the embeddings of another view, instead of the hard identity target of InfoNCE. On CIFAR, ImageNet-100, ImageNet, COCO transfer, fine-grained, and low-shot benchmarks, the paper reports that ReSA outperforms prior SSL methods and produces representations that separate both fine-grained and coarse-grained clusters. The point is that self-supervision can bootstrap itself from its own clustering geometry rather than treating every image pair as unrelated.

What carries the argument

The load-bearing object is the self-assignment matrix $A_H = \mathrm{Sinkhorn}(H^\top H)$: the encoder's $L_2$-normalized batch outputs are simultaneously the points to be clustered and the anchors, and Sinkhorn-Knopp converts their cosine self-similarity into a doubly stochastic matrix. The matrix acts as a soft target in a cross-entropy loss between embeddings $Z$ and $Z'$, so the model is trained to make cross-view similarities follow its own current cluster geometry. Because $A_H = I$ recovers InfoNCE exactly, the design is a strict generalization of contrastive learning; because the diagonal of $A_H$ is sharply larger than off-diagonal entries, the optimization initially behaves like invariant alignment, which the paper argues prevents collapse and early clustering errors.

What would settle it

Train ReSA on a standard dataset with the self-assignment matrix forced to the identity while keeping everything else identical; if this configuration still matches ReSA's reported accuracy, the clustering signal is not the driver. Conversely, compute the assignment matrix from a randomly initialized encoder, measure whether its off-diagonal entries agree with true class labels, and ablate those off-diagonal entries during training; if they carry no class information and their removal does not change performance, the paper's proposed mechanism is unsupported.

Watch

Extended reading notes

Core claim

ReSA replaces the identity matrix in the InfoNCE objective with the online self-assignment matrix $A_H = \mathrm{Sinkhorn}(H^\top H)$, computed without gradients from $L_2$-normalized encoder outputs $H$, and minimizes $\ell_{\mathrm{ReSA}} = -\frac{1}{2m}\sum_{i,j} A_H \circ \log D(Z^\top Z')$ plus its symmetric term. The matrix is doubly stochastic and its diagonal dominates because each sample is most similar to itself, which keeps early training focused on pulling augmented views of the same image together while avoiding hard negative repulsion between same-class samples. The paper shows the gradient with respect to any similarity $s_{i,j}$ is $\frac{1}{\tau}(P_{i,j} - A_H(i,j))$, so ReSA smoothly up-weights or down-weights every pair instead of splitting similarities into positive and negative regimes. With this single change ReSA is reported to beat contrastive and clustering SSL methods on linear and k-NN classification, COCO detection and segmentation, fine-grained transfer, and low-shot evaluation, and the learned representations show cleaner fine- and coarse-grained cluster structure.

Load-bearing premise

The method assumes that the encoder's output already carries a clustering structure stable and informative enough to serve as a training target, from random initialization onward; if early self-assignments are effectively identity matching, ReSA reduces to InfoNCE and the reported gains would disappear.

Editorial extensions

If this is right

  • Pretraining can be improved without labels, new data, extra prototypes, or hand-designed negative pairs, by using the encoder's own cluster assignments as the training target.
  • Because same-class negatives are no longer pushed apart as hard, the embedding space preserves semantic clusters, explaining the reported gains on k-NN classification and fine-grained transfer.
  • The self-assignment is computed once per iteration regardless of the number of views, so multi-view training with this loss is cheaper than prototype-based clustering methods.
  • Weak augmentation on the clustering view is enough, so ReSA points toward simpler augmentation pipelines in large-scale pretraining.
  • The loss family parameterized by how far $A_H$ is from identity interpolates between InfoNCE and self-clustering, giving a controlled way to test how much cluster structure helps.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the diagonal-dominance mechanism predicts ReSA's advantage should shrink with very small batch sizes, because the self-similarity matrix becomes so small that off-diagonal structure carries little semantic information; the paper does not directly test this regime.
  • Editorial extension: because only the cross-entropy form is tested, the same self-assignment target could plausibly be appended to other joint-embedding losses, such as covariance or redundancy-reduction objectives, but the paper leaves that open.
  • Editorial extension: a natural check is to run ReSA on modalities whose encodings do not obviously cluster, such as text or audio; the paper's own logic predicts that gains would scale with how clusterable the encoding is.
  • Editorial extension: the long-tailed dataset experiment suggests the method recovers from early noisy clusters, but does not isolate whether the benefit comes from softened negatives or from the diagonal prior; that separation is testable.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper studies the clustering properties of self-supervised joint embedding architectures. It presents empirical evidence, using Silhouette Coefficient, Adjusted Rand Index, and linear/k-NN evaluation on CIFAR-10/100, that the encoder's output (the encoding) has stronger and more stable clustering properties than the embedding or the projector's hidden-layer outputs. It then proposes Representation Self-Assignment (ReSA), which replaces the identity target of InfoNCE with a doubly stochastic assignment matrix obtained by applying Sinkhorn-Knopp to the encodings' cosine-similarity matrix, and trains the model with a cross-entropy loss between embeddings and this self-assignment. Results are reported on CIFAR-10/100, ImageNet-100, ImageNet with ResNet-50 and ViT-S/16, COCO detection/segmentation transfer, fine-grained datasets, low-shot and long-tailed settings, along with ablations and a gradient analysis of the proposed loss.

Significance. The central idea is novel and plausible: if SSL encodings already carry useful clustering structure, that structure can be recycled as a training target. The paper's analysis of where clustering properties reside (encoding vs. embedding vs. projector hidden layers) is a useful empirical contribution, and ReSA is simple, prototype-free, and computationally cheaper than comparable clustering-based methods such as SwAV and DINO. The gradient derivation in Eq. (8) is clear, the ablations are informative, and the authors provide code. However, the reported performance gains are currently confounded by differences in augmentation and in the linear evaluation protocol between ReSA and the baselines, so the central claim that the self-assignment mechanism is what improves representations is not yet established.

major comments (2)
  1. [§5.1, Table 1; Appendix B.3] The linear evaluation protocol is not matched between ReSA and the baselines. Appendix B.3 states that ReSA is evaluated on CIFAR-10/100 with the W-MSE/INTL Adam-based protocol and that applying this protocol to models from solo-learn makes their performance 'degrade noticeably,' so the paper reports the official solo-learn numbers instead. Consequently the linear columns of Table 1 compare ReSA under a favorable protocol against baselines under a different protocol, and the statement in §5.1 that the experiments 'strictly adhere to the same experimental settings as other methods' is not supported. The authors should either evaluate every method with the same protocol or report both protocols for all methods, quantifying the degradation explicitly.
  2. [§4.3, Figure 6; Appendix B.2; Tables 1–6, 10] ReSA is trained with weak augmentation (ResizedCrop + HorizontalFlip) for the clustering view, while the baselines are trained with the standard augmentation set. Figure 6 shows that moving ReSA from standard to weak augmentation improves ImageNet-100 linear accuracy by roughly 4 points and k-NN accuracy by roughly 3.5 points, so the gains in Table 1 cannot be attributed solely to the self-assignment mechanism. Because the weak augmentation is used in all reported ReSA pretraining runs, the comparisons in Tables 2, 3, 5, 6, and 10 are also affected. To isolate the contribution of the self-clustering loss, the authors should report ReSA with standard augmentation on both views and at least one baseline with weak augmentation, all under otherwise matched pretraining and evaluation settings.
minor comments (6)
  1. [§4.2] The text says 'can be writen as'; this should be 'written.'
  2. [Figure 3 caption] The legend label 'BarlowT wins' appears to be a typo for 'Barlow Twins'; please correct it for consistency with the main text.
  3. [Appendix C.3] The word 'evalutaion' should be 'evaluation.'
  4. [§2.1] The notation is ambiguous: Eq. (1) uses Fθ and F′θ′ for the two branches, but the subsequent definition Fθ(·) = Gθg(Eθe(·)) does not specify how θ′ relates to θ (e.g., a momentum or shared encoder). Please clarify the parameterization of both branches.
  5. [§4.1 vs. Appendix B.2] The main text describes the assignment matrix as SH = H⊤H computed on a single encoding H, whereas the PyTorch-style algorithm computes Sinkhorn(cos_sim(h1, h1m)) between the online and momentum encodings of the weak view. Please align the description with the actual implementation.
  6. [Appendix C.2] The argument that diagonal dominance of AH prevents early clustering errors is heuristic; providing a quantitative measure, such as the mean diagonal-to-off-diagonal ratio of AH during the first epochs or the assignment accuracy at initialization, would make the claim more convincing.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ReSA's self-assignment target is a self-training design, and its reported gains are evaluated on held-out downstream benchmarks rather than forced by construction.

full rationale

I find no circular step. ReSA's Eq. (2) uses the encoder's own Sinkhorn assignment A_H as a soft target for the embedding cross-entropy; this is a self-training objective, not a derivation that assumes its conclusion. A_H is recomputed online from the current encoding H (or its momentum version) without gradient propagation and is not fitted to any downstream label, so the reported linear, k-NN, transfer, and low-shot gains are not forced by construction. The motivating observation that encodings cluster better than embeddings is an empirical measurement (ARI, SC) on pretrained models using true CIFAR labels; it motivates the design but is not an output of the derivation. Appendix C.2's diagonal-dominance argument is heuristic rather than a formal proof, and Appendix B.3's non-uniform linear-evaluation protocol is a benchmarking confound, but neither is circularity. Self-citations to Weng et al. (2022, 2024) serve as baselines and optimizer-setting references; no load-bearing claim rests on a self-cited uniqueness theorem or prior result. The central claim that ReSA improves representation quality is checked on held-out tasks, so the derivation chain is self-contained with respect to circularity.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the choice of five hyperparameters or design choices, all of which affect the behavior of the self-assignment target, plus four assumptions about Sinkhorn stability, cosine similarity semantics, the encoder's clustering quality, and diagonal dominance. No new physical or architectural entities are postulated.

free parameters (5)
  • Temperature tau = 0.4
    Chosen from {0.2, 0.3, 0.4, 0.5} in Table 11 on ImageNet; the central loss Eq. (2) scales logits by 1/tau, so the softness of the assignment targets depends on this hand-tuned value.
  • Sinkhorn regularization epsilon = 0.05
    Borrowed from SwAV (Caron et al., 2020), not tuned; controls the sharpness of A_H and therefore how far ReSA is from InfoNCE.
  • Sinkhorn iterations T = 3
    Borrowed from SwAV; affects how close A_H is to a doubly stochastic matrix and how much off-diagonal mass remains in the assignment.
  • Weak augmentation set = ResizedCrop and HorizontalFlip
    Selected based on the augmentation study in Figure 6 on ImageNet-100; the ReSA loss is applied to the weakly augmented view, so this categorical choice materially affects performance.
  • Momentum EMA coefficient = 0.996 to 1
    Set in the pseudo-code; the assignment target is computed from momentum encodings, so the EMA schedule is part of the method's behavior.
assumptions (4)
  • standard math Sinkhorn-Knopp algorithm converges to a doubly stochastic matrix for positive similarity matrices
    Algorithm 1 in Section 4.1 treats the Sinkhorn output as a valid assignment without proving convergence or uniqueness for the finite-iteration setting used.
  • domain assumption Cosine similarity in the normalized encoding space reflects semantic similarity
    The self-assignment target A_H is built from H^T H under L2 normalization; the method assumes that high cosine similarity between different samples implies the same or related class.
  • domain assumption The encoder output holds superior and more stable clustering information than the projector output
    Section 3 provides empirical support for this premise, and the ReSA design relies on it when choosing the encoder output as the clustering source.
  • ad hoc to paper Diagonal dominance of A_H prevents early clustering errors from derailing training
    Appendix C.2 argues that diagonal elements of A_H are largest so that early errors have limited impact; this is a heuristic stability assumption, not a proven property of the full optimization trajectory.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Clustering Properties of Self-Supervised Learning." pith.science (2026). https://pith.science/paper/2J6GN73A

@misc{pith2026250118452,
  author       = {Pith},
  title        = {Pith review of: Clustering Properties of Self-Supervised Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2J6GN73A}},
  note         = {Machine review of arXiv:2501.18452}
}
abstract

Self-supervised learning (SSL) methods via joint embedding architectures have proven remarkably effective at capturing semantically rich representations with strong clustering properties, magically in the absence of label supervision. Despite this, few of them have explored leveraging these untapped properties to improve themselves. In this paper, we provide an evidence through various metrics that the encoder's output $encoding$ exhibits superior and more stable clustering properties compared to other components. Building on this insight, we propose a novel positive-feedback SSL method, termed Representation Self-Assignment (ReSA), which leverages the model's clustering properties to promote learning in a self-guided manner. Extensive experiments on standard SSL benchmarks reveal that models pretrained with ReSA outperform other state-of-the-art SSL methods by a significant margin. Finally, we analyze how ReSA facilitates better clustering properties, demonstrating that it effectively enhances clustering performance at both fine-grained and coarse-grained levels, shaping representations that are inherently more structured and semantically meaningful.

Figures

Figures reproduced from arXiv: 2501.18452 by the authors.

Figure 1
Figure 1. The positive-feedback SSL framework. It involves the model generating representations that possess semantically clus￾tering information. This clustering information is leveraged to design self-supervised loss function, which is then employed to more effectively guide the model’s learning process. under different transformations and noise (Bachman et al., 2019; He et al., 2020; Chen et al., 2020a), with demonstrated … view at source ↗
Figure 3
Figure 3. Comparison of clustering metrics in encoding H and embedding Z across various self-supervised pretrained models. All methods utilize a ResNet-18 encoder pretrained on CIFAR-10 for 1000 epochs. Circular markers represent metrics computed using encodings, while cross markers correspond to metrics derived from embeddings. All metrics are computed on the entire training set, and similar trends can be observed in the val… view at source ↗
Figure 2
Figure 2. The basic notations for joint embedding architectures (JEA) in SSL. Joint embedding architectures (JEA). Let B denote a mini-batch input sampled uniformly from a set of images D, and T denote the set of data transformations available for augmentation. We consider a pair of neural networks Fθ and F ′ θ ′ , parameterized by θ and θ ′ respectively. They take as input two randomly augmented views, X = T (B) and X′ = T ′… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Comparison of linear evaluation accuracy and clustering metrics of encoding H, embedding Z, and the hidden layer outputs within the projector P during the training process. The experiments are conducted using SimCLR, VICReg, and SwAV, employing a ResNet-18 encoder pret…
Figure 5
Figure 5. Figure 5: The framework of Representation Self-Assignment (ReSA). Here, no grad. denotes that the operation does not in￾volve gradient propagation, norm signifies that each sample is L2-normalized to compute cosine similarities, and sinkhorn refers to the Sinkhorn-Knopp algorith…
Figure 6
Figure 6. Figure 6: Investigate the impact of image augmentation on ReSA. The experiments are conducted employing a ResNet-18 encoder pretrained on ImageNet-100 for 200 epochs. The starting posi￾tions of bars represent results of the standard augmentation. The symbols ‘−’ on the x-axis in…
Figure 7
Figure 7. Figure 7: T-SNE visualization of SSL representations on CIFAR￾10. All methods are pretrained for 1000 epochs on CIFAR-10 using ResNet-18, with encodings utilized as representations to visualize all training data. For the multiple centroids observed in the bird and horse categori…
Figure 8
Figure 8. Figure 8: T-SNE visualization of SSL representations on CIFAR￾100. We enclose points of each subclass with convex polygons. To further substantiate this, we transfer models pretrained on ImageNet to fine-grained datasets for evaluation. As shown in [PITH_FULL_IMAGE:figures/full…
Figure 9
Figure 9. Figure 9: Comparison of evaluation accuracies and clustering metrics among various SSL methods during the training process. The experiments are conducted using SimCLR, SwAV, VICReg, and ReSA. The settings and notations are consistent with ones in [PITH_FULL_IMAGE:figures/full_f…
Figure 10
Figure 10. Figure 10: Ablation on extraction of clustering information from encoding vs. embedding to obtain the self-assignment AH. Both are pretrained for 1000 epochs on CIFAR-100 using ResNet-18 under totally the same experimental settings provided in Appendix B. 0 10 20 30 40 50 60 70 …
Figure 11
Figure 11. Figure 11: Visualization of the self-assignment matrix AH during the early stages of training. C.2. How ReSA Avoids Feature Collapse and Early Clustering Error? Although ReSA has demonstrated good performance in various experiments, it remains unclear how the model avoids featur…
Figure 12
Figure 12. Figure 12: Pretraining on the long-tailed dataset. Here We report the training losses of four self-supervised learning methods on the CIFAR100-LT dataset, along with their evalutaion performance on the full CIFAR-100 test set as measured by a linear probe and a k-NN classifier. …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

56 extracted references · 30 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    M., Rupprecht, C., and Vedaldi, A

    Asano, Y. M., Rupprecht, C., and Vedaldi, A. A critical analysis of self-supervision, or what we can learn from a single image. arXiv preprint arXiv:1904.13132, 2019

    arXiv 1904

  3. [3]

    The hidden uniform cluster prior in self-supervised learning

    Assran, M., Balestriero, R., Duval, Q., Bordes, F., Misra, I., Bojanowski, P., Vincent, P., Rabbat, M., and Ballas, N. The hidden uniform cluster prior in self-supervised learning. arXiv preprint arXiv:2210.07277, 2022 a

    arXiv 2022

  4. [4]

    Masked siamese networks for label-efficient learning

    Assran, M., Caron, M., Misra, I., Bojanowski, P., Bordes, F., Vincent, P., Joulin, A., Rabbat, M., and Ballas, N. Masked siamese networks for label-efficient learning. In European Conference on Computer Vision, pp.\ 456--473. Springer, 2022 b

    work page 2022

  5. [5]

    Self-supervised learning from images with a joint-embedding predictive architecture

    Assran, M., Duval, Q., Misra, I., Bojanowski, P., Vincent, P., Rabbat, M., LeCun, Y., and Ballas, N. Self-supervised learning from images with a joint-embedding predictive architecture. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 15619--15629, 2023

    2023

  6. [6]

    D., and Buchwalter, W

    Bachman, P., Hjelm, R. D., and Buchwalter, W. Learning representations by maximizing mutual information across views. In NeurIPS, 2019

    work page 2019

  7. [7]

    Beit: Bert pre-training of image transformers

    Bao, H., Dong, L., Piao, S., and Wei, F. Beit: Bert pre-training of image transformers. arXiv preprint arXiv:2106.08254, 2021

    arXiv 2021

  8. [8]

    Vicreg: Variance-invariance-covariance regularization for self-supervised learning

    Bardes, A., Ponce, J., and LeCun, Y. Vicreg: Variance-invariance-covariance regularization for self-supervised learning. In ICLR, 2022

    2022

Show all 56 references
  1. [9]

    Reverse engineering self-supervised learning

    Ben-Shaul, I., Shwartz-Ziv, R., Galanti, T., Dekel, S., and LeCun, Y. Reverse engineering self-supervised learning. Advances in Neural Information Processing Systems, 36: 0 58324--58345, 2023

    2023

  2. [10]

    No free lunch in self supervised representation learning

    Bendidi, I., Bardes, A., Cohen, E., Lamiable, A., Bollot, G., and Genovesio, A. No free lunch in self supervised representation learning. arXiv preprint arXiv:2304.11718, 2023

    2023 arXiv

  3. [11]

    A probabilistic model behind self-supervised learning

    Bizeul, A., Sch \"o lkopf, B., and Allen, C. A probabilistic model behind self-supervised learning. arXiv preprint arXiv:2402.01399, 2024

    2024 arXiv

  4. [12]

    Food-101--mining discriminative components with random forests

    Bossard, L., Guillaumin, M., and Van Gool, L. Food-101--mining discriminative components with random forests. In Computer vision--ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part VI 13, pp.\ 446--461. Springer, 2014

    2014

  5. [13]

    Deep clustering for unsupervised learning of visual features

    Caron, M., Bojanowski, P., Joulin, A., and Douze, M. Deep clustering for unsupervised learning of visual features. In ECCV, 2018

    2018

  6. [14]

    Unsupervised learning of visual features by contrasting cluster assignments

    Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., and Joulin, A. Unsupervised learning of visual features by contrasting cluster assignments. In NeurIPS, 2020

    2020

  7. [15]

    Emerging properties in self-supervised vision transformers

    Caron, M., Touvron, H., Misra, I., Jegou, H., Mairal, J., Bojanowski, P., and Joulin, A. Emerging properties in self-supervised vision transformers. In ICCV, 2021

    2021

  8. [16]

    A simple framework for contrastive learning of visual representations

    Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. A simple framework for contrastive learning of visual representations. In ICML, 2020 a

    2020

  9. [17]

    and He, K

    Chen, X. and He, K. Exploring simple siamese representation learning. In CVPR, 2021

    2021

  10. [18]

    Improved baselines with momentum contrastive learning

    Chen, X., Fan, H., Girshick, R., and He, K. Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297, 2020 b

    2003 arXiv

  11. [19]

    An empirical study of training self-supervised vision transformers

    Chen, X., Xie, S., and He, K. An empirical study of training self-supervised vision transformers. In CVPR, 2021

    2021

  12. [20]

    Sinkhorn distances: Lightspeed computation of optimal transport

    Cuturi, M. Sinkhorn distances: Lightspeed computation of optimal transport. Advances in neural information processing systems, 26, 2013

    2013

  13. [21]

    da Costa, V. G. T., Fini, E., Nabi, M., Sebe, N., and Ricci, E. solo-learn: A library of self-supervised methods for visual representation learning. Journal of Machine Learning Research, 23 0 (56): 0 1--6, 2022. URL http://jmlr.org/papers/v23/21-1155.html

    2022

  14. [22]

    ImageNet: A Large-Scale Hierarchical Image Database

    Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. ImageNet: A Large-Scale Hierarchical Image Database . In CVPR, 2009

    2009

  15. [23]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Devlin, J. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018

    2018 arXiv

  16. [24]

    Whitening for self-supervised representation learning

    Ermolov, A., Siarohin, A., Sangineto, E., and Sebe, N. Whitening for self-supervised representation learning. In ICML, 2021

    2021

  17. [25]

    Geiping, J., Goldblum, M., Somepalli, G., Shwartz-Ziv, R., Goldstein, T., and Wilson, A. G. How much data are augmentations worth? an investigation into scaling laws, invariance, and implicit regularization. arXiv preprint arXiv:2210.06441, 2022

    arXiv

  18. [26]

    Bootstrap your own latent - a new approach to self-supervised learning

    Grill, J.-B., Strub, F., Altch\' e , F., Tallec, C., Richemond, P., Buchatskaya, E., Doersch, C., Avila Pires, B., Guo, Z., Gheshlaghi Azar, M., Piot, B., kavukcuoglu, k., Munos, R., and Valko, M. Bootstrap your own latent - a new approach to self-supervised learning. In Neura...

    2020

  19. [27]

    Gupta, K., Ajanthan, T., Hengel, A. v. d., and Gould, S. Understanding and improving the role of projection head in self-supervised learning. arXiv preprint arXiv:2212.11491, 2022

    2022 arXiv

  20. [28]

    Momentum contrast for unsupervised visual representation learning

    He, K., Fan, H., Wu, Y., Xie, S., and Girshick, R. Momentum contrast for unsupervised visual representation learning. In CVPR, 2020

    2020

  21. [29]

    M., Khan, S., Yang, M.-H., and Khan, F

    Huang, L., Ni, Y., Weng, X., Anwer, R. M., Khan, S., Yang, M.-H., and Khan, F. S. Understanding whitening loss in self-supervised learning. IEEE Transactions on Pattern Analysis & Machine Intelligence, 0 (01): 0 1--12, 2024

    2024

  22. [30]

    and Arabie, P

    Hubert, L. and Arabie, P. Comparing partitions. Journal of classification, 2: 0 193--218, 1985

    1985

  23. [31]

    Understanding dimensional collapse in contrastive self-supervised learning

    Jing, L., Vincent, P., LeCun, Y., and Tian, Y. Understanding dimensional collapse in contrastive self-supervised learning. In ICLR, 2022

    2022

  24. [32]

    Learning multiple layers of features from tiny images

    Krizhevsky, A., Hinton, G., et al. Learning multiple layers of features from tiny images. 2009

    2009

  25. [33]

    Microsoft coco: Common objects in context

    Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollar, P., and Zitnick, L. Microsoft coco: Common objects in context. In ECCV, 2014

    2014

  26. [34]

    Self-supervised learning via maximum entropy coding

    Liu, X., Wang, Z., Li, Y.-L., and Wang, S. Self-supervised learning via maximum entropy coding. In Oh, A. H., Agarwal, A., Belgrave, D., and Cho, K. (eds.), Advances in Neural Information Processing Systems, 2022

    2022

  27. [35]

    and Hutter, F

    Loshchilov, I. and Hutter, F. SGDR: stochastic gradient descent with restarts. In ICLR, 2017

    2017

  28. [36]

    Deciphering the projection head: Representation evaluation self-supervised learning

    Ma, J., Hu, T., and Wang, W. Deciphering the projection head: Representation evaluation self-supervised learning. arXiv preprint arXiv:2301.12189, 2023

    2023 arXiv

  29. [37]

    Augmentations vs algorithms: What works in self-supervised learning

    Morningstar, W., Bijamov, A., Duvarney, C., Friedman, L., Kalibhat, N., Liu, L., Mansfield, P., Rojas-Gomez, R., Singhal, K., Green, B., et al. Augmentations vs algorithms: What works in self-supervised learning. arXiv preprint arXiv:2403.05726, 2024

    2024 arXiv

  30. [38]

    You don't need data-augmentation in self-supervised learning

    Moutakanni, T., Oquab, M., Szafraniec, M., Vakalopoulou, M., and Bojanowski, P. You don't need data-augmentation in self-supervised learning. arXiv preprint arXiv:2406.09294, 2024

    2024 arXiv

  31. [39]

    and Zisserman, A

    Nilsback, M.-E. and Zisserman, A. Automated flower classification over a large number of classes. In 2008 Sixth Indian conference on computer vision, graphics & image processing, pp.\ 722--729. IEEE, 2008

    2008

  32. [40]

    Oord, A. v. d., Li, Y., and Vinyals, O. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018

    2018 arXiv

  33. [41]

    Dinov2: Learning robust visual features without supervision

    Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023

    2023 arXiv

  34. [42]

    Papyan, V., Han, X., and Donoho, D. L. Prevalence of neural collapse during the terminal phase of deep learning training. Proceedings of the National Academy of Sciences, 117 0 (40): 0 24652--24663, 2020

    2020

  35. [43]

    M., Vedaldi, A., Zisserman, A., and Jawahar, C

    Parkhi, O. M., Vedaldi, A., Zisserman, A., and Jawahar, C. Cats and dogs. In 2012 IEEE conference on computer vision and pattern recognition, pp.\ 3498--3505. IEEE, 2012

    2012

  36. [44]

    and Gupta, A

    Purushwalkam, S. and Gupta, A. Demystifying contrastive self-supervised learning: Invariances, augmentations and dataset biases. Advances in Neural Information Processing Systems, 33: 0 3407--3418, 2020

    2020

  37. [45]

    Rousseeuw, P. J. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of computational and applied mathematics, 20: 0 53--65, 1987

    1987

  38. [46]

    Contrastive multiview coding

    Tian, Y., Krishnan, D., and Isola, P. Contrastive multiview coding. In European conference on computer vision, 2020

    2020

  39. [47]

    and Hinton, G

    van der Maaten, L. and Hinton, G. E. Visualizing data using t-sne. Journal of Machine Learning Research, 9: 0 2579--2605, 2008. URL https://api.semanticscholar.org/CorpusID:5855042

    2008

  40. [48]

    T., M \"u ller, S., and Hutter, F

    Wagner, D., Ferreira, F., Stoll, D., Schirrmeister, R. T., M \"u ller, S., and Hutter, F. On the importance of hyperparameters and data augmentation for self-supervised learning. arXiv preprint arXiv:2207.07875, 2022

    2022 arXiv

  41. [49]

    The caltech-ucsd birds-200-2011 dataset

    Wah, C., Branson, S., Welinder, P., Perona, P., and Belongie, S. The caltech-ucsd birds-200-2011 dataset. 2011

    2011

  42. [50]

    and Liu, H

    Wang, F. and Liu, H. Understanding the behaviour of contrastive loss. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 2495--2504, 2021

    2021

  43. [51]

    and Isola, P

    Wang, T. and Isola, P. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In Proceedings of the 37th International Conference on Machine Learning, 2020

    2020

  44. [52]

    M., Khan, S., and Khan, F

    Weng, X., Huang, L., Zhao, L., Anwer, R. M., Khan, S., and Khan, F. An investigation into whitening loss for self-supervised learning. In NeurIPS, 2022

    2022

  45. [53]

    M., Khan, S., Khan, F

    Weng, X., Ni, Y., Song, T., Luo, J., Anwer, R. M., Khan, S., Khan, F. S., and Huang, L. Modulate your spectrum in self-supervised learning. In ICLR, 2024

    2024

  46. [54]

    X., and Lin, D

    Wu, Z., Xiong, Y., Yu, S. X., and Lin, D. Unsupervised feature learning via non-parametric instance discrimination. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 3733--3742, 2018

    2018

  47. [55]

    Barlow twins: Self-supervised learning via redundancy reduction

    Zbontar, J., Jing, L., Misra, I., Lecun, Y., and Deny, S. Barlow twins: Self-supervised learning via redundancy reduction. In ICML, 2021

    2021

  48. [56]

    Ressl: Relational self-supervised learning with weak augmentation

    Zheng, M., You, S., Wang, F., Qian, C., Zhang, C., Wang, X., and Xu, C. Ressl: Relational self-supervised learning with weak augmentation. Advances in Neural Information Processing Systems, 34: 0 2543--2555, 2021

    2021

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.