Pith. sign in

REVIEW 4 major objections 4 minor 2 cited by

CLA: Latent Alignment for Online Continual Self-Supervised Learning

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Online self-supervised learning beats i.i.d. training at equal compute, via latent alignment

desk verdict Solid empirical OCSSL paper with a genuinely new combination, but the 'beats i.i.d.' claim holds only for SimSiam and is partly an artifact of the budget metric. read the letter →

arxiv 2507.10434 v2 pith:OEWVX2WV submitted 2025-07-14 cs.LG cs.CV

Giacomo Cignoni , Andrea Cossu , Alexandra Gomez-Villa , Joost van de Weijer , Antonio Carta This is my paper
classification cs.LGcs.CV
keywords onlinecontinuallearningself-supervisedlatentalignmentreplaybufferexponentialmovingaveragecomputationalbudgetcatastrophicforgettingrepresentation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tackles online continual self-supervised learning, where unlabeled data arrives as a one-pass minibatch stream with no task boundaries and a strict compute budget. The authors propose Continual Latent Alignment (CLA), a regularizer that pulls the current encoder's representations toward representations from the recent past, obtained either from an exponential-moving-average encoder (CLA-E) or from a replay buffer (CLA-R). The central claim is that, under a budget measured by cumulative backward passes (CBP), CLA surpasses existing continual-SSL methods and even beats standard i.i.d. training in final accuracy. A second, more surprising claim is that using CLA for the early phase of pretraining, then continuing with i.i.d. data, yields a better final model than full i.i.d. pretraining at the same total budget.

What carries the argument

The central object is the alignment regularizer L_reg = -SC(a_phi(z_r), z_hat)/2, a negative cosine similarity that pushes current representations of replayed samples, after a projection head a_phi, toward past target features. Two target sources are proposed: an exponential moving average network theta' = tau theta' + (1-tau) theta for CLA-E, and stored past features z* in a FIFO buffer for CLA-R. The alignment loss is added to the base SSL loss, so the model keeps learning on fresh stream data while being stabilized by past representations, which the paper argues yields fast convergence and mitigates forgetting without needing task boundaries.

What would settle it

A controlled experiment that measures wall-clock time or FLOPs for each method at the same CBP on identical hardware, then re-plots accuracy versus actual training time; if the gap between CLA and i.i.d. shrinks or reverses when forward passes and per-sample update counts are accounted for, the headline claim is an artifact of the CBP metric.

Watch

Extended reading notes

Core claim

The central discovery is that a lightweight alignment loss makes an online SSL learner converge much faster than plain SSL training, to the point where it overtakes the i.i.d. upper bound at the same computational budget. The paper formalizes the budget as the total number of backward passes, CBP = nv x nsteps x b, and reports results on Split CIFAR-100 and Split ImageNet100 with both SimSiam and SimCLR backbones. At the high budget, CLA-E reaches 43.1% final probing accuracy on CIFAR-100 versus 39.9% for i.i.d., and CLA variants also dominate average accuracy, keeping performance high across the whole stream. The paper further shows that a CLA-pretrained checkpoint, followed by ordinary i.i.d. training, matches or exceeds full i.i.d. training, which it interprets as evidence that CLA accelerates the early, convergence-dominated phase of self-supervised pretraining.

Load-bearing premise

The load-bearing premise is that counting backward passes is a fair way to measure computational budget: if two methods have the same CBP, they are assumed to be compared equally, even though CLA-E has an extra EMA forward pass and replay methods perform many more gradient updates on buffer samples than i.i.d. training does on any one sample.

Editorial extensions

If this is right

  • An online continual strategy can match or beat the i.i.d. upper bound when the budget is measured in backward passes, so the online stream is not inherently a handicap under small budgets.
  • CLA checkpoints are reusable warm starts: continuing with i.i.d. data from a CLA-pretrained model reaches the same accuracy as a full i.i.d. run, which is a concrete way to shorten SSL pretraining.
  • The FIFO buffer beating reservoir and MinRed buffers indicates that uniform iteration counts per sample matter more than unbiased stream sampling in the online setting.
  • CLA supports larger learning rates than plain replay baselines, suggesting the alignment term stabilizes optimization, which could transfer to other budget-limited training regimes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The i.i.d. comparison is exact only if backward passes are the dominant cost; the authors note CLA-E adds an EMA forward pass and SCALE copies the encoder. If forward-pass cost is material, the 'same budget' claim is approximate, though their wall-clock plots show modest differences.
  • The paper's own stated explanation for the i.i.d. gap is that replay methods perform many more gradient updates on buffer samples than i.i.d. does on any given sample, and that alignment adds fast adaptation. That suggests the headline result may owe part of its size to the non-i.i.d. update distribution rather than to alignment alone.
  • A testable extension is whether the early-pretraining benefit persists at larger scale, for example with full ImageNet or transformer backbones, where i.i.d. convergence dynamics differ from ResNet-18.
  • The finding hints at a broader principle: a small replay buffer plus feature alignment acts as a variance-reduction mechanism for optimization on non-stationary streams, potentially connecting to momentum or preconditioning ideas beyond continual learning.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces Continual Latent Alignment (CLA), a replay-based strategy for online continual self-supervised learning (OCSSL) that aligns current representations with representations from the recent past, supplied either by an exponential moving average (EMA) network (CLA-E) or by stored buffer features (CLA-R). A new budget metric, Cumulative Backward Passes (CBP), is proposed to compare methods under a fixed computational budget, and experiments on Split CIFAR-100 and Split ImageNet100 with SimSiam (and SimCLR in the appendix) claim that CLA surpasses existing OCSSL methods and, in some settings, even i.i.d. training at the same CBP. The paper also reports that CLA pretraining can serve as a more efficient early-stage initialization than i.i.d. pretraining. The manuscript includes ablations on buffer type, EMA parameter tau, regularization strength omega, and promises code release.

Significance. If the central claims are made precise and the comparison is properly controlled, the paper makes a useful contribution: it demonstrates that a replay-based alignment loss can accelerate early SSL training and provides a strong baseline for the understudied OCSSL scenario. The CBP metric, despite its limitations, is a constructive step toward standardizing compute-aware comparisons in online continual learning. The empirical protocol is largely careful: two datasets, two SSL backbones, matched CBP across methods, ablations, and multiple seeds. However, the headline claims in the abstract and conclusion are substantially stronger than what the full empirical record supports, and the CBP metric has a load-bearing fairness issue that needs to be addressed. With appropriate scoping and additional control experiments, the paper's core phenomenon is plausible and worth publishing.

major comments (4)
  1. [Abstract and Section 8 (Conclusion)] The claims "CLA surpasses existing methods, and even i.i.d. training" (Section 8) and "using CLA as a pretraining protocol in the early stages of pretraining leads to a better final performance" (Abstract) are not supported by the full empirical record. In the SimCLR experiments (Table 5, Appendix B.1), i.i.d. reaches 48.5 ± 0.7 on ImageNet100 versus 45.1 ± 0.2 for CLA-E and 44.2 ± 0.4 for CLA-R, and 42.9 ± 0.3 on CIFAR-100 versus 42.6 ± 0.3 for CLA-R; the appendix itself states that SimCLR i.i.d. baselines are "unbeaten by methods trained on the OCSSL stream." Likewise, Figure 8 shows that on CIFAR-100 with SimCLR, CLA pretraining only reaches, not surpasses, full i.i.d. training. The abstract and conclusion should be revised to restrict these claims to the SimSiam backbone and the specific CBP budgets/datasets where they hold, and to acknowledge the SimCLR counterexamples explicitly.
  2. [Section 7 (Results) and Table 2] The sentence "In Low CBP, again, CLA-E surpasses other strategies in both datasets" is contradicted by Table 2: on Split CIFAR-100 with b = 30, np = 1, CaSSLe-R reaches 26.6 ± 0.5 final accuracy, while CLA-E reaches 24.4 ± 1.3 and CLA-R 20.8 ± 0.8. The text also states that "CLA-R being comparable to CLA-E" in this setting, which is not the case on CIFAR-100. Please correct this summary and qualify the state-of-the-art claim to the configurations in which it actually holds.
  3. [Section 5, Eq. (4), and Section 7] The fairness of the CBP budget is the load-bearing premise for the comparison with i.i.d. training. CBP counts only backward passes: it does not count forward passes (so CLA-E's additional EMA forward pass is free), and it does not account for the number of gradient updates each unique sample receives. Because CLA's replay buffer samples are repeatedly drawn into the minibatch, they receive far more optimizer updates than any sample in the i.i.d. baseline (which sees each sample exactly nepochs times, Eq. (6)). The paper itself identifies "the presence of replay, as more training iterations are performed on buffer samples" as a cause of the i.i.d. improvement in Section 7. This means the headline "CLA surpasses i.i.d. under the same computational budget" may be an artifact of the budget definition rather than a property of the alignment loss. Please either provide a control experiment that gives the i.i.d. baseline a comparable number of gradient updates per sample (for example, by allowing it to revisit the full stream more often), or reformulate the claim to a "backward-pass budget" without implying equal sample-level training.
  4. [Section 6 and Appendix C] The comparison to prior methods is not fully controlled: the regularization weight ω of CLA is grid-searched on the validation set in every setting (Appendix C), while the baselines inherit all hyperparameters except learning rate from their original implementations. This asymmetry can inflate the reported margin of CLA over methods such as SCALE and Osiris-R. I ask for either a matched hyperparameter search for the baselines in this OCSSL scenario, or a sensitivity analysis showing that the relative ordering is stable when baseline hyperparameters are re-tuned.
minor comments (4)
  1. [Table 6 caption] The caption reads "CBP = for ImageNet100" with the value missing; the corresponding budget should be filled in (presumably 690k).
  2. [Equation (6)] The ceiling function in Eq. (6) is typeset as "l ... m"; please use proper ceiling brackets and define the notation.
  3. [Figure 1 caption] The caption states "CLA outperforms i.i.d. on the FINAL ACCURACY" without noting that this holds for the SimSiam backbone and specific CBP settings; please add the qualifier to match the rest of the paper.
  4. [Section 4, Algorithm 2] In the pseudocode, the variable names for the replay targets are introduced as z* in the text and as z1, z2 in the algorithm description; please unify the notation (e.g., use zr* consistently).

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: CLA's reported gains are empirical results against external benchmarks, not consequences of the paper's own definitions or fitted values.

full rationale

The paper's central claims are empirical comparisons, not derivations. CLA's loss (Eqs. 1-3) is defined independently of the results, and its advantage over baselines and i.i.d. is measured against held-out linear-probe accuracy on external benchmarks (CIFAR-100, ImageNet100) with ablations and downstream transfer. No parameter is fitted to the quantity it is then said to predict: learning rates and the alignment weight omega are tuned on a 10% validation set, while final and average accuracies are reported on test splits. The CBP budget metric (Eq. 4) equalizes backward-pass counts across methods by construction, and the i.i.d. baseline is allocated nepochs_iid = ceil(np * b / bs) (Eq. 6) so that CBP matches by construction; however, the outcome of the comparison is not forced by that construction. The paper itself reports conditions where i.i.d. wins, notably in the SimCLR appendix ('SimCLR demonstrates higher i.i.d. baselines than SimSiam, achieving results, remaining unbeaten by methods trained on the OCSSL stream') and at np=6 in Figure 5, so the 'surpasses i.i.d.' claim is an empirically contingent finding, not an identity. The paper also self-discloses the main validity caveats, including that CBP 'does not fully account for other sources of overhead' and that the CLA advantage is partly attributed to replay concentrating more iterations on buffer samples; these are fairness-of-metric and scope concerns, which belong to correctness risk rather than circularity. The EMA-based alignment design is motivated by Soutif-Cormerais et al. 2023b (co-authored by two of the present authors) and by external work (Michel et al. 2024), but the load-bearing evidence for the method's effectiveness is the paper's own experiments against non-self baselines, so the self-citation is not load-bearing. The conclusion's unqualified 'CLA surpasses ... even i.i.d. training' statement overgeneralizes relative to the full empirical record, but overgeneralization is a correctness issue, not a circular derivation. No load-bearing step in the derivation chain reduces to its own inputs by definition, by fitted parameter, or by a self-citation chain.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the fairness of the CBP budget metric and on the chosen evaluation protocol. CLA itself introduces no new physical or architectural entities beyond an EMA teacher and stored feature targets, both standard in SSL and continual learning literature. The main free parameters are the alignment weight omega and the EMA coefficient tau, with learning rates and buffer size also tuned.

free parameters (4)
  • omega (alignment loss weight) = 0.3 to 3.0 (Tables 8-9)
    Grid-searched on a 10% validation set for CLA methods only; baselines use fixed omega from their original papers. Controls the strength of the alignment term in Eq. (3).
  • tau (EMA update coefficient) = 0.999
    Set by hand and ablated in Fig. 9, showing limited sensitivity; still a free choice that affects the target features in CLA-E.
  • Learning rate = 0.003 to 0.3 depending on method and dataset
    Grid-searched on validation for all methods; standard but tuned.
  • Memory buffer size = 2000
    Chosen for all replay methods; not ablated, and it affects both convergence and the comparison with i.i.d.
assumptions (4)
  • domain assumption Linear probing accuracy on held-out classes measures SSL representation quality.
    Used for FINAL ACC. and AVG ACC.; standard practice in SSL evaluation but still a proxy.
  • domain assumption Backward-pass count (CBP) is an accurate proxy for total computational cost.
    Section 5, Eq. 4; the paper concedes it ignores forward passes and implementation overhead.
  • ad hoc to paper Equal CBP between replay-based and i.i.d. methods is a fair comparison.
    Underpins the claim that CLA surpasses i.i.d.; replay methods update buffer samples many more times, so per-sample gradient counts differ.
  • domain assumption The OCSSL protocol (one pass, small minibatches, no task boundaries) is the right testbed for real streaming.
    Section 3 defines this scenario and the evaluation metrics.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CLA: Latent Alignment for Online Continual Self-Supervised Learning." pith.science (2026). https://pith.science/paper/OEWVX2WV

@misc{pith2026250710434,
  author       = {Pith},
  title        = {Pith review of: CLA: Latent Alignment for Online Continual Self-Supervised Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OEWVX2WV}},
  note         = {Machine review of arXiv:2507.10434}
}
read the original abstract

Self-supervised learning (SSL) is able to build latent representations that generalize well to unseen data. However, only a few SSL techniques exist for the online CL setting, where data arrives in small minibatches, the model must comply with a fixed computational budget, and task boundaries are absent. We introduce Continual Latent Alignment (CLA), a novel SSL strategy for Online CL that aligns the representations learned by the current model with past representations to mitigate forgetting. We found that our CLA is able to speed up the convergence of the training process in the online scenario, outperforming state-of-the-art approaches under the same computational budget. Surprisingly, we also discovered that using CLA as a pretraining protocol in the early stages of pretraining leads to a better final performance when compared to a full i.i.d. pretraining.

Figures

Figures reproduced from arXiv: 2507.10434 by the authors.

Figure 1
Figure 1. Notable results: when trained with the same computational budget (CBP), CLA outperforms i.i.d. on the [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Compared to offline CSSL, in OCSSL data is presented as a one-pass continual stream, with no explicit [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Overview of the novel strategies based on alignment to past representations for OCSSL. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Top row: absolute training times in seconds with varying [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: FINAL ACC. on Split CIFAR-100 and Imagenet100 for different CBP values. i.i.d. performances increase proportionally to CBP, while on the CL stream performances stagnate after a certain np. A surprising result regards the i.i.d. training: it is commonly considered in CL…
Figure 6
Figure 6. Figure 6: Continuing the pretraining with i.i.d. data of a CLA [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: FINAL ACC. on Split CIFAR-100 and Imagenet100 for different CBP values. i.i.d. performances increase proportionally to CBP, while on the CL stream performances stagnate after a certain np. different computational budgets. In contrast, CLA-R exhibits slightly poorer res…
Figure 8
Figure 8. Figure 8: Continuing the pretraining with i.i.d. data of a CLA-pretrained learner (using SimCLR as the SSL backbone), [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: FINAL ACCURACY of CLA-E on High CBP CIFAR-100 with varying τ EMA hyperparameter, which controls θ ′ update speed, the EMA network used as target for the alignment. 0.3 1.0 3.0 values 32 34 36 38 40 42 44 Final Acc. CIFAR-100 - SimSiam CLA-E CLA-R 0.3 1.0 3.0 values 32 …
Figure 10
Figure 10. Figure 10: FINAL ACCURACY of CLA-E and CLA-R on High CBP CIFAR-100 with varying ω, the alignment strength. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_10.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Class Incremental Continual Learning with Self-Organizing Maps and Variational Autoencoders Using Synthetic Replay

    cs.LG 2025-08 conditional novelty 6.0 of 10

    A SOM-VAE generative replay method stores per-unit Gaussian statistics instead of raw data and reports competitive class-incremental accuracy on standard benchmarks.

  2. Lifelong Representations: A Survey on Continual Self-Supervised Learning for Vision Models

    cs.CV 2026-07 accept novelty 5.0 of 10

    CSSL for vision is more robust to forgetting than supervised CL due to task-agnostic features and flatter losses, yet still needs better protocols and scaling beyond small benchmarks to foundation-model continual pretraining.

Reference graph

Works this paper leans on

60 extracted references · 33 canonical work pages · cited by 2 Pith papers

  1. [1]

    Understanding intermediate layers using linear classifier probes

    Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classifier probes. arXiv preprint arXiv:1610.01644, 2016

    arXiv 2016

  2. [2]

    Vicreg: Variance-invariance-covariance regularization for self-supervised learning

    Adrien Bardes, Jean Ponce, and Yann LeCun. Vicreg: Variance-invariance-covariance regularization for self-supervised learning. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022 . OpenReview.net, 2022. URL https://openreview.net/forum?id=xm6YD62D1Ub

    work page 2022

  3. [3]

    Hypernetworks for continual semi-supervised learning

    Dhanajit Brahma, Vinay Kumar Verma, and Piyush Rai. Hypernetworks for continual semi-supervised learning. arXiv preprint arXiv:2110.01856, 2021

    arXiv 2021

  4. [4]

    Rethinking experience replay: a bag of tricks for continual learning

    Pietro Buzzega, Matteo Boschini, Angelo Porrello, and Simone Calderara. Rethinking experience replay: a bag of tricks for continual learning. In 2020 25th International Conference on Pattern Recognition (ICPR), pp.\ 2180--2187. IEEE, 2021

    work page 2020

  5. [5]

    New insights on reducing abrupt representation change in online continual learning

    Lucas Caccia, Rahaf Aljundi, Nader Asadi, Tinne Tuytelaars, Joelle Pineau, and Eugene Belilovsky. New insights on reducing abrupt representation change in online continual learning. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022 . OpenReview.net, 2022. URL https://openreview.net/forum?id=N8MaByOzUfb

    work page 2022

  6. [6]

    Deep clustering for unsupervised learning of visual features

    Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Matthijs Douze. Deep clustering for unsupervised learning of visual features. In Proceedings of the European conference on computer vision (ECCV), pp.\ 132--149, 2018

    work page 2018

  7. [7]

    Unsupervised learning of visual features by contrasting cluster assignments

    Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments. In Hugo Larochelle, Marc'Aurelio Ranzato, Raia Hadsell, Maria - Florina Balcan, and Hsuan - Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Ne...

    work page 2020

  8. [9]

    Emerging properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv \' e J \' e gou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021 , pp.\ 9630--9640. IEEE , 2021 b . doi:10.1109/ICCV48922.2021.009...

    arXiv 2021

Show all 60 references
  1. [10]

    Sy-con: Symmetric contrastive loss for continual self-supervised representation learning

    Sungmin Cha and Taesup Moon. Sy-con: Symmetric contrastive loss for continual self-supervised representation learning. arXiv e-prints, pp.\ arXiv--2306, 2023

    2023

  2. [11]

    On tiny episodic memories in continual learning

    Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet K Dokania, Philip HS Torr, and Marc'Aurelio Ranzato. On tiny episodic memories in continual learning. arXiv preprint arXiv:1902.10486, 2019

    1902 arXiv

  3. [12]

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey E. Hinton. A simple framework for contrastive learning of visual representations. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event , volume 119 of Proce...

    2020

  4. [13]

    Exploring simple siamese representation learning

    Xinlei Chen and Kaiming He. Exploring simple siamese representation learning. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021 , pp.\ 15750--15758. Computer Vision Foundation / IEEE , 2021. doi:10.1109/CVPR46437.2021.01549. UR...

    2021

  5. [14]

    Improved baselines with momentum contrastive learning

    Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297, 2020 b

    2003 arXiv

  6. [15]

    Lifelong Machine Learning

    Zhiyuan Chen and Bing Liu. Lifelong Machine Learning. Morgan & Claypool Publishers, 2016

    2016

  7. [16]

    Replay-free online continual learning with self-supervised multipatches

    Giacomo Cignoni, Andrea Cossu, Alex Gomez-Villa, Joost van de Weijer, and Antonio Carta. Replay-free online continual learning with self-supervised multipatches. arXiv preprint arXiv:2502.09140, 2025

    2025 arXiv

  8. [17]

    Continual pre-training mitigates forgetting in language and vision

    Andrea Cossu, Antonio Carta, Lucia Passaro, Vincenzo Lomonaco, Tinne Tuytelaars, and Davide Bacciu. Continual pre-training mitigates forgetting in language and vision. Neural Networks, 179: 0 106492, 2024

    2024

  9. [18]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li - Jia Li, Kai Li, and Fei - Fei Li. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2009), 20-25 June 2009, Miami, Florida, USA , pp.\ 248--255...

    2009

  10. [19]

    Hospedales

    Linus Ericsson, Henry Gouk, Chen Change Loy, and Timothy M. Hospedales. Self-supervised representation learning: Introduction, advances, and challenges. IEEE Signal Processing Magazine, 39 0 (3): 0 42–62, May 2022. ISSN 1558-0792. doi:10.1109/msp.2021.3134634. URL http://dx.do...

    2022

  11. [20]

    Turrisi da Costa, Xavier Alameda - Pineda, Elisa Ricci, Karteek Alahari, and Julien Mairal

    Enrico Fini, Victor G. Turrisi da Costa, Xavier Alameda - Pineda, Elisa Ricci, Karteek Alahari, and Julien Mairal. Self-supervised models are continual learners. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 202...

    2022

  12. [21]

    Gallardo, Tyler L

    Gianmarco J. Gallardo, Tyler L. Hayes, and Christopher Kanan. Self-supervised training enhances online continual learning. In 32nd British Machine Vision Conference 2021, BMVC 2021, Online, November 22-25, 2021 , pp.\ 168. BMVA Press, 2021. URL https://www.bmvc2021-virtualconf...

    2021

  13. [22]

    Bagdanov, and Joost van de Weijer

    Alex Gomez - Villa, Bartlomiej Twardowski, Lu Yu, Andrew D. Bagdanov, and Joost van de Weijer. Continually learning self-supervised representations with projected functional regularization. In IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPR Works...

    2022

  14. [23]

    Exemplar-free continual representation learning via learnable drift compensation

    Alex Gomez-Villa, Dipam Goswami, Kai Wang, Andrew D Bagdanov, Bartlomiej Twardowski, and Joost van de Weijer. Exemplar-free continual representation learning via learnable drift compensation. In European Conference on Computer Vision, pp.\ 473--490. Springer, 2024 a

    2024

  15. [24]

    Plasticity-optimized complementary networks for unsupervised continual learning

    Alex Gomez-Villa, Bartlomiej Twardowski, Kai Wang, and Joost Van de Weijer. Plasticity-optimized complementary networks for unsupervised continual learning. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp.\ 1690--1700, 2024 b

    2024

  16. [25]

    An empirical investigation of catastrophic forgetting in gradient-based neural networks

    Ian J Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville, and Yoshua Bengio. An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211, 2013

    2013 arXiv

  17. [26]

    Richemond, Elena Buchatskaya, Carl Doersch, Bernardo \' A vila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, R \' e mi Munos, and Michal Valko

    Jean - Bastien Grill, Florian Strub, Florent Altch \' e , Corentin Tallec, Pierre H. Richemond, Elena Buchatskaya, Carl Doersch, Bernardo \' A vila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, R \' e mi Munos, and Michal Valko. Bootstrap your ow...

    2020

  18. [27]

    A survey on self-supervised learning: Algorithms, applications, and future trends

    Jie Gui, Tuo Chen, Jing Zhang, Qiong Cao, Zhenan Sun, Hao Luo, and Dacheng Tao. A survey on self-supervised learning: Algorithms, applications, and future trends. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46 0 (12): 0 9052--9071, 2024

    2024

  19. [28]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016 , pp.\ 770--778. IEEE Computer Society, 2016. doi:10.1109/CV...

    2016 doi

  20. [29]

    Girshick

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross B. Girshick. Momentum contrast for unsupervised visual representation learning. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020 , pp.\ 9726--9735. IEEE ...

    2020

  21. [30]

    Overcoming catastrophic forgetting in neural networks

    James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of scien...

    2017

  22. [31]

    3d object representations for fine-grained categorization

    Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained categorization. In Proceedings of the IEEE international conference on computer vision workshops, pp.\ 554--561, 2013

    2013

  23. [32]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009

    2009

  24. [33]

    Continual learning for robotics: Definition, framework, learning strategies, opportunities and challenges

    Timoth \'e e Lesort, Vincenzo Lomonaco, Andrei Stoian, Davide Maltoni, David Filliat, and Natalia D \' az-Rodr \' guez. Continual learning for robotics: Definition, framework, learning strategies, opportunities and challenges. Information fusion, 58: 0 52--68, 2020

    2020

  25. [34]

    Gradient episodic memory for continual learning

    David Lopez - Paz and Marc'Aurelio Ranzato. Gradient episodic memory for continual learning. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 30:...

    2017

  26. [35]

    Representational continuity for unsupervised continual learning

    Divyam Madaan, Jaehong Yoon, Yuanchun Li, Yunxin Liu, and Sung Ju Hwang. Representational continuity for unsupervised continual learning. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022 . OpenReview.net, 2022. URL ...

    2022

  27. [36]

    Online continual learning in image classification: An empirical survey

    Zheda Mai, Ruiwen Li, Jihwan Jeong, David Quispe, Hyunwoo Kim, and Scott Sanner. Online continual learning in image classification: An empirical survey. Neurocomputing, 469: 0 28--51, 2022

    2022

  28. [37]

    Catastrophic interference in connectionist networks: The sequential learning problem

    Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pp.\ 109--165. Elsevier, 1989

    1989

  29. [38]

    Rethinking momentum knowledge distillation in online continual learning

    Nicolas Michel, Maorong Wang, Ling Xiao, and Toshihiko Yamasaki. Rethinking momentum knowledge distillation in online continual learning. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024 . OpenReview.net, 2024. URL https...

    2024

  30. [39]

    Reading digits in natural images with unsupervised feature learning

    Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng, et al. Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learning, number 2, pp.\ 4. Granada, 2011

    2011

  31. [40]

    Representation learning with contrastive predictive coding

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018

    2018 arXiv

  32. [41]

    Quang Pham, Chenghao Liu, and Steven C. H. Hoi. Dualnet: Continual learning, fast and slow. In Marc'Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), Advances in Neural Information Processing Systems 34: Annual Conference on...

    2021

  33. [42]

    Online continual learning without the storage constraint

    Ameya Prabhu, Zhipeng Cai, Puneet Dokania, Philip Torr, Vladlen Koltun, and Ozan Sener. Online continual learning without the storage constraint. arXiv preprint arXiv:2305.09253, 2023 a

    2023 arXiv

  34. [43]

    Dokania, Philip H

    Ameya Prabhu, Hasan Abed Al Kader Hammoud, Puneet K. Dokania, Philip H. S. Torr, Ser - Nam Lim, Bernard Ghanem, and Adel Bibi. Computationally budgeted continual learning: What does matter? In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver...

    2023

  35. [44]

    The challenges of continuous self-supervised learning

    Senthil Purushwalkam, Pedro Morgado, and Abhinav Gupta. The challenges of continuous self-supervised learning. In Shai Avidan, Gabriel Brostow, Moustapha Ciss \'e , Giovanni Maria Farinella, and Tal Hassner (eds.), Computer Vision -- ECCV 2022, pp.\ 702--721, Cham, 2022. Sprin...

    2022

  36. [45]

    Sylvestre - Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H. Lampert. icarl: Incremental classifier and representation learning. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017 , pp.\ 5533--5...

    2017 doi

  37. [46]

    Self-supervised online learning of basic object push affordances

    Barry Ridge, Ale s Leonardis, Ale s Ude, Miha Deni s a, and Danijel Sko c aj. Self-supervised online learning of basic object push affordances. International Journal of Advanced Robotic Systems, 12 0 (3): 0 24, 2015

    2015

  38. [47]

    Psa--a new scalable space partition based selection algorithm for moeas

    Shaul Salomon, Gideon Avigad, Alex Goldvard, and Oliver Sch \"u tze. Psa--a new scalable space partition based selection algorithm for moeas. In EVOLVE-A Bridge between Probability, Set Oriented Numerics, and Evolutionary Computation II, pp.\ 137--151. Springer, 2013

    2013

  39. [48]

    Memory-efficient semi-supervised continual learning: The world is its own replay buffer

    James Smith, Jonathan Balloch, Yen-Chang Hsu, and Zsolt Kira. Memory-efficient semi-supervised continual learning: The world is its own replay buffer. In 2021 International Joint Conference on Neural Networks (IJCNN), pp.\ 1--8. IEEE, 2021

    2021

  40. [49]

    Improved deep metric learning with multi-class n-pair loss objective

    Kihyuk Sohn. Improved deep metric learning with multi-class n-pair loss objective. In Daniel D. Lee, Masashi Sugiyama, Ulrike von Luxburg, Isabelle Guyon, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Pr...

    2016

  41. [50]

    A comprehensive empirical evaluation on online continual learning

    Albin Soutif-Cormerais, Antonio Carta, Andrea Cossu, Julio Hurtado, Vincenzo Lomonaco, Joost Van de Weijer, and Hamed Hemati. A comprehensive empirical evaluation on online continual learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.\ 351...

    2023

  42. [51]

    Improving online continual learning performance and stability with temporal ensembles

    Albin Soutif-Cormerais, Antonio Carta, and Joost Van de Weijer. Improving online continual learning performance and stability with temporal ensembles. In Conference on Lifelong Learning Agents, pp.\ 828--845. PMLR, 2023 b

    2023

  43. [52]

    Three scenarios for continual learning

    Gido M Van de Ven and Andreas S Tolias. Three scenarios for continual learning. arXiv preprint arXiv:1904.07734, 2019

    1904 arXiv

  44. [53]

    Random sampling with a reservoir

    Jeffrey S Vitter. Random sampling with a reservoir. ACM Transactions on Mathematical Software (TOMS), 11 0 (1): 0 37--57, 1985

    1985

  45. [54]

    Forgetting, ignorance or myopia: Revisiting key challenges in online continual learning

    Xinrui Wang, Chuanxing Geng, Wenhai Wan, Shao - Yuan Li, and Songcan Chen. Forgetting, ignorance or myopia: Revisiting key challenges in online continual learning. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhan...

    2024

  46. [55]

    Scale: Online self-supervised lifelong learning without prior knowledge

    Xiaofan Yu, Yunhui Guo, Sicun Gao, and Tajana Rosing. Scale: Online self-supervised lifelong learning without prior knowledge. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 2484--2495, 2023

    2023

  47. [56]

    Barlow twins: Self-supervised learning via redundancy reduction

    Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and St \' e phane Deny. Barlow twins: Self-supervised learning via redundancy reduction. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Vir...

    2021

  48. [57]

    Integrating present and past in unsupervised continual learning

    Yipeng Zhang, Laurent Charlin, Richard Zemel, and Mengye Ren. Integrating present and past in unsupervised continual learning. In Conference on Lifelong Learning Agents, pp.\ 388--409. PMLR, 2025

    2025

  49. [58]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  50. [59]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  51. [60]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  52. [61]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.