Pith. sign in

REVIEW 4 major objections 5 minor 50 references

Dealing with the Evil Twins: Improving Random Augmentation by Addressing Catastrophic Forgetting of Diverse Augmentations

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Random augmentation underperforms because its stochastic sampling creates colliding 'evil twin' gradients that erase previously learned features, and selectively merging the most-changed weights with a recent snapshot fixes the problem.

desk verdict A cheap weight-averaging trick that improves random augmentation, wrapped in an 'evil twin' story that the data does not yet distinguish from plain averaging. read the letter →

arxiv 2506.08240 v2 pith:E3CPE3BC submitted 2025-06-09 cs.LG

Dongkyu Cho , Rumi Chunara This is my paper
classification cs.LG
keywords randomaugmentationcatastrophicforgettinggradientcollisionweightmergingsinglesourcedomaingeneralizationout-of-distributionreplaymemoryeviltwin
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks why cheap random augmentation lags behind expensive targeted augmentation for out-of-distribution (OOD) generalization. It argues that the problem is not a lack of transformation diversity but too much uncontrolled diversity: stochastic sampling can draw two augmentations of the same image whose gradients disagree in sign across many parameters, so the model's updates destructively interfere and it forgets features learned from earlier augmentations. The proposed fix is a weight-space replay: every few iterations, average the most-changed 80% of parameters with a snapshot of the weights from earlier in training, leaving stable parameters untouched. On five single-source domain generalization benchmarks this raises random augmentation's target-domain accuracy substantially (PACS from 49.45 to 58.38, Digits from 71.84 to 78.92), closing much of the gap with targeted augmentation at nearly no extra training cost.

What carries the argument

The central object is the 'evil twin'—a pair of gradients $g_1, g_2$ from different augmentations of the same sample whose parameter-wise signs disagree on a large fraction of coordinates, quantified by the aggregated sign discrepancy $\mathrm{SD}$. The remedy is a selective weight merge: keep a snapshot $\theta_s$ from $k$ iterations earlier, compute the absolute per-parameter drift $d_i = |(\theta)_i - (\theta_s)_i|$, build a mask $m$ on the top $p=80\%$ of drifted parameters, and update $\theta \leftarrow m \odot \frac{\theta+\theta_s}{2} + (1-m)\odot \theta$. The mask is the load-bearing choice: it aims the averaging at parameters where forgetting is presumed to have happened, distinguishing this from plain weight averaging.

What would settle it

Train the same architecture and hyperparameters on random augmentation with plain periodic weight averaging (all weights averaged with the snapshot, no 80% mask); if this matches or beats the reported PACS 58.38 and Digits 78.92, then selective drift-based merging and the evil-twin mechanism are not needed to explain the gains.

Watch

Extended reading notes

Core claim

The paper's central claim is that random augmentation induces a distinct form of catastrophic forgetting. Because the transformation distribution has maximal entropy, two randomly chosen augmentations of the same sample can be 'evil twins': their gradients have extensively different parameter-wise signs, and training on the second erodes what the model learned from the first. The paper supports this with an entropy comparison, a Taylor-expansion argument that higher perturbation variance lowers expected gradient cosine similarity, and an MNIST rotation experiment where forgetting grows with an aggregated sign-discrepancy metric. The remedy is to substitute model weights for replay memory by periodically merging the current weights with a recent snapshot, restricted to the 80% of parameters that drifted most, and the paper reports consistent accuracy gains across PACS, Digits, Office-Home, VLCS, and Terra Incognita.

Load-bearing premise

That the accuracy gains come specifically from counteracting evil-twin forgetting, not from generic weight averaging; the paper's own ablation shows full-weight merging is nearly as effective, and no baseline against plain periodic weight averaging is reported.

Editorial extensions

If this is right

  • Random augmentation, already the cheapest form of augmentation, can reach the accuracy of targeted augmentation methods without their extra inference passes or auxiliary modules.
  • Replay memory becomes unnecessary: a snapshot of model weights carries the information needed to prevent forgetting of earlier augmentations.
  • The aggregated sign-discrepancy metric gives a training-time diagnostic: a rising value between sampled augmentations signals feature erosion and says a merge is due.
  • Because the fix operates on weights rather than data, it is architecture-agnostic and adds no test-time cost.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's ablation shows full-weight merging is nearly as good as the selective 80% merge on PACS (57.01 vs 58.38) and Digits (77.82 vs 78.92), so the selective mask may not be the active ingredient; plain periodic weight averaging could explain most of the gain.
  • If generic averaging is the active ingredient, the evil-twin narrative may still describe the failure mode, but the proposed remedy reduces to well-known weight averaging, and a direct comparison against standard averaging would settle which explanation holds.
  • The same mechanism should appear wherever augmentations are sampled from a high-entropy distribution, for example in self-supervised pretraining with heavy random augmentation, so weight-snapshot merging might transfer beyond single-source domain generalization.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper argues that random augmentation underperforms in single-source domain generalization because its stochasticity produces colliding augmentations ("evil twins") whose gradients interfere, causing a form of catastrophic forgetting. It proposes a simple selective weight-merging method: every k iterations, the current weights are averaged with a snapshot for the top 80% of parameters ranked by absolute drift (Eq. 15). The authors report large OOD accuracy gains over vanilla RandAugment on PACS, Digits, Office-Home, VLCS, and Terra Incognita, and support the mechanism with entropy comparisons, a sign-discrepancy (SD) metric, CKA feature-similarity analyses, and an ablation over the merge percentage.

Significance. If the mechanism were established, the paper would make a useful conceptual connection between data augmentation and continual learning, while also offering a low-cost practical method. The empirical gains are nontrivial (e.g., PACS 49.45 to 58.38, Digits 71.84 to 78.92), and the method avoids the extra inference and optimization costs of targeted augmentation. The authors deserve credit for the CKA and sign-discrepancy analyses, for including several benchmarks, and for honestly stating limitations. However, the central mechanistic claim is currently not secured: the absence of a standard weight-averaging baseline and the near-parity of full-weight merging with the proposed selective merging leave open a generic regularization/ensemble explanation, which the authors themselves invoke in Appendix C. The theoretical section also does not currently constitute a proof. The paper is, at this stage, a promising empirical study whose main interpretation needs substantial additional support.

major comments (4)
  1. [Sec. 5.2, Table 4 and Appendix C] The central claim that the gains come from counteracting evil-twin forgetting is not discriminated from generic weight averaging. Table 4 shows that full merging achieves 57.01 versus 58.38 on PACS and 77.82 versus 78.92 on Digits, so the selective top-80% mask contributes little, and Appendix C explicitly attributes the effect to smoother loss surfaces, ensembling, and implicit regularization. Please add a standard SWA baseline with the same snapshot interval and learning rate schedule (Izmailov et al. [21] is cited but never run), and report whether selective top-80% merging significantly outperforms full merging with error bars. Without this baseline, the selective mechanism and the evil-twin interpretation are not validated.
  2. [Sec. 3.2, Eq. (20) and Appendix B] The theoretical argument is not a proof. The step "the denominator grows faster than the numerator" is asserted, not demonstrated; moreover, E[cos(phi)] is not E[numerator]/E[denominator], so the variance computation for the numerator does not establish a decrease in the expectation. Higher-order terms in the Taylor expansion are dropped without bounds. Either provide a rigorous bound on E[cos(phi)] under stated assumptions on the covariance P_T, or reframe the section as an informal plausibility argument and remove the claim of a theoretical demonstration.
  3. [Sec. 3.3, Table 1 and Fig. 3] The "evil twin" experiment trains sequentially on two fixed rotations of MNIST, whereas the proposed method is applied to stochastically interleaved RandAugment transformations. The correlation between SD and forgetting in Table 1 therefore does not directly show that random augmentation's stochasticity causes the forgetting that Eq. (15) is meant to fix. Please add a measurement under the actual training distribution (e.g., SD between gradients from successive RandAugment draws, or forgetting of a held-out augmentation when training continues with random augmentations), or explicitly limit the claim to sequential fine-tuning.
  4. [Sec. 5.1 and Algorithm 1] The snapshot interval k is never specified or ablated. Since k, together with p, controls the method's behavior, omitting it makes the main experiments irreproducible. Please report the value of k for each benchmark and include a sensitivity analysis, or state that the results are insensitive to k with supporting evidence.
minor comments (5)
  1. [Table 4] The column header appears misaligned: the row "Full Merge 77.82 57.01 ..." is inconsistent with the stated "PACS Digits" order, since the PACS value should be approximately 57.01 and the Digits value approximately 77.82. Please swap or relabel the columns.
  2. [Sec. 5.2] The text says "71.84 → 78.03" for Digits, but Table 2 reports the RandAugment baseline as 71.84 and Ours as 78.92; please correct the reported improvement.
  3. [Algorithm 1] Line 7 of Algorithm 1 says "Select layers to merge based on rank," while Eqs. (12)-(14) select individual parameters; please align the wording.
  4. [Sec. 5.1] No standard deviations, number of seeds, or code release are reported for the main benchmark results; adding these would substantially improve reproducibility.
  5. [Sec. 6] In the Limitations section, the phrase "parameters to be reset" is inaccurate for Eq. (15), which averages rather than resets parameters; please rephrase.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the empirical and theoretical claims are not reduced to their inputs; the only self-citation is non-load-bearing.

full rationale

The paper's central claims are not circular. The entropy comparison (Eqs. 3-5 and Appendix A) follows directly from the definitions of uniform versus loss-weighted transformation selection; it is a mathematical consequence, not an assumption of the conclusion. The gradient-collision argument (Appendix B) uses a Taylor expansion to show that larger perturbation covariance lowers expected cosine similarity; this is a bound on a statistic, and the link to forgetting is made by independent citations [30,38] plus the paper's own measurements. The empirical 'evil twin' analysis (Table 1, Fig. 3) computes the sign-discrepancy metric independently of the final benchmark accuracies and demonstrates a correlation with forgetting, so the mechanism evidence is not fitted to the headline results. The proposed update (Eq. 15) is an acknowledged form of weight averaging; the paper explicitly cites the weight-averaging and model-merging literature [21,32,38] and reports the p=80 choice as a grid-searched hyperparameter ('The merging percentage was found using a grid search (see Tab. 4)'), so it is a disclosed design choice rather than a fitted parameter renamed as a prediction. The only self-citation, [6], appears in supporting citation groups alongside independent works [5,48] and is not used to justify any exclusive premise or uniqueness claim. The concern that generic averaging may explain the gains is an alternative-interpretation/correctness issue, not a circular reduction. No load-bearing step reduces to its own inputs by construction.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a finite augmentation set, an additive perturbation model with a valid first-order Taylor expansion, and an assumed causal link from gradient misalignment to forgetting. The method itself adds a heuristic (top-80% drift merging) whose mechanism is admitted to be unclear. The only fitted hyperparameter reported in the main text is p=80%, selected on test accuracy.

free parameters (2)
  • merge percentage p = 80%
    Chosen by grid search over target-domain test accuracies (Sec. 4, Table 4); the reported improvements are partly a result of selection on the test set.
  • snapshot interval k = not reported
    Algorithm 1 introduces k; the paper does not state the value used or its sensitivity.
assumptions (5)
  • domain assumption The set of available augmentations is finite and discrete, with uniform random selection.
    Sec. 3 Notation states T = {T1,...,Tn} finite; the authors acknowledge this limits the framework in Sec. 6.
  • domain assumption Every augmentation can be written as an additive input perturbation T(x) = x + δ with zero mean.
    Eq. (6) and Appendix B assume E[δ]=0 and covariance Σ_T; many augmentations (e.g., cropping, cutout) are not additive.
  • domain assumption A first-order Taylor expansion of the gradient around x is accurate.
    Eq. (7) and Eq. (19) use g_T(x) ≈ g_x + Mδ; for the large perturbations typical of random augmentation, higher-order terms are not negligible.
  • domain assumption A decrease in expected cosine similarity between gradients implies catastrophic forgetting.
    Sec. 3.2 takes gradient collision as the cause of forgetting, citing multi-task and continual learning literature [30,38]; the causal link is asserted, not proven.
  • ad hoc to paper Averaging the top-drift parameters with a snapshot approximates replay memory.
    Sec. 4, Eq. (15) and Algorithm 1; the paper itself admits in Sec. 5.2 that there is no understanding of why this selection criterion works.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Dealing with the Evil Twins: Improving Random Augmentation by Addressing Catastrophic Forgetting of Diverse Augmentations." pith.science (2026). https://pith.science/paper/E3CPE3BC

@misc{pith2026250608240,
  author       = {Pith},
  title        = {Pith review of: Dealing with the Evil Twins: Improving Random Augmentation by Addressing Catastrophic Forgetting of Diverse Augmentations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/E3CPE3BC}},
  note         = {Machine review of arXiv:2506.08240}
}
read the original abstract

Data augmentation is a promising tool for enhancing out-of-distribution generalization, where the key is to produce diverse, challenging variations of the source domain via costly targeted augmentations that maximize its generalization effect. Conversely, random augmentation is inexpensive but is deemed suboptimal due to its limited effect. In this paper, we revisit random augmentation and explore methods to address its shortcomings. We show that the stochastic nature of random augmentation can produce a set of colliding augmentations that distorts the learned features, similar to catastrophic forgetting. We propose a simple solution that improves the generalization effect of random augmentation by addressing forgetting, which displays strong generalization performance across various single source domain generalization (sDG) benchmarks.

Figures

Figures reproduced from arXiv: 2506.08240 by the authors.

Figure 1
Figure 1. Data Augmentation enhances generalization by gener [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. CKA Similarity between models trained with two distinct [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Aggregated Sign Discrepancy (SD) vs. Forgetting. Increase in the level of collision between augmented samples (measured as SD) coincides with the worsening of the forgetting. Formally, we define the sign discrepancy between g1 and g2 as SD(g1, g2) = 1 d X d i=1 1  sign(g1,i) ̸= sign(g2,i)  , (10) where g1,i denotes the ith component of g1, and d is the total number of parameters. In practice, to reduce noise from … view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Replay memory helps address forgetting. In our scenario, [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Average target domain accuracy on the Digits (left) and PACS (right) datasets. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Layer-wise Feature Similarity (CKA) of the model between two models trained with two distinct augmentations of the same [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

50 extracted references · 36 canonical work pages

  1. [21]

    Averaging weights leads to wider optima and better generalization

    Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry Vetrov, and Andrew Gordon Wilson. Averaging weights leads to wider optima and better generalization. arXiv preprint arXiv:1803.05407, 2018. 12

    arXiv 2018

  2. [1]

    Domain generalization by rejecting extreme augmentations

    Masih Aminbeidokhti, Fidel A Guerrero Pena, Heitor Rapela Medeiros, Thomas Dubail, Eric Granger, and Marco Pedersoli. Domain generalization by rejecting extreme augmentations. In Proceedings of the IEEE/CVF Winter Conference on Ap- plications of Computer Vision, pages 2215–2225, 2024. 1, 2

    work page 2024

  3. [2]

    A data- augmentation is worth a thousand samples: Analytical mo- ments and sampling-free training

    Randall Balestriero, Ishan Misra, and Yann LeCun. A data- augmentation is worth a thousand samples: Analytical mo- ments and sampling-free training. Advances in Neural Infor- mation Processing Systems, 35:19631–19644, 2022. 2

    work page 2022

  4. [3]

    Recognition in terra incognita, 2018

    Sara Beery, Grant van Horn, and Pietro Perona. Recognition in terra incognita, 2018. 6

    work page 2018

  5. [4]

    On tiny episodic memories in continual learning

    Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet K Dokania, Philip HS Torr, and Marc’Aurelio Ranzato. On tiny episodic memories in continual learning. In International Conference on Machine Learning. PMLR, 2019. 2

    work page 2019

  6. [5]

    Just pick a sign: Optimizing deep multitask models with gradi- ent sign dropout

    Zhao Chen, Jiquan Ngiam, Yanping Huang, Thang Luong, Henrik Kretzschmar, Yuning Chai, and Dragomir Anguelov. Just pick a sign: Optimizing deep multitask models with gradi- ent sign dropout. Advances in Neural Information Processing Systems, 33:2039–2050, 2020. 3

    work page 2020

  7. [6]

    Peer pressure: Model-to-model regularization for single source domain generalization

    Dong Kyu Cho, Inwoo Hwang, and Sanghack Lee. Peer pressure: Model-to-model regularization for single source domain generalization. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 15360–15370,

    work page

  8. [7]

    Autoaugment: Learning augmentation policies from data

    Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasude- van, and Quoc V Le. Autoaugment: Learning augmentation policies from data. arXiv preprint arXiv:1805.09501, 2018. 2

    arXiv 2018

Show all 50 references
  1. [8]

    Randaugment: Practical automated data augmentation with a reduced search space

    Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le. Randaugment: Practical automated data augmentation with a reduced search space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition work- shops, pages 702–703, 2020. 2, 5, 6

    2020

  2. [9]

    The mnist database of handwritten digit images for machine learning research

    Li Deng. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Maga- zine, 29(6):141–142, 2012. 6

    2012

  3. [10]

    Craft- ing distribution shifts for validation and training in single source domain generalization, 2024

    Nikos Efthymiadis, Giorgos Tolias, and Ondˇrej Chum. Craft- ing distribution shifts for validation and training in single source domain generalization, 2024. 1, 2

    2024

  4. [11]

    Adversarially adaptive normal- ization for single domain generalization

    Xinjie Fan, Qifei Wang, Junjie Ke, Feng Yang, Boqing Gong, and Mingyuan Zhou. Adversarially adaptive normal- ization for single domain generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8208–8217, 2021. 6

    2021

  5. [12]

    Unbiased met- ric learning: On the utilization of multiple datasets and web images for softening bias

    Chen Fang, Ye Xu, and Daniel N Rockmore. Unbiased met- ric learning: On the utilization of multiple datasets and web images for softening bias. In Proceedings of the IEEE Inter- national Conference on Computer Vision, pages 1657–1664,

  6. [13]

    Unsupervised domain adaptation by backpropagation

    Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In International Conference on Machine Learning, pages 1180–1189. PMLR, 2015. 6

    2015

  7. [14]

    Domain-adversarial training of neural networks

    Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marc- 8 hand, and Victor Lempitsky. Domain-adversarial training of neural networks. Journal of Machine Learning Research 17 (2016) 1-35, 2015. 6

    2016

  8. [15]

    How much data are augmentations worth? an investiga- tion into scaling laws, invariance, and implicit regularization

    Jonas Geiping, Micah Goldblum, Gowthami Somepalli, Ravid Shwartz-Ziv, Tom Goldstein, and Andrew Gordon Wil- son. How much data are augmentations worth? an investiga- tion into scaling laws, invariance, and implicit regularization. arXiv preprint arXiv:2210.06441, 2022. 2, 11

    arXiv

  9. [16]

    An empirical investigation of catas- trophic forgetting in gradient-based neural networks

    Ian J Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville, and Yoshua Bengio. An empirical investigation of catas- trophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211, 2013. 2

    2013 arXiv

  10. [17]

    Goodfellow, Jonathon Shlens, and Christian Szegedy

    Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples, 2015. 2

    2015

  11. [18]

    Out-of-distribution forgetting: vulnerability of continual learning to intra-class distribution shift

    Liangxuan Guo, Yang Chen, and Shan Yu. Out-of-distribution forgetting: vulnerability of continual learning to intra-class distribution shift. arXiv preprint arXiv:2306.00427, 2023. 2

    2023 arXiv

  12. [19]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 6

    2016

  13. [20]

    Data augmentation revisited: Rethinking the distribution gap between clean and augmented data, 2019

    Zhuoxun He, Lingxi Xie, Xin Chen, Ya Zhang, Yanfeng Wang, and Qi Tian. Data augmentation revisited: Rethinking the distribution gap between clean and augmented data, 2019. 2

    2019

  14. [22]

    Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell

    James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Overcoming catastrophic forgetting i...

    2017

  15. [23]

    Similarity of neural network representations revisited

    Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network representations revisited. In International conference on machine learning, pages 3519–3529. PMLR, 2019. 4

    2019

  16. [24]

    Continual learning with weight interpo- lation

    J˛ edrzej Kozal, Jan Wasilewski, Bartosz Krawczyk, and Michał Wo´ zniak. Continual learning with weight interpo- lation. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 4187–4195, 2024. 5, 12

    2024

  17. [25]

    Im- agenet classification with deep convolutional neural networks

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Im- agenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25, 2012. 6

    2012

  18. [26]

    Le Cun, B

    Y . Le Cun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel. Handwritten digit recognition with a back-propagation network. In Proceedings of the 2nd International Conference on Neural Information Processing Systems, page 396–404, Cambridge, MA, USA,

  19. [27]

    Deeper, broader and artier domain generaliza- tion

    Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales. Deeper, broader and artier domain generaliza- tion. In Proceedings of the IEEE international conference on computer vision, pages 5542–5550, 2017. 6

    2017

  20. [28]

    Domain generalization for medical imaging classification with linear-dependency regularization

    Haoliang Li, YuFei Wang, Renjie Wan, Shiqi Wang, Tie- Qiang Li, and Alex Kot. Domain generalization for medical imaging classification with linear-dependency regularization. Advances in neural information processing systems, 33:3118– 3129, 2020. 6

    2020

  21. [29]

    L. Li, K. Gao, J. Cao, Z. Huang, Y . Weng, X. Mi, Z. Yu, X. Li, and B. Xia. Progressive domain expansion network for single domain generalization. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 224–233, Los Alamitos, CA, USA, 2021. IEEE Co...

    2021

  22. [30]

    Gradient episodic memory for continual learning

    David Lopez-Paz and Marc’Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30, 2017. 11

    2017

  23. [31]

    Continual federated learning based on knowledge distillation

    Yuhang Ma, Zhongle Xie, Jue Wang, Ke Chen, and Lidan Shou. Continual federated learning based on knowledge distillation. In IJCAI, pages 2182–2188, 2022. 2

    2022

  24. [32]

    Weighted ensemble models are strong continual learners

    Imad Eddine Marouf, Subhankar Roy, Enzo Tartaglione, and Stéphane Lathuilière. Weighted ensemble models are strong continual learners. arXiv preprint arXiv:2312.08977, 2023. 5

    2023 arXiv

  25. [33]

    Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y . Ng. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning 2011 ,

    2011

  26. [34]

    Learning to learn single domain generalization

    Fengchun Qiao, Long Zhao, and Xi Peng. Learning to learn single domain generalization. In IEEE Conference on Com- puter Vision and Pattern Recognition (CVPR), pages 12556– 12565, 2020. 1, 6

    2020

  27. [35]

    Diverse weight averaging for out-of-distribution generaliza- tion

    Alexandre Rame, Matthieu Kirchmeyer, Thibaud Rahier, Alain Rakotomamonjy, Patrick Gallinari, and Matthieu Cord. Diverse weight averaging for out-of-distribution generaliza- tion. Advances in Neural Information Processing Systems , 35:10821–10836, 2022. 12

    2022

  28. [36]

    Experience replay for continual learning

    David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lill- icrap, and Gregory Wayne. Experience replay for continual learning. Advances in neural information processing systems, 32, 2019. 2, 4

    2019

  29. [37]

    Complementary learning for overcoming catas- trophic forgetting using experience replay

    Mohammad Rostami, Soheil Kolouri, and Praveen K Pilly. Complementary learning for overcoming catas- trophic forgetting using experience replay. arXiv preprint arXiv:1903.04566, 2019. 2

    1903 arXiv

  30. [38]

    Continual learning with scaled gradient projection

    Gobinda Saha and Kaushik Roy. Continual learning with scaled gradient projection. In Proceedings of the AAAI Con- ference on Artificial Intelligence, pages 9677–9685, 2023. 1, 5, 12

    2023

  31. [39]

    Deep hashing network for unsupervised domain adaptation

    Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 5018–5027, 2017. 6

    2017

  32. [40]

    Generalizing to unseen domains via adversarial data augmentation

    Riccardo V olpi, Hongseok Namkoong, Ozan Sener, John C Duchi, Vittorio Murino, and Silvio Savarese. Generalizing to unseen domains via adversarial data augmentation. Advances in neural information processing systems, 31, 2018. 1, 2, 5 9

    2018

  33. [41]

    Meta convolutional neural networks for single domain generalization

    Chaoqun Wan, Xu Shen, Yonggang Zhang, Zhiheng Yin, Xinmei Tian, Feng Gao, Jianqiang Huang, and Xian-Sheng Hua. Meta convolutional neural networks for single domain generalization. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4672–4681,

    2022

  34. [42]

    Foster: Feature boosting and compression for class- incremental learning

    Fu-Yun Wang, Da-Wei Zhou, Han-Jia Ye, and De-Chuan Zhan. Foster: Feature boosting and compression for class- incremental learning. arXiv preprint arXiv:2204.04662, 2022. 2

    2022 arXiv

  35. [43]

    A comprehensive survey of continual learning: Theory, method and application (2023)

    L Wang, X Zhang, H Su, and J Zhu. A comprehensive survey of continual learning: Theory, method and application (2023). arXiv preprint arXiv:2302.00487, 1(5), 2023. 2

    2023 arXiv

  36. [44]

    Learning to diversify for single domain generalization

    Zijian Wang, Yadan Luo, Ruihong Qiu, Zi Huang, and Mahsa Baktashmotlagh. Learning to diversify for single domain generalization. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision (ICCV), pages 834– 843, 2021. 2, 6

    2021

  37. [45]

    Learning structured sparsity in deep neural networks,

    Wei Wen, Chunpeng Wu, Yandan Wang, Yiran Chen, and Hai Li. Learning structured sparsity in deep neural networks,

  38. [46]

    Cihang Xie, Mingxing Tan, Boqing Gong, Jiang Wang, Alan Yuille, and Quoc V . Le. Adversarial examples improve image recognition, 2020. 2

    2020

  39. [47]

    Simde: A simple domain expansion approach for single-source domain generalization

    Qinwei Xu, Ruipeng Zhang, Yi-Yan Wu, Ya Zhang, Ning Liu, and Yanfeng Wang. Simde: A simple domain expansion approach for single-source domain generalization. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4798–4808, 2023. 2

    2023

  40. [48]

    Gradient surgery for multi- task learning

    Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi- task learning. Advances in neural information processing systems, 33:5824–5836, 2020. 3

    2020

  41. [49]

    Maximum-entropy adversarial data augmentation for im- proved generalization and robustness

    Long Zhao, Ting Liu, Xi Peng, and Dimitris Metaxas. Maximum-entropy adversarial data augmentation for im- proved generalization and robustness. Advances in Neural Information Processing Systems, 33:14435–14447, 2020. 2

    2020

  42. [50]

    Advst: Revisiting data augmentations for single domain generaliza- tion

    Guangtao Zheng, Mengdi Huai, and Aidong Zhang. Advst: Revisiting data augmentations for single domain generaliza- tion. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 21832–21840, 2024. 1, 2 10 A. Quantifying Augmentation Diversity: Ran- dom vs. Target...

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.