Pith. sign in

REVIEW 4 major objections 4 minor 49 references

ConsistentFeature: A Plug-and-Play Component for Neural Network Regularization

T0 review · 4 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that a network trained so its features cannot reveal which random half of the training set they came from generalizes better, and supports this with a plug-in adversarial regularizer.

desk verdict A clever random-half adversarial regularizer, but the paper's own equations contradict the described method and the results as written are not reproducible. read the letter →

arxiv 2412.01476 v2 pith:3QL4DJAD submitted 2024-12-02 cs.LG

RuiZhe Jiang , Haotian Lei This is my paper · ORCID
classification cs.LG
keywords regularizationoverfittingdomain-invariantfeaturesadversarialtrainingfeatureconsistencygeneralizationdiscriminatormemorizationsuppression
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes that overfitting can be understood as models learning different representations on different i.i.d. draws of the same data, and that suppressing those representation differences should improve generalization. To test this, it introduces ConsistentFeature, a regularizer that randomly splits the training set into two halves, trains a small discriminator to tell the halves apart from their features, and adversarially updates the backbone to make the halves' features indistinguishable. The paper argues that this forces the model to rely on features shared across i.i.d. sets, which should also transfer to any third set. Across several architectures and datasets, including noisy and small-scale ones, it reports lower validation loss, higher accuracy, strong suppression of memorization on randomly labeled data, and the ability to recover from overfitting when applied mid-training.

What carries the argument

The load-bearing mechanism is an adversarial feature-consistency constraint between two random halves of the training set. A small discriminator head $H_{desc}$ is trained to tell whether a feature came from $D_A$ or $D_B$, while the backbone is trained to fool it on data from $D_B$ (Eqs. 6–7), so that the features of the two halves become statistically indistinguishable. The paper uses a hinge loss, a historical feature recording technique from reference [30] to stabilize the discriminator, and warm-up and shut-off epochs to avoid interfering with early training; the specific choice of feeding only one half to the adversarial update is introduced to keep the discriminator informative.

What would settle it

Train with a deliberately non-i.i.d. split, for example placing all samples of some classes in one half and the rest in the other, so the two halves are not drawn from the same distribution; if the method does not degrade relative to baseline on a held-out third set, the proposed mechanism of suppressing split-specific features is not what drives the reported gains. A second check: if validation loss keeps improving even when the discriminator is unable to distinguish the halves at chance level, then the improvement cannot be attributed to the adversarial consistency signal.

Watch

Extended reading notes

Core claim

The central claim is that constraining the feature distribution of two random halves of the training set to be indistinguishable acts as a general explicit regularizer. Formally, if $D_A$, $D_B$, $D_C$ are i.i.d. samples from $P(X,Y)$, and a model trained on $D_A \cup D_B$ learns features $\phi_{org} = \phi_{inv} + \phi_A + \phi_B + \phi_{AB}$, then penalizing the feature differences between $D_A$ and $D_B$ forces the model to use only $\phi_{reg} = \phi_{inv} + \phi_{AB}$, and the paper asserts (Eq. 5) that the loss on $D_C$ is no larger under $\phi_{reg}$ than under $\phi_{org}$. The proposed implementation adds a lightweight adversarial discriminator head that labels features by which half they came from, using a hinge loss, and updates the backbone to fool it on a subset of samples; this is claimed to reduce overfitting and improve accuracy across tasks and architectures with low sensitivity to hyperparameters.

Load-bearing premise

The load-bearing premise, Assumption 1, is that splitting a training set randomly produces two independent, identically distributed samples from the same population, so that any feature not shared by the two halves cannot be useful on a third draw from that population; the inequality in Eq. (5) restates this rather than proving it.

Editorial extensions

If this is right

  • The regularizer can be added to almost any supervised architecture without changing the training pipeline, and in the reported experiments it consistently lowers validation loss and raises top-1 accuracy relative to the same model without it.
  • On randomly labeled data, the method almost entirely suppresses the model's ability to memorize the training set, whereas label smoothing, weight decay, and dropout show weaker or no memory suppression under the same protocol.
  • When the regularizer is activated after the model has already begun to overfit, it can pull the validation loss back to a normal convergence trajectory.
  • Even without significant overfitting, the method still improves accuracy and validation loss, and it combines additively with other regularizers such as label smoothing and weight decay.
  • The learned features also produce lower validation loss on out-of-distribution data (ImageNet-A) across all tested architectures.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A straightforward extension would replace the adversarial hinge loss with any distributional divergence, for example MMD or a Wasserstein critic; if the core claim is right, the specific discriminator choice is not what carries the benefit.
  • The mechanism implies a testable scaling prediction: the gains should increase as the training set shrinks or becomes noisier, because dataset-specific features then occupy a larger share of what the model can fit; the paper's noisy-label results are consistent with this but do not isolate it.
  • The random split index and the feature vector admit a mutual-information reading: the discriminator is estimating how much the feature reveals which half of the data it came from, so the regularizer is effectively minimizing an upper bound on that mutual information.
  • A control experiment that deliberately breaks the i.i.d. assumption, for example by splitting classes unevenly between the two halves, should make the method's advantage shrink or vanish; that would confirm that the shared-population premise, rather than the adversarial machinery, is responsible for the reported gains.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes ConsistentFeature (CF), a plug-and-play regularization method that randomly splits the training set into two halves (DA and DB), trains a feature discriminator to distinguish features from the two halves, and adversarially updates the backbone to make features from the subsets indistinguishable. The motivation is that suppressing training-set-specific features learned on one random half will improve generalization to a third i.i.d. set, formalized as Assumption 1 and Eq. (5). Experiments on ImageNet-200, CIFAR-100, WebVision-mini, and a randomized Flower-102 dataset compare CF against weight decay, dropout, and label smoothing, and report validation loss, accuracy, and memorization-suppression behavior.

Significance. If the method works as advertised, it would be a simple, architecture-agnostic regularizer with low overhead and a plausible mechanism (removing dataset-specific features). The paper includes experiments across several architectures and tasks, and the randomized-label experiment is a useful probe of memorization. However, the significance is muted by three issues: the formal objectives in Eqs. (6)–(7) are inconsistent with the described adversarial mechanism, the theoretical justification is an assumption restated as a result, and the empirical evidence does not support the abstract's 'consistently improves accuracy' claim (e.g., CIFAR-100 in Table 3). The paper ships no code or machine-checked proofs, so reproducibility rests entirely on the written specification, which is currently incomplete.

major comments (4)
  1. [§3.2, Eq. (6)] The generator/backbone objective in Eq. (6) has the wrong sign relative to the described hinge-loss semantics. The text states that the hinge loss encourages H_desc(φ(X)) to approach 1 and H_desc(φ(X')) to approach −1, and that adversarial training is limited to X'∈D_B. Minimizing H_desc(φ(X')) drives the discriminator output for B-subset features toward −∞, making them more distinguishable from A-subset features, not less. The intended objective should be to maximize H_desc(φ(X')) (equivalently, minimize −H_desc(φ(X'))) so that the backbone pushes B features toward the A side, thereby fooling the discriminator. As written, Eq. (6) trains the backbone to amplify the feature difference, which is the opposite of the proposed mechanism.
  2. [§3.2, Eq. (7)] Eq. (7) is not a well-defined discriminator objective: L(H_desc(φ(X)), H_desc(φ(X'))) with L described as 'the hinge loss' lacks target labels. The stated goal (H_desc(φ(X)) → 1, H_desc(φ(X')) → −1) requires a loss such as L_hinge(y·H_desc(φ(X))) with y=+1 for DA and y=−1 for DB, but the equation as written would, if L is a pairwise loss, push the two outputs toward equality rather than toward the specified targets. The discriminator update is thus underspecified, and a reader cannot reproduce the method from Eqs. (6)–(7) alone.
  3. [§3.1, Assumption 1 and Eq. (5)] The claimed theoretical support is circular: Eq. (5) restates Assumption 1 rather than deriving it. The assumption that suppressing features specific to DA and DB yields better generalization on a third i.i.d. set DC is asserted with no formal argument, and the decomposition into ϕ_inv, ϕ_A, ϕ_B, and ϕ_AB is informal. Moreover, a random split without replacement of a finite training set does not create two independent samples from the population in the sense required by domain adaptation; the two halves are negatively correlated and share the same marginal distribution. If the paper intends to make a theoretical claim, it must either provide a rigorous derivation or explicitly frame Eq. (5) as a hypothesis, not a result.
  4. [Abstract and Table 3 (Appendix A)] The abstract claims that 'even in the absence of significant overfitting, our method consistently improves accuracy and reduces validation loss.' This is contradicted by the CIFAR-100 results in Table 3: CF 0.2 achieves Max ACC1 54.4±0.47 versus baseline 56.6±0.12, a drop of 2.2 points, and Avg Last 10 ACC1 drops from 55.8 to 50.6. The hyperparameter-sensitivity results in Table 2 also show several configurations with accuracy below the 74.7 baseline (e.g., p=0.5: 74.3, warm-up 10: 73.3, warm-up 100: 73.6). The paper should either revise the 'consistently' claim to reflect the actual trade-offs or provide an explanation for the accuracy regression on CIFAR-100.
minor comments (4)
  1. [§3.1, Definition 1] The notation is confusing: S = {ϕ(X) | X∈X} uses X both as an input space and as a sample; the empirical distributions P_hat_train and P_hat_val are not defined. Please clarify the notation.
  2. [§4.1] The text says 'For the specific default parameters of our method, please refer to Figure 2.' This likely refers to Table 2 or Figure 3; please correct the cross-reference.
  3. [§4.4] The references to 'Figure 10j' and 'Figure 10k' point to appendix figures without a clear pointer; consider using 'Figure 5' in the main text for the key memory-suppression and convergence curves.
  4. [§4.3] The phrase 'all regularization methods demonstrate some degree of overfitting suppression' followed by 'label smoothing ... achieves the highest accuracy' is inconsistent with the later statement that CF yields the lowest validation loss; please align the text with the quantitative tables.

Circularity Check

1 steps flagged · score 4.0 of 10

The theoretical generalization inequality (Eq. 5) restates Assumption 1 rather than being derived; the empirical claims are independently tested, so circularity is partial.

  1. self definitional [Section 3.1, Assumption 1, Eqs. (4)-(5)]
    "By penalizing the feature differences learned between A and B, the model is forced to suppress ϕ_A and ϕ_B, relying instead on a refined pattern: ϕ_reg = ϕ_inv + ϕ_AB (4) to make predictions. Thus, E_{(X,Y)∼\hat{P}_C}[L(f(ϕ_reg(X)),Y)] ≤ E_{(X,Y)∼\hat{P}_C}[L(f(ϕ_org(X)),Y)] (5) indicating that the constrained model achieves better generalization on dataset C."

    Eq. (5) is the same assertion as Assumption 1: suppressing dataset-specific patterns (ϕ_A, ϕ_B) leaves a pattern that generalizes better on another i.i.d. set. The paper defines ϕ_reg as exactly the pattern that remains after suppression and defines ϕ_inv as 'domain-invariant features, which are more generalizable,' so the inequality is built into the definitions rather than derived from the discriminator objective. No independent argument connects the hinge-loss adversarial updates in Eqs. (6)-(7) to the inequality; the 'Thus' is a restatement of the assumption, not a proof. The circularity is confined to the explanatory theory because the reported accuracy and loss gains are measured on held-out validation sets.

full rationale

The paper's central empirical contribution is independently grounded: improvements in accuracy and reductions in validation loss are measured on held-out validation sets and OOD data, so those results do not reduce to a fit or to self-citation. However, the theoretical framing in Section 3.1 is partially circular. Eq. (5), presented as a consequence of penalizing feature differences, merely restates Assumption 1. Because ϕ_reg is defined as the representation obtained after removing dataset-specific features and ϕ_inv as 'more generalizable,' the claimed inequality on a third i.i.d. set is true by construction of the definitions rather than by an independent argument. The paper is transparent in labeling this an assumption, which keeps the circularity moderate. No load-bearing self-citation appears; references such as [8] and [1] are external prior work. Separately, the formal objectives in Eqs. (6)-(7) appear inconsistent with the described hinge-loss targets, but that is an implementation-specification issue rather than a circularity and does not affect this score. Overall score: 4, reflecting one definitional restatement in the explanatory theory while the main empirical claims stand on independent held-out evaluation.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

No new entities are postulated. The discriminator head is a standard convolutional feature discriminator; the method combines existing components (DANN-style adversarial loss, random splitting). The free parameters are the split ratio, regularization weight, warm-up length, history length, and discriminator width, all chosen by hand. The assumptions are the i.i.d. random-split premise and the convergence of adversarial training.

free parameters (5)
  • dataset split ratio p = default 0.5
    Controls the fraction of data in the two random subsets; the ablation in Table 2 varies p between 0.2 and 0.8, showing mild performance differences.
  • regularization loss weight w = default 0.1
    Scales the adversarial consistency penalty added to the task loss; varied in the sensitivity analysis.
  • discriminator warm-up steps = default 1600 (Table 2); also 20 epochs on ImageNet200 (Appendix Table 5)
    Delay before the adversarial loss starts influencing the backbone; chosen by hand.
  • history length for discriminator = default 100
    Number of historical feature vectors used to stabilize the discriminator; varied in Table 2.
  • discriminator channel width = default 64
    Number of convolutional kernels in the discriminator head; varied in Table 2.
assumptions (4)
  • domain assumption Assumption 1: features decompose as phi_inv + phi_data and penalizing differences between random subsets A and B yields lower loss on a third i.i.d. set C (Eqs 3-5).
    This is the theoretical core; it is labeled as an assumption, but the inequality in Eq (5) is presented as a consequence without proof.
  • domain assumption Random halves of a finite training set are treated as independent i.i.d. samples from the population.
    Used to justify applying domain-adversarial training inside a single dataset; a finite split without replacement creates dependent subsets.
  • domain assumption The adversarial min-max game between the feature extractor and the discriminator converges and provides useful gradient signal.
    The method depends on stable GAN-style training; the paper uses historical features and warm-up to mitigate instability but does not prove convergence.
  • domain assumption AdamW without data augmentation isolates the effect of the regularizer (Section 4.1.2).
    The experiments assume that removing augmentation and using AdamW makes comparisons fair, yet the appendix describes augmentation and Adam for the same experiments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ConsistentFeature: A Plug-and-Play Component for Neural Network Regularization." pith.science (2026). https://pith.science/paper/3QL4DJAD

@misc{pith2026241201476,
  author       = {Pith},
  title        = {Pith review of: ConsistentFeature: A Plug-and-Play Component for Neural Network Regularization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3QL4DJAD}},
  note         = {Machine review of arXiv:2412.01476}
}
read the original abstract

Over-parameterized neural network models often lead to significant performance discrepancies between training and test sets, a phenomenon known as overfitting. To address this, researchers have proposed numerous regularization techniques tailored to various tasks and model architectures. In this paper, we introduce a simple perspective on overfitting: models learn different representations in different i.i.d. datasets. Based on this viewpoint, we propose an adaptive method, ConsistentFeature, that regularizes the model by constraining feature differences across random subsets of the same training set. Due to minimal prior assumptions, this approach is applicable to almost any architecture and task. Our experiments show that it effectively reduces overfitting, with low sensitivity to hyperparameters and minimal computational cost. It demonstrates particularly strong memory suppression and promotes normal convergence, even when the model has already started to overfit. Even in the absence of significant overfitting, our method consistently improves accuracy and reduces validation loss.

Figures

Figures reproduced from arXiv: 2412.01476 by the authors.

Figure 1
Figure 1. Validate Loss/Top-1 ACC with and without Consistent [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. T-SNE Visualization of Semantically Similar Categories [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Illustration of the Proposed Method. By randomly splitting the data into two subsets (i.e., [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Performance of Different Regularization Methods on [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Performance of Different Regularization Methods: (a) Training Loss (Memory) of Different Regularization Methods on Noisy [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Training Curves with Combined Methods on CIFAR-100 and WebVision [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Performance of label smoothing, dropout, label smoothing, consistent feature, weight decay on CIFAR-100 and Webvision-mini. [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: Training accuracy (a) on the Flower102-Random dataset with different methods, and training accuracy (b) on the standard [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Ablation Study on CF parameters: Impact on Top-1 Accuracy and Validation Loss. [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Training Curves Across Different Model Architectures on ImageNet200. [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 1
Figure 1. Figure 1: Training Curves with Combined Methods on CIFAR-100 and WebVision [PITH_FULL_IMAGE:figures/full_fig_p017_1.png]
Figure 2
Figure 2. Figure 2: Performance of label smoothing, dropout, label smoothing, consistent feature, weight decay on CIFAR-100 and Webvision-mini. [PITH_FULL_IMAGE:figures/full_fig_p018_2.png]
Figure 3
Figure 3. Figure 3: Training accuracy (a) on the Flower102-Random dataset with different methods, and training accuracy (b) on the standard [PITH_FULL_IMAGE:figures/full_fig_p018_3.png]
Figure 4
Figure 4. Figure 4: Ablation Study on CF parameters: Impact on Top-1 Accuracy and Validation Loss. [PITH_FULL_IMAGE:figures/full_fig_p019_4.png]
Figure 5
Figure 5. Figure 5: Training Curves Across Different Model Architectures on ImageNet200. [PITH_FULL_IMAGE:figures/full_fig_p020_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

49 extracted references · 29 canonical work pages

  1. [1]

    Adaptive consistency regular- ization for semi-supervised transfer learning

    Abulikemu Abuduweili, Xingjian Li, Humphrey Shi, Cheng- Zhong Xu, and Dejing Dou. Adaptive consistency regular- ization for semi-supervised transfer learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6923–6932, 2021. 3, 4

    work page 2021

  2. [2]

    The effects of adding noise during backprop- agation training on a generalization performance

    Guozhong An. The effects of adding noise during backprop- agation training on a generalization performance. Neural computation, 8(3):643–674, 1996. 3

    work page 1996

  3. [3]

    A closer look at memorization in deep networks

    Devansh Arpit, Stanisław Jastrzebski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S Kanwal, Tegan Ma- haraj, Asja Fischer, Aaron Courville, Yoshua Bengio, et al. A closer look at memorization in deep networks. In Inter- national conference on machine learning , pages 233–242. PMLR, 2017. 6, 7

    work page 2017

  4. [4]

    Inte- grating structured biological data by kernel maximum mean discrepancy

    Karsten M Borgwardt, Arthur Gretton, Malte J Rasch, Hans- Peter Kriegel, Bernhard Sch¨olkopf, and Alex J Smola. Inte- grating structured biological data by kernel maximum mean discrepancy. Bioinformatics, 22(14):e49–e57, 2006. 2

    work page 2006

  5. [5]

    Understanding and utilizing deep neural networks trained with noisy labels

    Pengfei Chen, Ben Ben Liao, Guangyong Chen, and Shengyu Zhang. Understanding and utilizing deep neural networks trained with noisy labels. In International conference on machine learning, pages 1062–1070. PMLR, 2019. 5

    work page 2019

  6. [6]

    Sample prior guided robust model learning to suppress noisy labels

    Wenkai Chen, Chuang Zhu, and Mengting Li. Sample prior guided robust model learning to suppress noisy labels. InJoint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 3–19. Springer, 2023. 5

    work page 2023

  7. [7]

    Reducing transformer depth on demand with structured dropout

    Angela Fan, Edouard Grave, and Armand Joulin. Reducing transformer depth on demand with structured dropout. arXiv preprint arXiv:1909.11556, 2019. 3

    arXiv 1909

  8. [8]

    Unsupervised domain adaptation by backpropagation

    Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In International conference on machine learning, pages 1180–1189. PMLR, 2015. 2, 3

    work page 2015

Show all 49 references
  1. [9]

    Regu- larization theory and neural networks architectures

    Federico Girosi, Michael Jones, and Tomaso Poggio. Regu- larization theory and neural networks architectures. Neural computation, 7(2):219–269, 1995. 3

    1995

  2. [10]

    Connecting the dots with landmarks: Discriminatively learning domain- invariant features for unsupervised domain adaptation

    Boqing Gong, Kristen Grauman, and Fei Sha. Connecting the dots with landmarks: Discriminatively learning domain- invariant features for unsupervised domain adaptation. In International conference on machine learning , pages 222–

  3. [11]

    Generative adversarial nets

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014. 1

    2014

  4. [12]

    Natural adversarial examples

    Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15262–15271, 2021. 4

    2021

  5. [13]

    Data augmen- tation instead of explicit regularization

    Alex Hern ´andez-Garc´ıa and Peter K ¨onig. Data augmen- tation instead of explicit regularization. arXiv preprint arXiv:1806.03852, 2018. 5

    2018 arXiv

  6. [14]

    Fcns in the wild: Pixel-level adversarial and constraint-based adaptation

    Judy Hoffman, Dequan Wang, Fisher Yu, and Trevor Darrell. Fcns in the wild: Pixel-level adversarial and constraint-based adaptation. arXiv preprint arXiv:1612.02649, 2016. 2

    2016 arXiv

  7. [15]

    Searching for mo- bilenetv3

    Andrew Howard, Mark Sandler, Grace Chu, Liang-Chieh Chen, Bo Chen, Mingxing Tan, Weijun Wang, Yukun Zhu, Ruoming Pang, Vijay Vasudevan, et al. Searching for mo- bilenetv3. In Proceedings of the IEEE/CVF international conference on computer vision, pages 1314–1324, 2019. 5

    2019

  8. [16]

    Deep networks with stochastic depth

    Gao Huang, Yu Sun, Zhuang Liu, Daniel Sedra, and Kilian Q Weinberger. Deep networks with stochastic depth. In Com- puter Vision–ECCV 2016: 14th European Conference, Ams- terdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14, pages 646–661. Springer, 2016. 2, 3

    2016

  9. [17]

    Correcting sample selection bias by unlabeled data

    Jiayuan Huang, Arthur Gretton, Karsten Borgwardt, Bernhard Sch¨olkopf, and Alex Smola. Correcting sample selection bias by unlabeled data. Advances in neural information processing systems, 19, 2006. 2

    2006

  10. [18]

    Batch normalization: Accelerating deep network training by reducing internal covariate shift

    Sergey Ioffe. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167, 2015. 2, 3

    2015 arXiv

  11. [19]

    Image-to-image translation with conditional adversarial net- works

    Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial net- works. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1125–1134, 2017. 7

    2017

  12. [20]

    Adversar- ial adaptation of synthetic or stale data

    Young-Bum Kim, Karl Stratos, and Dongchan Kim. Adversar- ial adaptation of synthetic or stale data. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1297–1307, 2017. 2

    2017

  13. [21]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009. 5

    2009

  14. [22]

    Reg- ularization for deep learning: A taxonomy

    Jan Kukaˇcka, Vladimir Golkov, and Daniel Cremers. Reg- ularization for deep learning: A taxonomy. arXiv preprint arXiv:1710.10686, 2017. 1, 3

    2017 arXiv

  15. [23]

    Generalization and network design strategies

    Y LeCun. Generalization and network design strategies. Con- nections in Perspective, 1989. 2, 3

    1989

  16. [24]

    Layer normalization

    Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. ArXiv e-prints, pages arXiv–1607, 2016. 3

    2016

  17. [25]

    Dividemix: Learning with noisy labels as semi-supervised learning

    Junnan Li, Richard Socher, and Steven CH Hoi. Dividemix: Learning with noisy labels as semi-supervised learning. arXiv preprint arXiv:2002.07394, 2020. 5

    2002 arXiv

  18. [26]

    Webvision database: Visual learning and under- standing from web data

    Wen Li, Limin Wang, Wei Li, Eirikur Agustsson, and Luc Van Gool. Webvision database: Visual learning and under- standing from web data. arXiv preprint arXiv:1708.02862,

    arXiv

  19. [27]

    A convnet for the 2020s

    Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feicht- enhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition , pages 11976–11986,

  20. [28]

    Revisit- ing activation regularization for language rnns

    Stephen Merity, Bryan McCann, and Richard Socher. Revisit- ing activation regularization for language rnns. arXiv preprint arXiv:1708.01009, 2017. 2

    2017 arXiv

  21. [29]

    Which training methods for gans do actually converge? In International conference on machine learning, pages 3481–

    Lars Mescheder, Andreas Geiger, and Sebastian Nowozin. Which training methods for gans do actually converge? In International conference on machine learning, pages 3481–

  22. [30]

    Unrolled generative adversarial networks

    Luke Metz, Ben Poole, David Pfau, and Jascha Sohl- Dickstein. Unrolled generative adversarial networks. arXiv preprint arXiv:1611.02163, 2016. 7 9

    2016 arXiv

  23. [31]

    When does label smoothing help? Advances in neural in- formation processing systems, 32, 2019

    Rafael M ¨uller, Simon Kornblith, and Geoffrey E Hinton. When does label smoothing help? Advances in neural in- formation processing systems, 32, 2019. 6

    2019

  24. [32]

    Automated flower classification over a large number of classes

    Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In In- dian Conference on Computer Vision, Graphics and Image Processing, 2008. 5

    2008

  25. [33]

    Wasser- stein distance guided representation learning for domain adap- tation

    Jian Shen, Yanru Qu, Weinan Zhang, and Yong Yu. Wasser- stein distance guided representation learning for domain adap- tation. In Proceedings of the AAAI conference on artificial intelligence, 2018. 2

    2018

  26. [34]

    Domain adaptation: challenges, methods, datasets, and applications

    Peeyush Singhal, Rahee Walambe, Sheela Ramanna, and Ketan Kotecha. Domain adaptation: challenges, methods, datasets, and applications. IEEE access, 11:6973–7020, 2023. 2

    2023

  27. [35]

    On the origin of implicit regularization in stochastic gradient descent

    Samuel L Smith, Benoit Dherin, David GT Barrett, and So- ham De. On the origin of implicit regularization in stochastic gradient descent. arXiv preprint arXiv:2101.12176, 2021. 1

    2021 arXiv

  28. [36]

    How does early stopping help generalization against label noise? arXiv preprint arXiv:1911.08059, 2019

    Hwanjun Song, Minseok Kim, Dongmin Park, and Jae-Gil Lee. How does early stopping help generalization against label noise? arXiv preprint arXiv:1911.08059, 2019. 5

    1911 arXiv

  29. [37]

    Dropout: A simple way to prevent neural networks from overfitting

    Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(56):1929–1958, 2014. 2, 3

    1929

  30. [38]

    Rethinking the inception ar- chitecture for computer vision

    Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception ar- chitecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2818–2826, 2016. 2

    2016

  31. [39]

    Aug-kd: Anchor-based mixup generation for out-of-domain knowledge distillation

    Zihao Tang, Zheqi Lv, Shengyu Zhang, Yifan Zhou, Xinyu Duan, Fei Wu, and Kun Kuang. Aug-kd: Anchor-based mixup generation for out-of-domain knowledge distillation. arXiv preprint arXiv:2403.07030, 2024. 3

    2024 arXiv

  32. [40]

    A comprehensive survey on regularization strategies in machine learning

    Yingjie Tian and Yuqi Zhang. A comprehensive survey on regularization strategies in machine learning. Information Fusion, 80:146–166, 2022. 1

    2022

  33. [41]

    Efficient object localization using con- volutional networks

    Jonathan Tompson, Ross Goroshin, Arjun Jain, Yann LeCun, and Christoph Bregler. Efficient object localization using con- volutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 648–656,

  34. [42]

    Deep domain confusion: Maximizing for domain invariance

    Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. Deep domain confusion: Maximizing for domain invariance. arXiv preprint arXiv:1412.3474, 2014. 2

    2014 arXiv

  35. [43]

    Adversarial discriminative domain adaptation

    Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 7167–7176, 2017. 2

    2017

  36. [44]

    On the uniform convergence of relative frequencies of events to their prob- abilities

    Vladimir N Vapnik and A Ya Chervonenkis. On the uniform convergence of relative frequencies of events to their prob- abilities. In Measures of complexity: festschrift for alexey chervonenkis, pages 11–30. Springer, 2015. 3

    2015

  37. [45]

    Bayesian learning via stochas- tic gradient langevin dynamics

    Max Welling and Yee W Teh. Bayesian learning via stochas- tic gradient langevin dynamics. In Proceedings of the 28th international conference on machine learning (ICML-11) , pages 681–688. Citeseer, 2011. 3

    2011

  38. [46]

    Group normalization

    Yuxin Wu and Kaiming He. Group normalization. InProceed- ings of the European conference on computer vision (ECCV), pages 3–19, 2018. 3

    2018

  39. [47]

    Adversar- ial domain adaptation for stance detection

    Brian Xu, Mitra Mohtarami, and James Glass. Adversar- ial domain adaptation for stance detection. arXiv preprint arXiv:1902.02401, 2019. 2

    1902 arXiv

  40. [48]

    Understanding deep learning (still) re- quires rethinking generalization

    Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning (still) re- quires rethinking generalization. Communications of the ACM, 64(3):107–115, 2021. 1, 5, 6

    2021

  41. [49]

    Random erasing data augmentation

    Zhun Zhong, Liang Zheng, Guoliang Kang, Shaozi Li, and Yi Yang. Random erasing data augmentation. In Proceedings of the AAAI conference on artificial intelligence, pages 13001– 13008, 2020. 3 10 A. Appendix Section Method Min Val Loss Max ACC1 Max ACC5 Avg Last 10 Val Loss Avg...

    2020 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.