MCTS discovers superior data encoding circuits for QCCNNs that outperform standard encodings on medical datasets, with effective rank of feature maps serving as a performance predictor.
Reducing Overfitting in Deep Networks by Decorrelating Representations
3 Pith papers cite this work. Polarity classification is still indexing.
3
Pith papers citing it
abstract
One major challenge in training Deep Neural Networks is preventing overfitting. Many techniques such as data augmentation and novel regularizers such as Dropout have been proposed to prevent overfitting without requiring a massive amount of training data. In this work, we propose a new regularizer called DeCov which leads to significantly reduced overfitting (as indicated by the difference between train and val performance), and better generalization. Our regularizer encourages diverse or non-redundant representations in Deep Neural Networks by minimizing the cross-covariance of hidden activations. This simple intuition has been explored in a number of past works but surprisingly has never been applied as a regularizer in supervised learning. Experiments across a range of datasets and network architectures show that this loss always reduces overfitting while almost always maintaining or increasing generalization performance and often improving performance over Dropout.
years
2026 3representative citing papers
Feedback alignment in deep networks is limited by low-rank error signals; orthogonal weight updates and activity normalization raise effective rank and boost performance.
Model collapse threatens AI democratization by disproportionately impacting low-resource and marginalized communities through reduced training efficiency and data distributions skewed away from distribution tails.
citing papers explorer
-
Discovering Data Encoding Strategies for Quantum-Classical Neural Networks Using Monte Carlo Tree Search
MCTS discovers superior data encoding circuits for QCCNNs that outperform standard encodings on medical datasets, with effective rank of feature maps serving as a performance predictor.
-
Overcoming Rank Collapse in Feedback Alignment
Feedback alignment in deep networks is limited by low-rank error signals; orthogonal weight updates and activity normalization raise effective rank and boost performance.
-
Position: the Stochastic Parrot in the Coal Mine. Model Collapse is a Threat to Low-Resource Communities
Model collapse threatens AI democratization by disproportionately impacting low-resource and marginalized communities through reduced training efficiency and data distributions skewed away from distribution tails.