The Influence Eliminating Unlearning framework maximizes relearning convergence delay via weight decay and noise injection to remove the influence of a forgetting set while preserving accuracy on retained data.
On weight initialization in deep neural networks.ArXiv, abs/1704.08863
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
abstract
A proper initialization of the weights in a neural network is critical to its convergence. Current insights into weight initialization come primarily from linear activation functions. In this paper, I develop a theory for weight initializations with non-linear activations. First, I derive a general weight initialization strategy for any neural network using activation functions differentiable at 0. Next, I derive the weight initialization strategy for the Rectified Linear Unit (RELU), and provide theoretical insights into why the Xavier initialization is a poor choice with RELU activations. My analysis provides a clear demonstration of the role of non-linearities in determining the proper weight initializations.
citation-role summary
background 1
citation-polarity summary
verdicts
UNVERDICTED 2roles
background 1polarities
background 1representative citing papers
Presents a non-distortive cancelable face template method via targeted image distortion that maintains identity signals for neural embedding models on MNIST and LFW data.
citing papers explorer
-
Efficient Unlearning through Maximizing Relearning Convergence Delay
The Influence Eliminating Unlearning framework maximizes relearning convergence delay via weight decay and noise injection to remove the influence of a forgetting set while preserving accuracy on retained data.
-
Embedding Non-Distortive Cancelable Face Template Generation
Presents a non-distortive cancelable face template method via targeted image distortion that maintains identity signals for neural embedding models on MNIST and LFW data.