The paper reports that dropout-based variance, embedding distribution shape, and neuron sparsity all shift around the moment a modular arithmetic network groks, and proposes these as forecasting signals.
Hidden progress in deep learning: Sgd learns parities near the computational limit
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
citation-role summary
background 1
citation-polarity summary
fields
cs.LG 1years
2025 1verdicts
REJECT 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
Tracing the Path to Grokking: Embeddings, Dropout, and Network Activation
The paper reports that dropout-based variance, embedding distribution shape, and neuron sparsity all shift around the moment a modular arithmetic network groks, and proposes these as forecasting signals.