Grokking reflects escape from a metastable low-dimensional regime where transverse curvature accumulates before generalization, with subspace motion necessary but curvature boost insufficient.
Shalima Binta Manir and Anamika Paul Rupa
5 Pith papers cite this work. Polarity classification is still indexing.
5
Pith papers citing it
fields
cs.LG 5years
2026 5representative citing papers
Multi-task grokking in Transformers produces staggered generalization, low-dimensional manifolds, weight-decay phase structure, holographic solutions, and transverse redundancy.
Weight decay controls distinct learning regimes in grokking transformers on modular arithmetic, tracked by new cheap attention-based diagnostics with empirical critical value and exponent fits.
In a structured-output NW matrix task, Transformers generalize fastest at intermediate dataset sizes while larger sets can accelerate memorization in partial-competence regimes.
Grokking in linear DNNs is explained as hysteresis in L2 phase transitions where SGD noise enables escape from low-accuracy metastable phases with Arrhenius scaling; the same mechanism is suggested for nonlinear networks.
citing papers explorer
-
Low-Dimensional and Transversely Curved Optimization Dynamics in Grokking
Grokking reflects escape from a metastable low-dimensional regime where transverse curvature accumulates before generalization, with subspace motion necessary but curvature boost insufficient.
-
The Geometry of Multi-Task Grokking: Transverse Instability, Superposition, and Weight Decay Phase Structure
Multi-task grokking in Transformers produces staggered generalization, low-dimensional manifolds, weight-decay phase structure, holographic solutions, and transverse redundancy.
-
Weight Decay Regimes in Grokking Transformers: Cheap Online Diagnostics
Weight decay controls distinct learning regimes in grokking transformers on modular arithmetic, tracked by new cheap attention-based diagnostics with empirical critical value and exponent fits.
-
Slower Generalization, Faster Memorization: A Sweet Spot in Algorithmic Learning
In a structured-output NW matrix task, Transformers generalize fastest at intermediate dataset sizes while larger sets can accelerate memorization in partial-competence regimes.
-
Noise-Driven Escape from Metastable Phases explains Grokking in Deep Neural Networks
Grokking in linear DNNs is explained as hysteresis in L2 phase transitions where SGD noise enables escape from low-accuracy metastable phases with Arrhenius scaling; the same mechanism is suggested for nonlinear networks.