Dead-Direction Conditioners provide gauge-equivariant preconditioning by conditioning optimizer state on symmetry orbits, yielding improved resistance to over-training collapse and higher detection of dead directions compared to AdamW and Muon.
David Saad and Sara A Solla
6 Pith papers cite this work. Polarity classification is still indexing.
6
Pith papers citing it
fields
cs.LG 6representative citing papers
Grokking delay under cross-entropy is mediated primarily by logit scale and resulting softmax saturation, with weight norm acting only as an upstream handle that adds 1-2% beyond the scale.
EGD equalizes gradient speeds across singular directions, eliminating or shortening grokking plateaus on modular addition and sparse parity problems.
TRM with 7M parameters achieves 45% accuracy on ARC-AGI-1 and 8% on ARC-AGI-2, surpassing most LLMs with under 0.01% of their parameters.
One training example via RLVR boosts LLM math reasoning from 17.6% to 35.7% average across six benchmarks.
A residual from Hankel DMD on Wasserstein-mapped training distributions localizes grokking transitions in modular-addition Transformers with AUROC 0.93 and can precede onset under a sustained-threshold rule.
citing papers explorer
-
Dead-Direction Conditioners: Gauge-Equivariant Preconditioning for Deep Networks
Dead-Direction Conditioners provide gauge-equivariant preconditioning by conditioning optimizer state on symmetry orbits, yielding improved resistance to over-training collapse and higher detection of dead directions compared to AdamW and Muon.
-
What Does the Weight Norm Control in Grokking? Logit-Scale Mediation under Cross-Entropy
Grokking delay under cross-entropy is mediated primarily by logit scale and resulting softmax saturation, with weight norm acting only as an upstream handle that adds 1-2% beyond the scale.
-
Egalitarian Gradient Descent: A Simple Approach to Accelerated Grokking
EGD equalizes gradient speeds across singular directions, eliminating or shortening grokking plateaus on modular addition and sparse parity problems.
-
Less is More: Recursive Reasoning with Tiny Networks
TRM with 7M parameters achieves 45% accuracy on ARC-AGI-1 and 8% on ARC-AGI-2, surpassing most LLMs with under 0.01% of their parameters.
-
Reinforcement Learning for Reasoning in Large Language Models with One Training Example
One training example via RLVR boosts LLM math reasoning from 17.6% to 35.7% average across six benchmarks.
-
Distributional Spectral Diagnostics for Localizing Grokking Transitions
A residual from Hankel DMD on Wasserstein-mapped training distributions localizes grokking transitions in modular-addition Transformers with AUROC 0.93 and can precede onset under a sustained-threshold rule.