Optimal depth-wise learning-rate scaling in deep scalar linear networks is data-dependent, so data-agnostic rules fail to transfer while the data-aware rule yields depth-independent linear convergence.
International Conference on Learning Representations , year=
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
fields
cs.LG 2years
2026 2representative citing papers
StableGrad applies scale correction to weight gradients after backpropagation to enable stable optimization of deep BatchNorm-free networks including PINNs.
citing papers explorer
-
Optimal Learning Rate Scaling Depends on Data in Deep Scalar Linear Networks
Optimal depth-wise learning-rate scaling in deep scalar linear networks is data-dependent, so data-agnostic rules fail to transfer while the data-aware rule yields depth-independent linear convergence.
-
StableGrad: Backward Scale Control without Batch Normalization
StableGrad applies scale correction to weight gradients after backpropagation to enable stable optimization of deep BatchNorm-free networks including PINNs.