Depth-L ReLU networks with weight decay have native function spaces that are (2/L)-normable quasi-Banach spaces — the induced regularizer is not a norm for L > 2.
On the inductive bias of infinite-depth ResNets and the bottleneck rank
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
abstract
We compute the minimum-norm weights of a deep linear ResNet, and find that the inductive bias of this architecture lies between minimizing nuclear norm and rank. This implies that, with appropriate hyperparameters, deep nonlinear ResNets have an inductive bias towards minimizing bottleneck rank.
years
2026 2verdicts
CONDITIONAL 2representative citing papers
DP-NGD enables second-order optimization under differential privacy by decoupling curvature estimation onto public data, performing isotropic DP operations in a whitened space, and dynamically clamping curvature eigenvalues to prevent instability.
citing papers explorer
-
Representation Costs in Data Science: Foundations and the Quasi-Banach Spaces of Deep Neural Networks
Depth-L ReLU networks with weight decay have native function spaces that are (2/L)-normable quasi-Banach spaces — the induced regularizer is not a norm for L > 2.
-
Differentially Private Natural Gradient Descent
DP-NGD enables second-order optimization under differential privacy by decoupling curvature estimation onto public data, performing isotropic DP operations in a whitened space, and dynamically clamping curvature eigenvalues to prevent instability.