In a teacher-student setup, maximal expressive power for a fixed learnable-weight budget is characterized by even row or column distribution in linear RNNs and feedforward networks.
Concentration for the zero set of large random polynomial systems
1 Pith paper cite this work. Polarity classification is still indexing.
abstract
For random systems of $K$ polynomials in $N + 1$ real variables which include the models of Kostlan (1987) and Shub and Smale (1993), we prove that the number of zeros on the unit sphere for $K = N$ or the Hausdorff measure of the zero set for $K < N$ concentrates around its mean as $N\to\infty$. To prove concentration we show that the variance of the latter random variable normalized by its mean goes to zero. The polynomial systems we consider depend on a set of parameters which determine the variance of their Gaussian coefficients. We prove that the convergence is uniform in those parameters and $K$.
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
The impact of allocation strategies in subset learning on the expressive power of neural networks
In a teacher-student setup, maximal expressive power for a fixed learnable-weight budget is characterized by even row or column distribution in linear RNNs and feedforward networks.