A neural-network posterior can be written as a log-concave mixture when the parameter count is large, making sampling tractable, with separate risk guarantees of N^{-1/4} and N^{-1/3} for a discrete-prior variant.
Complexity, Statistical Risk, and Metric Entropy of Deep Nets Using Total Path Variation
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
For any ReLU network there is a representation in which the sum of the absolute values of the weights into each node is exactly $1$, and the input layer variables are multiplied by a value $V$ coinciding with the total variation of the path weights. Implications are given for Gaussian complexity, Rademacher complexity, statistical risk, and metric entropy, all of which are shown to be proportional to $V$. There is no dependence on the number of nodes per layer, except for the number of inputs $d$. For estimation with sub-Gaussian noise, the mean square generalization error bounds that can be obtained are of order $V \sqrt{L + \log d}/\sqrt{n}$, where $L$ is the number of layers and $n$ is the sample size.
fields
math.ST 1years
2024 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Rapid Bayesian Computation and Estimation for Neural Networks via Log-Concave Coupling
A neural-network posterior can be written as a log-concave mixture when the parameter count is large, making sampling tractable, with separate risk guarantees of N^{-1/4} and N^{-1/3} for a discrete-prior variant.