REVIEW 3 cited by
Lower Bounds on the Depth of Integral ReLU Neural Networks via Lattice Polytopes
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
abstract
We prove that the set of functions representable by ReLU neural networks with integer weights strictly increases with the network depth while allowing arbitrary width. More precisely, we show that $\lceil\log_2(n)\rceil$ hidden layers are indeed necessary to compute the maximum of $n$ numbers, matching known upper bounds. Our results are based on the known duality between neural networks and Newton polytopes via tropical geometry. The integrality assumption implies that these Newton polytopes are lattice polytopes. Then, our depth lower bounds follow from a parity argument on the normalized volume of faces of such polytopes.
Forward citations
Cited by 3 Pith papers
-
Approximation Depth of Convex Polytopes
For every depth d below the exact depth of the n-simplex, every depth-d polytope misses the simplex by an empty-corner distance of exactly n+1-2^d.
-
On the Depth of Monotone ReLU Neural Networks and ICNNs
Monotone ReLU networks cannot compute or approximate the max function, input convex networks need depth n to compute the n-ary max, and some depth-2 ReLU networks beat every depth-k ICNN.
-
Toric geometry of ReLU neural networks
A toric-geometry framework for ReLU networks yields a necessary and sufficient intersection-number criterion for exact realizability by unbiased shallow ReLU networks.
Discussion (0). Continue with ORCID to comment.