Algebraic geometry of rational neural networks
read the original abstract
We study the expressivity of rational neural networks (RationalNets) through the lens of algebraic geometry. We consider rational functions that arise from a given RationalNet to be tuples of fractions of homogeneous polynomials of fixed degrees. For a given architecture, the neuromanifold is the set of all such expressible tuples. For RationalNets with one hidden layer and fixed activation function $1/x$, we characterize the dimension of the neuromanifold and provide defining equations for some architectures. We also propose algorithms that determine whether a given rational function belongs to the neuromanifold. For deep binary RationalNets, i.e., RationalNets all of whose layers except potentially for the last one are binary, we classify when the Zarisky closure of the neuromanifold equals the whole ambient space, and give bounds on its dimensions.
This paper has not been read by Pith yet.
Forward citations
Cited by 3 Pith papers
-
Copositive Matrices with Ordered Off-Diagonal Entries
Copositive matrices with nondecreasing off-diagonal entries admit a PSD plus nonnegative decomposition, which implies exactness of a natural relaxation for separable quadratic optimization over the simplex.
-
Patnaik-Pearson intrinsic dimension for internal representations of neural networks
Introduces the Patnaik-Pearson intrinsic dimension estimator, relates it to HTSR/SETOL for Pareto spectral densities, and applies it to measure embedding dimension evolution in BERT-base and DeepSeek-R1-Distill-Qwen-1.
-
Patnaik-Pearson intrinsic dimension for internal representations of neural networks
Introduces the Patnaik-Pearson intrinsic dimension estimator, proves some of its properties, relates it to HTSR/SETOL for Pareto spectra, and applies it to track embedding dimension evolution in BERT-base and DeepSeek...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.