pith. sign in

arxiv: 2606.02826 · v1 · pith:F63K76XTnew · submitted 2026-06-01 · 🧮 math.AG

On the fibers and semi-algebraicity of ReLU neuromanifolds

Axel Flinth , Stefano Mereta , Michele Pernice This is my paper

Pith reviewed 2026-06-28 12:24 UTC · model grok-4.3

classification 🧮 math.AG
keywords ReLU networksneuromanifoldssemi-algebraic setshonest open subsetsnetwork symmetriesfibers of realization mapsalgebraic geometry of neural networks
0
0 comments X

The pith

The neuromanifold realized by a ReLU network is not a semi-algebraic quotient of its weight space.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper studies the neuromanifold, the set of all functions a feedforward ReLU network can realize, and asks whether this set arises as a semi-algebraic quotient from the space of network weights. It proves that no such quotient exists in general. The authors define honest open subsets of weight space as regions free of hidden symmetries and show that the largest honest open set is Zariski open when the network is shallow. They conjecture that this largest honest set is always semi-algebraic. These facts matter because they describe the precise geometric and algebraic structure that links parameters to realized functions and their equivalences.

Core claim

We prove that the neuromanifold M_d is not a semi-algebraic quotient of the weight space under the network realization map. We introduce honest open subsets of the weight space on which the realization map exhibits no hidden symmetries. We conjecture that the maximal honest open subset is always semi-algebraic and prove that it is Zariski open in the shallow-network case.

What carries the argument

The honest open subset of weight space, the region where the ReLU realization map has no hidden symmetries.

If this is right

  • The realization map from weights to functions does not factor through any semi-algebraic quotient.
  • Hidden symmetries exist beyond those captured by semi-algebraic equivalence relations on the weights.
  • The maximal honest open set in weight space is semi-algebraic (conjectured for all depths).
  • The maximal honest open set is Zariski open when the network is shallow.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The fibers of the realization map over the neuromanifold likely carry additional structure not reducible to semi-algebraic data.
  • Honest open sets may serve as a practical domain for studying generic identifiability of ReLU networks.
  • The conjecture, if true, would allow algebraic-geometry tools to analyze the largest symmetry-free parameter regions for networks of arbitrary depth.

Load-bearing premise

The neuromanifold is exactly the image of the weight space under the standard realization map and semi-algebraic quotients are taken with respect to the usual semi-algebraic structure on Euclidean space.

What would settle it

An explicit ReLU network whose neuromanifold can be shown to arise as a semi-algebraic quotient of its weight space would falsify the main negative result.

read the original abstract

We study the semi-algebraicity of the neuromanifold $\mathcal{M}_\mathbf{d}$ of a feedforward ReLU neural network and its symmetries. We prove that $\mathcal{M}_\mathbf{d}$ is not a semi-algebraic quotient of the space of weights of the network. We introduce and study the notion of \emph{honest} open subset of the space of weights, where the network does not show any hidden symmetries. Finally, we conjecture that the maximal honest open is always semi-algebraic and prove that in the shallow case it is even Zariski.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 2 minor

Summary. The paper studies the semi-algebraicity of the neuromanifold M_d of a feedforward ReLU neural network and its symmetries. It proves that M_d is not a semi-algebraic quotient of the weight space, introduces the notion of honest open subsets of weight space (where the network exhibits no hidden symmetries), conjectures that the maximal honest open is always semi-algebraic, and proves that this holds with the stronger Zariski property in the shallow case.

Significance. If the central negative result and the shallow-case Zariski theorem hold, the work establishes that neuromanifolds cannot be reduced to semi-algebraic quotients, providing a structural distinction from algebraic varieties that may inform geometric analyses of neural network parameter spaces and symmetries. The explicit construction of honest opens and the Zariski result in the shallow case are concrete, falsifiable contributions that strengthen the geometric toolkit for ReLU networks.

major comments (1)
  1. [§3 (main theorem on non-quotient)] The proof that M_d is not a semi-algebraic quotient (stated in the abstract and presumably in §3 or §4) relies on the auxiliary notion of honest opens; however, the reduction step showing that the existence of an honest open with non-trivial fiber structure precludes a semi-algebraic quotient structure is not load-bearing without an explicit reference to the precise definition of 'semi-algebraic quotient' used (e.g., whether it is with respect to the standard semi-algebraic structure on Euclidean space or a quotient in the category of semi-algebraic sets).
minor comments (2)
  1. [Abstract] The abstract introduces 'honest open subset' without a one-sentence definition; adding a brief parenthetical would improve readability for readers outside algebraic geometry.
  2. [§2 (preliminaries)] Notation for the network realization map and the weight space should be introduced consistently in §2 before being used in later statements.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the careful reading and the constructive comment. We address the single major comment below and will incorporate the requested clarification.

read point-by-point responses

  1. Referee: [§3 (main theorem on non-quotient)] The proof that M_d is not a semi-algebraic quotient (stated in the abstract and presumably in §3 or §4) relies on the auxiliary notion of honest opens; however, the reduction step showing that the existence of an honest open with non-trivial fiber structure precludes a semi-algebraic quotient structure is not load-bearing without an explicit reference to the precise definition of 'semi-algebraic quotient' used (e.g., whether it is with respect to the standard semi-algebraic structure on Euclidean space or a quotient in the category of semi-algebraic sets).

    Authors: We agree that the reduction argument in §3 would benefit from an explicit reference to the definition of semi-algebraic quotient. In the revised manuscript we will add a short subsection (or paragraph) stating the precise definition we employ—a quotient in the category of semi-algebraic sets equipped with the standard semi-algebraic structure on Euclidean space—and we will spell out the reduction step showing why a non-trivial honest open with non-constant fiber dimension precludes the existence of such a quotient map. revision: yes

Circularity Check

0 steps flagged

No significant circularity

full rationale

This is a pure mathematical proof paper in algebraic geometry. The central result is a negative theorem (M_d is not a semi-algebraic quotient of weight space) derived from the standard definition of the neuromanifold as the image of the realization map and the standard semi-algebraic structure on Euclidean space. No equations reduce claims to fitted quantities, self-definitions, or load-bearing self-citations. The introduced notions (honest opens, Zariski property) are auxiliary and do not create circular reductions. The derivation is self-contained against external benchmarks in real algebraic geometry.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 1 invented entities

The claims rest on standard definitions of semi-algebraic sets, quotients, and the realization map of ReLU networks; the honest open subset is a new auxiliary object introduced without independent evidence outside the paper.

axioms (2)
  • domain assumption The neuromanifold is the image of the weight space under the network map, equipped with the quotient topology and semi-algebraic structure.
    Invoked throughout the abstract as the object whose properties are studied.
  • standard math Semi-algebraic quotients are well-defined for the relevant spaces of weights and functions.
    Background fact from real algebraic geometry used to state the main negative result.
invented entities (1)
  • honest open subset of weight space no independent evidence
    purpose: Open set in which the network map exhibits no hidden symmetries
    Newly defined object whose maximal version is conjectured to be semi-algebraic.

pith-pipeline@v0.9.1-grok · 5622 in / 1395 out tokens · 25075 ms · 2026-06-28T12:24:10.452926+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

45 extracted references · 1 linked inside Pith

  1. [1]

    arXiv preprint arXiv:2605.03601 , year=

    Most ReLU Networks Admit Identifiable Parameters , author=. arXiv preprint arXiv:2605.03601 , year=

    Pith/arXiv arXiv

  2. [2]

    Transactions on Machine Learning Research , year=

    The real tropical geometry of neural networks for binary classification , author=. Transactions on Machine Learning Research , year=

  3. [3]

    Advances in neural information processing systems , volume=

    On the number of linear regions of deep neural networks , author=. Advances in neural information processing systems , volume=

  4. [4]

    SIAM Journal on Applied Algebra and Geometry , volume=

    Sharp bounds for the number of regions of maxout networks and vertices of minkowski sums , author=. SIAM Journal on Applied Algebra and Geometry , volume=. 2022 , publisher=

    2022

  5. [5]

    parametric equivalence of ReLU networks , author=

    Functional vs. parametric equivalence of ReLU networks , author=. International Conference on Learning Representations , year=

  6. [6]

    International Conference on Machine Learning , pages=

    Tropical geometry of deep neural networks , author=. International Conference on Machine Learning , pages=. 2018 , organization=

    2018

  7. [7]

    2018 , eprint=

    Understanding Deep Neural Networks with Rectified Linear Units , author=. 2018 , eprint=

    2018

  8. [8]

    Multiclass Neural Network Minimization via Tropical

    Smyrnis, Georgios and Maragos, Petros , booktitle =. Multiclass Neural Network Minimization via Tropical. 2020 , editor =

    2020

  9. [9]

    International Conference on Learning Representations , year=

    Neural Network Approximation based on Hausdorff distance of Tropical Zonotopes , author=. International Conference on Learning Representations , year=

  10. [10]

    arXiv preprint arXiv:2306.15157 , year=

    Revisiting Tropical Polynomial Division: Theory, Algorithms and Application to Neural Networks , author=. arXiv preprint arXiv:2306.15157 , year=

    arXiv

  11. [11]

    1928 , publisher=

    Tietze, Heinrich , journal=. 1928 , publisher=

    1928

  12. [12]

    Key questions on energy and AI , year=

    IEA , journal=. Key questions on energy and AI , year=

  13. [13]

    Tropical Geometry and Machine Learning , year=

    Maragos, Petros and Charisopoulos, Vasileios and Theodosis, Emmanouil , journal=. Tropical Geometry and Machine Learning , year=

  14. [14]

    Quotients of semialgebraic spaces , year=

    Scheiderer, Claus , journal=. Quotients of semialgebraic spaces , year=

  15. [15]

    Elisenda and Lindsey, Kathryn and Rolnick, David , booktitle =

    Grigsby, J. Elisenda and Lindsey, Kathryn and Rolnick, David , booktitle =. 2023 , month =

    2023

  16. [16]

    2020 , month =

    Rolnick, David and Körding, Konrad , booktitle =. 2020 , month =

    2020

  17. [17]

    2021 , volume =

    Armenta, Marco Antonio and Jodoin, Pierre‐Marc , journal =. 2021 , volume =

    2021

  18. [18]

    2018 , publisher =

    Arora, Sanjeev and Li, Zhiyuan and Lyu, Kaifeng , booktitle =. 2018 , publisher =

    2018

  19. [19]

    arXiv preprint arXiv:2403.11871 , year=

    The real tropical geometry of neural networks , author=. arXiv preprint arXiv:2403.11871 , year=

    arXiv

  20. [20]

    Foundations of computational mathematics , volume=

    Topological properties of the set of functions generated by neural networks of fixed size , author=. Foundations of computational mathematics , volume=. 2021 , publisher=

    2021

  21. [21]

    The Thirteenth International Conference on Learning Representations , year=

    Decomposition Polyhedra of Piecewise Linear Functions , author=. The Thirteenth International Conference on Learning Representations , year=

  22. [22]

    Approximation by superpositions of a sigmoidal function , journal=

    Cybenko, George , volume=. Approximation by superpositions of a sigmoidal function , journal=. 1989 , publisher=

    1989

  23. [23]

    arXiv preprint arXiv:2310.03482 , year=

    The geometric structure of fully-connected relu layers , author=. arXiv preprint arXiv:2310.03482 , year=

    arXiv

  24. [24]

    arXiv preprint arXiv:2407.03851 , year=

    Implicit Hypersurface Approximation Capacity in Deep ReLU Networks , author=. arXiv preprint arXiv:2407.03851 , year=

    arXiv

  25. [25]

    Advances in Neural Information Processing Systems , volume=

    Towards lower bounds on the depth of ReLU neural networks , author=. Advances in Neural Information Processing Systems , volume=

  26. [26]

    Advances in Mathematics , volume=

    Functional dimension of feedforward ReLU neural networks , author=. Advances in Mathematics , volume=. 2025 , publisher=

    2025

  27. [27]

    SIAM Journal on Applied Algebra and Geometry , volume=

    Algorithmic determination of the combinatorial structure of the linear regions of relu neural networks , author=. SIAM Journal on Applied Algebra and Geometry , volume=. 2025 , publisher=

    2025

  28. [28]

    Zeitschrift f

    Some criteria for the minimality of pairs of compact convex sets , author=. Zeitschrift f. 1993 , publisher=

    1993

  29. [29]

    Archiv der Mathematik , volume=

    Minimal pairs of convex compact sets , author=. Archiv der Mathematik , volume=. 1994 , publisher=

    1994

  30. [30]

    Mathematika , volume=

    Minimal pairs of convex bodies in two dimensions , author=. Mathematika , volume=. 1992 , publisher=

    1992

  31. [31]

    Archiv der Mathematik , volume=

    On minimal convex pairs of convex compact sets , author=. Archiv der Mathematik , volume=. 1996 , publisher=

    1996

  32. [32]

    Journal of Applied Mathematics and Physics , volume=

    Minkowski Sum of Polytopes Defined by Their Vertices , author=. Journal of Applied Mathematics and Physics , volume=

  33. [33]

    2508.03867 , journal=

    Yulia Alexandr and Guido Montúfar , year=. 2508.03867 , journal=

    arXiv

  34. [34]

    IEEE Transactions on Neural Networks and Learning Systems , year=

    Revisiting Tropical Polynomial Division: Theory, Algorithms, and Application to Neural Networks , author=. IEEE Transactions on Neural Networks and Learning Systems , year=

  35. [35]

    Algebraic Statistics , volume=

    Minimal representations of tropical rational functions , author=. Algebraic Statistics , volume=. 2024 , publisher=

    2024

  36. [36]

    arXiv preprint arXiv:1911.12922 , year=

    Tropical polynomial division and neural networks , author=. arXiv preprint arXiv:1911.12922 , year=

    arXiv 1911

  37. [37]

    2509.05894 , journal=

    Yaoying Fu , year=. 2509.05894 , journal=

    arXiv

  38. [38]

    Forty-second International Conference on Machine Learning Position Paper Track , year=

    Algebra Unveils Deep Learning-An Invitation to Neuroalgebraic Geometry , author=. Forty-second International Conference on Machine Learning Position Paper Track , year=

  39. [39]

    2025 Joint Mathematics Meetings (JMM 2025) , organization=

    On the Geometry and Optimization of Polynomial Convolutional Networks , author=. 2025 Joint Mathematics Meetings (JMM 2025) , organization=

    2025

  40. [40]

    The Thirteenth International Conference on Learning Representations , year=

    Geometry of Lightning Self-Attention: Identifiability and Dimension , author=. The Thirteenth International Conference on Learning Representations , year=

  41. [41]

    arXiv preprint arXiv:1910.01671 , year=

    Pure and spurious critical points: a geometric study of linear networks , author=. arXiv preprint arXiv:1910.01671 , year=

    arXiv 1910

  42. [42]

    SIAM Journal on Applied Algebra and Geometry , volume=

    Function space and critical points of linear convolutional networks , author=. SIAM Journal on Applied Algebra and Geometry , volume=. 2024 , publisher=

    2024

  43. [43]

    SIAM Journal on Applied Algebra and Geometry , volume=

    Geometry of linear convolutional networks , author=. SIAM Journal on Applied Algebra and Geometry , volume=. 2022 , publisher=

    2022

  44. [44]

    arXiv preprint arXiv:2410.00722 , year=

    On the geometry and optimization of polynomial convolutional networks , author=. arXiv preprint arXiv:2410.00722 , year=

    arXiv

  45. [45]

    SIAM Journal on Matrix Analysis and Applications , volume=

    Geometry of linear neural networks: Equivariance and invariance under permutation groups , author=. SIAM Journal on Matrix Analysis and Applications , volume=. 2025 , publisher=

    2025