On the fibers and semi-algebraicity of ReLU neuromanifolds

We study the semi-algebraicity of the neuromanifold $\mathcal{M}_\mathbf{d}$ of a feedforward ReLU neural network and its symmetries. We prove that $\mathcal{M}_\mathbf{d}$ is not a semi-algebraic quotient of the space of weights of the network. We introduce and study the notion of \emph{honest} open subset of the space of weights, where the network does not show any hidden symmetries. Finally, we conjecture that the maximal honest open is always semi-algebraic and prove that in the shallow case it is even Zariski.