REVIEW 2 major objections 5 minor 53 references
A Quotient Homology Theory of Representation in Neural Networks
T0 review · 2 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash
Pith's one-line read The paper claims that for a ReLU network whose polyhedral pieces intersect the input manifold convexly, the homology of the network's representation is isomorphic to the quotient homology of the input manifold by the overlap…
desk verdict Genuinely new overlap decomposition and a promising quotient-homology approach, but Theorem 3.2 rests on an unproved continuity step in B.5; worth refereeing, needs a fix. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the overlap decomposition $\mathcal{O}_\Phi$, defined as the equivalence classes generated by points of different polyhedra whose images under the network intersect; it is computed by solving feasibility linear programs over the H-representations of the polyhedra and then running union-find on the detected overlapping pairs. The identity that carries the argument is $H_k(\Phi(M)) \simeq H_k(M/\mathcal{O}_\Phi)$, proven by showing that $M/\sim_\Phi$ is homotopy equivalent to $M/\mathcal{O}_\Phi$ when every $M \cap G_J$ is convex. This homotopy uses the fact that low-rank equivalence classes inside a convex intersection are themselves convex and hence contractible, so they can be collapsed without altering homology. The same machinery also yields the coarser, algorithmically convenient decomposition $\widehat{\mathcal{O}}_\Phi$ used in the numerical experiments.
What would settle it
Take a circle $S^1$ embedded in $\mathbb{R}^2$ and a ReLU network whose polyhedron intersects $S^1$ in a non-contractible arc and whose affine map on that polyhedron has rank one; if the computed $H_1(\Phi(M))$ differs from $H_1(M/\mathcal{O}_\Phi)$, then the theorem's convexity hypothesis is load-bearing and not merely technical.
Extended reading notes
Core claim
On its own terms, the paper's discovery is Theorem 3.2: given a ReLU network $\Phi$ with polyhedral decomposition $\{G_J\}$ such that $M \cap G_J$ is convex for every $J$, there is an isomorphism $H_k(\Phi(M)) \simeq H_k(M/\mathcal{O}_\Phi)$. The proof separates all non-injectivity into a rank source (an affine map of low rank inside one polyhedron) and an overlap source (points in different polyhedra mapped to the same output), and shows that under the convex-intersection condition the rank source contributes only contractible equivalence classes, so it can be quotiented out without changing homology. Consequently the Betti numbers of the neural representation are determined by the topology of the input manifold together with the gluing pattern encoded in the overlap decomposition, and no metric on the output space is needed. The paper further shows numerically that this overlap-based computation differs from persistent homology, which flags geometric near-identifications as topological features, and that training decreases the volume of overlap regions while increasing their number.
Load-bearing premise
The load-bearing condition is that every intersection $M \cap G_J$ between the input manifold and a polyhedron of the network is convex (or at least contractible), which the paper supports with a heuristic polyhedron-shrinking argument and an MNIST histogram rather than a proof or a checkable condition.
Editorial extensions
If this is right
- If Theorem 3.2 is correct, the Betti numbers of a neural representation are intrinsic: they depend only on the input manifold's topology and the network's gluing pattern, not on any metric in the output space.
- Layer-by-layer topology estimates based on persistent homology (including the reproduced Naitzat et al. curves) partly measure geometric distortion; the paper's quotient homology shows purely topological simplification happens more gradually across layers.
- For a convex input manifold the convex-intersection condition holds automatically, so the quotient-homology computation applies without further assumptions.
- Counting overlap regions at initialization offers an expressivity measure for non-injective maps, distinct from counting linear regions, and the number of such regions tends to increase with training while their total volume decreases.
Reading between the lines
- A consequence the paper leaves implicit: because quotient Betti numbers depend only on the activation pattern and input topology, they could serve as a scale-free invariant for comparing architectures, removing the need to choose a persistence scale.
- The convex-intersection hypothesis could be checked per polyhedron on real datasets by testing whether the populated points in each $M \cap G_J$ are linearly separable; such a test would tell practitioners when the rank source can actually be ignored.
- If polyhedral representations that scale better with dimension (as the paper suggests) are used, the same overlap-finding pipeline could extend quotient homology to high-dimensional inputs where listing H-representations is infeasible.
- The quotient construction is close to a Reeb space; requiring $x$ and $y$ to lie in the same connected component of $\Phi^{-1}(\Phi(x))$ would give a continuous counterpart whose homology could be approximated by the discrete overlap decomposition.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper introduces an 'overlap decomposition' of a ReLU network's polyhedral decomposition, defined by nontrivial intersections of the images of the affine maps on different polyhedra. It claims (Theorem 3.2) that, when the intersection of each polyhedron with the input manifold is convex, the homology of the network's output representation is isomorphic to the quotient homology H_k(M/O_Φ) of the input manifold by the overlap decomposition. The paper proposes a linear-programming and union-find algorithm to compute the overlap decomposition from a finite sample, and reports experiments on toy curves, on a reproduction of Naitzat et al.'s layer-wise Betti number study, and on sphere classification, showing that quotient homology tracks topological rather than geometric changes and that overlap regions shrink after training.
Significance. The proposed invariant is genuinely interesting: if Theorem 3.2 holds, it gives an intrinsic, metric-free way to compute homology of neural representations from the input manifold's topology and the network's polyhedral structure, avoiding the geometric contamination of persistent homology. The computational pipeline (LP feasibility plus union-find) is concrete, and the empirical comparisons with persistent homology on toy data are thoughtful. The paper also explicitly identifies limitations (type 1/2 errors, convexity heuristics), which is commendable. However, the central proof currently has a gap in the continuity of the homotopy equivalence, and the convexity hypothesis is not yet justified beyond heuristics.
major comments (2)
- [Appendix B, Theorem B.5] The proof defines π and the homotopy F piecewise, but it never establishes continuity of π along the boundary between the rank-only region and the full-rank region, nor does it show that the paths γ_z(t) can be chosen continuously in [x]_p. Even the existence of a single path in M/O_Φ is not immediate: M∩G_J convex implies the rank fiber is convex in M, but the quotient by O_Φ can identify points of that fiber with points elsewhere, so the image of the fiber in M/O_Φ need not be convex or even contractible. The paper needs either an explicit continuous selection/deformation retraction or an appeal to a cell-like map theorem, together with a proof that the quotient map is cell-like under the stated hypotheses.
- [Section 3.2 and end of Appendix B] The hypothesis 'M∩G_J is convex (or contractible) for every G_J' is load-bearing for Theorem 3.2, yet the paper justifies it by a heuristic scaling argument and the empirical histogram in Figure 6. For non-convex input manifolds (e.g., the spheres in Section 4.3), the condition can fail, and the paper falls back on the observed rarity of low-rank regions (bottom-right of Figure 2) without proving that the rank source is then homology-trivial. The paper itself states that the prevalence of the convexity condition is 'more of a heuristic argument.' Please prove a sufficient condition for the rank source to be homology-invariant when the convexity assumption fails, or provide a testable condition under which Theorem 3.2 applies.
minor comments (5)
- [Abstract and Section 3.1] The claim that the overlap decomposition is 'exactly determined' is too strong, because Algorithm 1 only considers populated polyhedra and Section 5.1 acknowledges type 2 errors from unpopulated polyhedra; please qualify the exactness statement (e.g., 'exact with respect to the sampled polyhedra').
- [Definition 2.3, Eq. (4)] The set-builder notation for ∼_Φ is circular because it uses [x] inside the definition of [x]; define it instead as the quotient set of M by the equivalence relation x∼y iff Φ(x)=Φ(y).
- [Eqs. (5) and (7)] The phrase 'for any I⊊K' is ambiguous because K is not introduced; it should read 'for every index set K with I⊊K'.
- [Appendix C.1, Algorithm 1] The loops 'for ∀y∈P_i and ∀z∈P_j' range over continuous sets; the pseudocode should specify that the loops iterate over the data points contained in each polyhedron.
- [Figure 2, bottom-right panel] The text says low-rank regions 'appear very rarely,' but no quantitative threshold or fraction is reported; please provide the proportion of low-rank polyhedra among all populated polyhedra.
Circularity Check
No circularity found: the quotient-homology isomorphism is a derived conditional statement, and the overlap decomposition is not fitted to the Betti numbers it is used to compute. The paper's self-citation to its own rank-decomposition work is not load-bearing, although Theorem B.5 has a continuity gap that is a correctness risk rather than a circular one.
full rationale
I walked the derivation chain. Theorem 3.2 postulates convexity of M∩G_J and asserts H_k(Φ(M)) ≃ H_k(M/O_Φ). This does not reduce to the theorem's inputs by construction: O_Φ is defined from the polyhedral images (Def. 2.3) and computed by linear-programming feasibility (Eq. 6), without using the target homology groups as a fitting target. The proof route is B.1–B.5: Φ(M) ≅ M/∼_Φ is a standard homeomorphism (Thm B.3), and B.5 attempts an independent homotopy equivalence between M/∼_Φ and M/O_Φ. The rank-decomposition idea is attributed to Beshkov & Einevoll (2024), but Theorem B.2 restates the rank/overlap dichotomy and B.5 gives its own convex-fiber argument; the self-citation is therefore not the load-bearing justification. The Naitzat reproduction chooses ε and k on known ground-truth topology, but this is stated openly in Appendix C and affects only the persistent-homology baseline, not the quotient-homology construction. The paper itself flags a real weakness: "At this stage this is more of a heuristic argument" (Appendix B) about how often the convexity condition holds, and B.5's homotopy F requires a continuous choice of representatives and paths that is asserted, not established. Those are rigor concerns, not cases of a derivation being equivalent to its inputs. Hence no circular step is identifiable.
Assumptions & free parameters
free parameters (3)
- sensitivity threshold δ =
1 (Section 4.3), 10 (Section 4.2)
- bounding box for polytope package =
[-100, 100] per dimension
- persistent homology scale epsilon and k for Naitzat reproduction =
epsilon=2.5, k=14 (D-I), k=19 (D-II, D-III)
assumptions (4)
- standard math ReLU neural networks are exactly continuous piecewise-linear maps (Arora et al. 2018)
- domain assumption The input manifold M is compact and the dataset is sampled from it
- domain assumption For all polyhedra G_J, the intersection M∩G_J is convex (or contractible)
- standard math Intersections of images of polyhedra are convex sets
Cite this review
Pith. "Pith review of A Quotient Homology Theory of Representation in Neural Networks." pith.science (2026). https://pith.science/paper/NZ27TM4B
@misc{pith2026250201360,
author = {Pith},
title = {Pith review of: A Quotient Homology Theory of Representation in Neural Networks},
year = {2026},
howpublished = {\url{https://pith.science/paper/NZ27TM4B}},
note = {Machine review of arXiv:2502.01360}
}
abstract
Previous research has proven that the set of maps implemented by neural networks with a ReLU activation function is identical to the set of piecewise linear continuous maps. Furthermore, such networks induce a hyperplane arrangement splitting the input domain of the network into convex polyhedra $G_J$ over which a network $\Phi$ operates in an affine manner. In this work, we leverage these properties to define an equivalence relation $\sim_\Phi$ on top of an input dataset, which defines a quotient space that can be split into two sets related to the local rank of $\Phi_J$ and the intersections $\cap \text{Im}\Phi_{J_i}$. We refer to the latter as the \textit{overlap decomposition} $\mathcal{O}_\Phi$ and prove that if the intersections between each polyhedron and an input manifold are convex, the homology groups of neural representations are isomorphic to quotient homology groups $H_k(\Phi(\mathcal{M})) \simeq H_k(\mathcal{M}/\mathcal{O}_\Phi)$. This lets us intrinsically calculate the Betti numbers of neural representations without the choice of an external metric. We develop methods to numerically compute the overlap decomposition through linear programming and a union-find algorithm. Using this framework, we perform several experiments on toy datasets showing that, compared to standard persistent homology, our overlap homology-based computation of Betti numbers tracks purely topological rather than geometric features. Finally, we study the evolution of the overlap decomposition during training on several classification problems and discuss some shortcomings of our method.
Figures
Figures from the paper (6 more)
Reference graph
Works this paper leans on
-
[1]
Adams, H., Emerson, T., Kirby, M., Neville, R., Peterson, C., Shipman, P., Chepushtanova, S., Hanson, E., Motta, F., and Ziegelmeier, L. (2017). Persistence images: A stable vector representation of persistent homology. Journal of Machine Learning Research , 18(8):1--35
work page 2017
-
[2]
Andreeva, R., Limbeck, K., Rieck, B., and Sarkar, R. (2023). Metric space magnitude and generalisation in neural networks. In Topological, Algebraic and Geometric Learning Workshops 2023 , pages 242--253. PMLR
work page 2023
-
[3]
Arora, R., Basu, A., Mianjy, P., and Mukherjee, A. (2018). Understanding deep neural networks with rectified linear units. International Conference on Learning Representations
work page 2018
-
[4]
Balestriero, R., Cosentino, R., Aazhang, B., and Baraniuk, R. (2019). The geometry of deep networks: Power diagram subdivision. Advances in Neural Information Processing Systems , 32
work page 2019
-
[5]
Bauer, U. (2021). Ripser: efficient computation of V ietoris- R ips persistence barcodes. J. Appl. Comput. Topol. , 5(3):391--423
work page 2021
-
[6]
A rank decomposition for the topological classification of neural representations
Beshkov, K. and Einevoll, G. T. (2024). A rank decomposition for the topological classification of neural representations. arXiv preprint arXiv:2404.19710
work page Pith review arXiv 2024
-
[7]
J., and Simsekli, U
Birdal, T., Lou, A., Guibas, L. J., and Simsekli, U. (2021). Intrinsic dimension, persistent homology and generalization in neural networks. Advances in Neural Information Processing Systems , 34:6776--6789
2021
-
[8]
Blaser, N. and Brun, M. (2022). Relative persistent homology. Discrete & Computational Geometry , 68(4):949--963
work page 2022
Show all 53 references
-
[9]
Bubenik, P. et al. (2015). Statistical topological data analysis using persistence landscapes. J. Mach. Learn. Res. , 16(1):77--102
2015
-
[10]
Bubenik, P., Hull, M., Patel, D., and Whittle, B. (2020). Persistent homology detects curvature. Inverse Problems , 36(2):025008
2020
-
[11]
Carlsson, G. (2009). Topology and data. Bulletin of the American Mathematical Society , 46(2):255--308
2009
-
[12]
Dijkstra, E. W. (1959). A note on two problems in connexion with graphs. Numerische mathematik , 1(1):269--271
1959
-
[13]
Dupuis, B., Deligiannidis, G., and Simsekli, U. (2023). Generalization bounds using data-dependent fractal dimensions. In International Conference on Machine Learning , pages 8922--8968. PMLR
2023
-
[14]
Edelsbrunner, H., Harer, J., and Patel, A. K. (2008). Reeb spaces of piecewise linear mappings. In Proceedings of the twenty-fourth annual symposium on Computational geometry , pages 242--250
2008
-
[15]
Fan, F.-L., Huang, W., Zhong, X., Ruan, L., Zeng, T., Xiong, H., and Wang, F. (2023). Deep relu networks have surprisingly simple polytopes. arXiv preprint arXiv:2305.09145
2023 arXiv
-
[16]
Gabrielsson, R. B. and Carlsson, G. (2019). Exposition and interpretation of the topology of neural networks. In 2019 18th ieee international conference on machine learning and applications (icmla) , pages 1069--1076. IEEE
2019
-
[17]
Grigsby, J. E. and Lindsey, K. (2022). On transversality of bent hyperplane arrangements and the topological expressiveness of relu neural networks. SIAM Journal on Applied Algebra and Geometry , 6(2):216--242
2022
-
[18]
Guss, W. H. and Salakhutdinov, R. (2018). On characterizing the capacity of neural networks using algebraic topology. arXiv preprint arXiv:1802.04443
2018 arXiv
-
[19]
Guti \'e rrez-Fandi \ n o, A., P \'e rez-Fern \'a ndez, D., Armengol-Estap \'e , J., and Villegas, M. (2021). Persistent homology captures the generalization of neural networks without a validation set. arXiv preprint arXiv:2106.00012
2021 arXiv
-
[20]
and Rolnick, D
Hanin, B. and Rolnick, D. (2019). Deep relu networks have surprisingly few activation patterns. Advances in neural information processing systems , 32
2019
-
[21]
Hatcher, A. (2002). Algebraic topology . Cambridge University Press
2002
-
[22]
He, K., Zhang, X., Ren, S., and Sun, J. (2015). Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision , pages 1026--1034
2015
-
[23]
Hornik, K. (1991). Approximation capabilities of multilayer feedforward networks. Neural networks , 4(2):251--257
1991
-
[24]
and Hall, J
Huangfu, Q. and Hall, J. J. (2018). Parallelizing the dual revised simplex method. Mathematical Programming Computation , 10(1):119--142
2018
-
[25]
Kleinberg, J. (2006). Algorithm Design . Addison Wesley
2006
-
[26]
and Althoff, M
Kochdumper, N. and Althoff, M. (2019). Representation of polytopes as polynomial zonotopes. arXiv preprint arXiv:1910.07271
2019 arXiv
-
[27]
Kruskal, W. H. and Wallis, W. A. (1952). Use of ranks in one-criterion variance analysis. Journal of the American statistical Association , 47(260):583--621
1952
-
[28]
M., Peterson, C., and Kirby, M
Liu, Y., Cole, C. M., Peterson, C., and Kirby, M. (2023). Relu neural networks, polyhedral decompositions, and persistent homology. In Topological, Algebraic and Geometric Learning Workshops 2023 , pages 455--468. PMLR
2023
-
[29]
Masden, M. (2022). Algorithmic determination of the combinatorial structure of the linear regions of relu neural networks. arXiv preprint arXiv:2207.07696
2022 arXiv
-
[30]
M \'e moli, F., Wan, Z., and Wang, Y. (2022). Persistent laplacians: Properties, algorithms and implications. SIAM Journal on Mathematics of Data Science , 4(2):858--884
2022
-
[31]
F., Pascanu, R., Cho, K., and Bengio, Y
Montufar, G. F., Pascanu, R., Cho, K., and Bengio, Y. (2014). On the number of linear regions of deep neural networks. Advances in neural information processing systems , 27
2014
-
[32]
Munkres, J. (2000). Topology . Pearson, second edition
2000
-
[33]
Munkres, J. R. (2018). Elements of algebraic topology . CRC press
2018
-
[34]
Naitzat, G., Zhitnikov, A., and Lim, L.-H. (2020). Topology of deep neural networks. Journal of Machine Learning Research , 21(184):1--40
2020
-
[35]
M., Carlsson, G
Papamarkou, T., Birdal, T., Bronstein, M. M., Carlsson, G. E., Curry, J., Gao, Y., Hajij, M., Kwitt, R., Lio, P., Di Lorenzo, P., et al. (2024). Position: Topological deep learning is the new frontier for relational learning. In Forty-first International Conference on Machine Learning
2024
-
[36]
Papyan, V., Han, X., and Donoho, D. L. (2020). Prevalence of neural collapse during the terminal phase of deep learning training. Proceedings of the National Academy of Sciences , 117(40):24652--24663
2020
-
[37]
Pascanu, R., Montufar, G., and Bengio, Y. (2013). On the number of response regions of deep feed forward networks with piece-wise linear activations. arXiv preprint arXiv:1312.6098
2013 arXiv
-
[38]
and Leit \ a o, A
Petri, G. and Leit \ a o, A. (2020). On the topological expressive power of neural networks. In TDA \ & \ Beyond
2020
-
[39]
T., Goldberg, K., and Kragic, D
Pokorny, F. T., Goldberg, K., and Kragic, D. (2016). Topological trajectory clustering with relative persistent homology. In 2016 IEEE International Conference on Robotics and Automation (ICRA) , pages 16--23. IEEE
2016
-
[40]
Poole, B., Lahiri, S., Raghu, M., Sohl-Dickstein, J., and Ganguli, S. (2016). Exponential expressivity in deep neural networks through transient chaos. Advances in neural information processing systems , 29
2016
-
[41]
Raghu, M., Poole, B., Kleinberg, J., Ganguli, S., and Sohl-Dickstein, J. (2017). On the expressive power of deep neural networks. In international conference on machine learning , pages 2847--2854. PMLR
2017
-
[42]
Rangamani, A., Lindegaard, M., Galanti, T., and Poggio, T. A. (2023). Feature learning in deep classifiers through intermediate neural collapse. In International Conference on Machine Learning , pages 28729--28745. PMLR
2023
-
[43]
Rieck, B., Togninalli, M., Bock, C., Moor, M., Horn, M., Gumbsch, T., and Borgwardt, K. (2018). Neural persistence: A complexity measure for deep neural networks using algebraic topology. arXiv preprint arXiv:1812.09764
2018 arXiv
-
[44]
Serra, T., Tjandraatmadja, C., and Ramalingam, S. (2018). Bounding and counting linear regions of deep neural networks. In International conference on machine learning , pages 4558--4566. PMLR
2018
-
[45]
and Althoff, M
Sigl, S. and Althoff, M. (2023). M-representation of polytopes. arXiv preprint arXiv:2303.05173
2023 arXiv
-
[46]
E., et al
Singh, G., M \'e moli, F., Carlsson, G. E., et al. (2007). Topological methods for the analysis of high dimensional data sets and 3d object recognition. PBG@ Eurographics , 2:091--100
2007
-
[47]
Tralie, C., Saul, N., and Bar-On, R. (2018). Ripser.py : A lean persistent homology library for python. The Journal of Open Source Software , 3(29):925
2018
-
[48]
F., and Otter, N
Turkes, R., Montufar, G. F., and Otter, N. (2022). On the effectiveness of persistent homology. Advances in Neural Information Processing Systems , 35:35432--35448
2022
-
[49]
Vick, J. W. (2012). Homology theory: an introduction to algebraic topology , volume 145. Springer Science & Business Media
2012
-
[50]
Wasserman, L. (2018). Topological data analysis. Annual Review of Statistics and Its Application , 5(1):501--532
2018
-
[51]
Wheeler, M., Bouza, J., and Bubenik, P. (2021). Activation landscapes as a topological summary of neural network performance. In 2021 IEEE International Conference on Big Data (Big Data) , pages 3865--3870. IEEE
2021
-
[52]
and Wu, D
Zhang, X. and Wu, D. (2020). Empirical studies on the properties of linear regions in deep neural networks. arXiv preprint arXiv:2001.01072
2020 arXiv
-
[53]
and Carlsson, G
Zomorodian, A. and Carlsson, G. (2004). Computing persistent homology. In Proceedings of the twentieth annual symposium on Computational geometry , pages 347--356
2004
Reviewed August 9, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.