REVIEW 3 major objections 5 minor 1 cited by
Evaluating Loss Landscapes from a Topology Perspective
T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read By quantifying the topology of loss landscapes with merge trees and persistence diagrams, this paper shows that saddle count and average persistence track model accuracy, curvature, and the onset of optimization failure in PINNs.
desk verdict A solid exploratory application of TDA to loss landscapes with some real insights, but the quantitative claims rest on unexamined discretization choices and thin experimental evidence. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the merge tree of the loss function's sublevel sets, along with its 0-dimensional persistence diagram. As a threshold on loss is raised, new connected components are born at local minima and die at saddles; the merge tree records these events as degree-three nodes (saddles) and degree-one nodes (minima), while the persistence diagram plots each feature's birth against its death and measures its lifetime by distance to the diagonal. The paper counts the degree-three saddle nodes and averages the persistence values to get two scalar descriptors. These descriptors are what connect the topology of the landscape to accuracy, Hessian eigenvalue, and Hessian trace.
What would settle it
A concrete check: compute merge trees on 3D and higher-dimensional subspaces, sampled along the top 3 to 10 Hessian eigenvectors, for the same trained ResNet-20 and PINN models. If the reported trends in saddle count and average persistence reverse or disappear as the subspace dimension grows, the 2D-projection topology is not representative of the true landscape. A complementary test is to vary the sampling density and the k-nearest-neighbor connectivity of the loss grid; if the relationships between saddle count, persistence, and accuracy or Hessian metrics change sign or vanish, the descriptors are artifacts of the discretization.
Extended reading notes
Core claim
The central discovery is that the 0-dimensional persistent homology of a loss landscape, encoded in a merge tree and persistence diagram, provides quantitative, reproducible descriptors that track model performance and learning dynamics. For ResNet-20, the paper reports that deleting residual connections increases the number of saddle points and decreases the average persistence, and these topological changes move oppositely to the model's accuracy, top Hessian eigenvalue, and Hessian trace. For PINNs, raising the convection coefficient from 1 to 9 makes the loss landscape more complex in both the saddle-count and persistence senses, and this complexity grows together with absolute error, top Hessian eigenvalue, and Hessian trace. The paper reads these paired results as evidence that topological summary statistics capture the shape of the loss function in a way that complements, and sometimes reveals more than, visual inspection.
Load-bearing premise
The load-bearing premise is that a two-dimensional slice through parameter space, defined by two random or Hessian-based directions, preserves enough of the loss landscape's topology that merge trees and persistence diagrams computed on that slice reflect the actual high-dimensional landscape.
Editorial extensions
If this is right
- For image classifiers like ResNet-20, saddle count and average persistence offer a quantitative, parameter-free way to measure landscape smoothness and to compare architectures without needing to inspect plots.
- For physics-informed neural networks, the same two numbers can flag the onset of optimization failure, since they rise together with absolute error and Hessian-based curvature measures as the convection problem gets harder.
- Because merge trees and persistence diagrams are defined for arbitrary-dimensional functions, the approach extends naturally to sampling along more than two directions, potentially revealing structure invisible in 2D slices.
- The merge tree's scale-free nature means it captures structure across different loss scales automatically, removing the need to manually choose visualization ranges to see small-scale features.
Reading between the lines
- A natural extension the authors leave implicit is to use topological descriptors as training-time diagnostics: monitoring saddle count and persistence during optimization could reveal when a run is about to fail, before error metrics rise.
- The 2D-projection caveat suggests a stress test: if the same correlations hold in 3D and higher subspaces, the method becomes a practical tool; if not, the current results are a property of 2D slices. This is testable with existing software.
- The same pipeline could be applied to loss landscapes of other structured models, such as graph neural networks or transformers, where the parameter space is too high-dimensional for visual inspection but low-dimensional projections are still feasible.
- Because the persistence diagram is computed from sublevel sets, it is sensitive to the exact choice of connectivity graph; checking whether the results are stable under different k values would show how much of the signal is intrinsic.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes using topological data analysis (TDA) — specifically merge trees and 0-dimensional persistence diagrams — to quantify the loss landscapes of neural networks. The authors construct 2D loss surfaces by perturbing trained model parameters along either random directions or the top two Hessian eigenvectors (Eq. 4), represent the sampled losses as an unstructured k-nearest-neighbor graph (k = 8 in Appendix A.1), and extract two scalar metrics: the number of saddle points in the merge tree and the average persistence of the 0-dimensional persistence diagram. These metrics are compared with traditional ML metrics (accuracy, absolute error, top Hessian eigenvalue, Hessian trace) for two case studies: ResNet-20 on CIFAR-10 with and without residual connections, and a physics-informed neural network (PINN) solving a 1D convection problem with increasing convection coefficient β. The central claims are that removing residual connections increases the number of saddle points and decreases average persistence, while increasing β increases both saddle count and average persistence, tracking the deterioration of PINN accuracy. The paper also highlights that merge trees reveal multiscale structure not visible in raw loss landscape visualizations.
Significance. If the empirical trends hold, the proposed TDA-based metrics would provide a useful quantitative complement to existing loss landscape visualization and Hessian-based analysis. The paper leverages well-established TDA libraries (TTK) and connects topological summaries to both performance metrics and local curvature metrics, which is a sensible and potentially reproducible framework. The use of four random seeds for the ResNet experiment is a positive feature. However, the evidence in the current manuscript is thin: the central quantitative claims rest on discrete graph summaries whose dependence on the kNN graph, grid resolution, and projection choice is not examined, and the PINN experiment uses a single seed. The claimed insights are plausible but not yet established at the level of a journal publication.
major comments (3)
- [Appendix A.1, Figs. 5 and 6] The saddle count and average persistence are computed on a k-nearest-neighbor graph with k = 8 over a finite 2D grid of loss samples from Eq. (4), but the paper reports no sensitivity analysis or convergence check with respect to k, grid resolution, or perturbation range. Since every subsequent comparison in Figs. 5 and 6 is based on these discrete quantities, the reported differences (e.g., more saddles without residual connections, higher persistence at β = 9) could in principle be artifacts of graph connectivity or sampling density rather than properties of the underlying continuous loss landscape. The authors should either add a sensitivity/convergence study or provide a formal argument that the chosen discretization is sufficient.
- [Section 3.2 and Fig. 6] The PINN experiment uses a fixed random seed (the Fig. 6 caption states seed = 0) and reports no error bars, no repeated trials, and no statistical significance tests, yet the text states as a general trend that increasing β increases saddle count and average persistence. A single optimization trajectory cannot support a claim about the relationship between β and the TDA metrics. Multiple seeds with reported variability are needed before the PINN conclusion can be considered robust.
- [Eq. (4) and Section 4] The TDA descriptors are computed on a 2D projection of the parameter space, and the future-work paragraph explicitly acknowledges that this is not the full space. Because the merge tree and persistence diagram of a projection need not reflect the topology of the high-dimensional loss function, the paper's framing as 'quantifying the topology of loss landscapes' is stronger than what the method supports. The authors should either justify that the chosen 2D projections preserve the relevant topological features (e.g., by comparing random and Hessian projections across repeated draws), or consistently re-frame the claims as properties of the 2D slices. As written, the paper oscillates between these two readings.
minor comments (5)
- [Section 2] The definition of a sublevel set writes L−(v) = {x ∈ D; x ≤ v}; this should be {x ∈ D | f(x) ≤ v} for the scalar function f under consideration.
- [Eq. (3)] The PINN loss in Eq. (3) is written with 1/Nu and 1/Nf prefactors but no square root or explicit weighting convention; please clarify whether the terms are mean squared errors and specify how the weights λi are set.
- [Fig. 6 caption] The caption states β ∈ [1..10], while the text and Fig. 2 show β = 1, 3, 5, 7, 9; please reconcile these ranges and specify exactly which values were used.
- [Fig. 5 caption] The caption reports four random seeds for each ResNet condition, but the figure as described does not show per-seed points or error bars; consider overlaying individual seeds or reporting standard deviations so the reader can assess the spread.
- [Section 2] The statement that saddle nodes in the merge tree are 'degree-three nodes (each connecting two local minima and one other saddle point)' may be too restrictive; a merge tree can have nodes where more than two components merge at the same threshold. Clarify how such degeneracies are handled in the implementation.
Circularity Check
No significant circularity: the TDA metrics are measured directly from sampled loss values and compared, not fitted, and the load-bearing claims do not reduce to self-citation.
full rationale
The paper's quantitative claims are saddle-point counts from the merge tree and average persistence from the 0-dimensional persistence diagram, both computed by applying standard TDA software (TTK) to the sampled 2D loss landscape f(α1, α2) = L(θ + α1δ1 + α2δ2) (Eq. 4). These quantities are direct summaries of the evaluated loss values; no parameter is fitted to accuracy, error, top Hessian eigenvalue, or Hessian trace, and the comparisons with those ML metrics are correlational rather than derived from them. The choice of 2D subspace follows Li et al. (2018) and Yao et al. (2020), and the PINN setup follows Krishnapriyan et al. (2021), but those works supply the sampling methodology and known qualitative phenomena, not the TDA-based measurements or the specific quantitative trends reported here. Even where cited authors overlap with the present author list, the load-bearing content is the newly computed topology of independently evaluated loss surfaces, so the citation is not doing the argumentative work. The acknowledged limitation that only 2D projections are studied is an honest scope caveat, not a circular reduction of the high-dimensional landscape to the projection. The lack of a sensitivity analysis for grid resolution or k-nearest-neighbor connectivity is a robustness concern about discretization artifacts, but it is not evidence that the saddle counts or persistence values are equivalent to the paper's inputs by construction. No circular step can be exhibited from the paper's equations or self-citations.
Assumptions & free parameters
free parameters (3)
- k in k-NN graph =
8
- Landscape grid resolution =
Not stated
- Perturbation range =
Not stated
assumptions (3)
- domain assumption The 2D projection preserves the topology of the true loss landscape.
- domain assumption The k-NN graph with k=8 approximates the connectivity of the sampled landscape.
- domain assumption 0-dimensional persistent homology is sufficient to capture meaningful landscape structure.
Cite this review
Pith. "Pith review of Evaluating Loss Landscapes from a Topology Perspective." pith.science (2026). https://pith.science/paper/DW6Q2AMX
@misc{pith2026241109807,
author = {Pith},
title = {Pith review of: Evaluating Loss Landscapes from a Topology Perspective},
year = {2026},
howpublished = {\url{https://pith.science/paper/DW6Q2AMX}},
note = {Machine review of arXiv:2411.09807}
}
read the original abstract
Characterizing the loss of a neural network with respect to model parameters, i.e., the loss landscape, can provide valuable insights into properties of that model. Various methods for visualizing loss landscapes have been proposed, but less emphasis has been placed on quantifying and extracting actionable and reproducible insights from these complex representations. Inspired by powerful tools from topological data analysis (TDA) for summarizing the structure of high-dimensional data, here we characterize the underlying shape (or topology) of loss landscapes, quantifying the topology to reveal new insights about neural networks. To relate our findings to the machine learning (ML) literature, we compute simple performance metrics (e.g., accuracy, error), and we characterize the local structure of loss landscapes using Hessian-based metrics (e.g., largest eigenvalue, trace, eigenvalue spectral density). Following this approach, we study established models from image pattern recognition (e.g., ResNets) and scientific ML (e.g., physics-informed neural networks), and we show how quantifying the shape of loss landscapes can provide new insights into model performance and learning dynamics.
Figures
Figures from the paper (3 more)
Forward citations
Cited by 1 Pith paper
-
Visualizing Loss Functions as Topological Landscape Profiles
A new topological landscape profile representation lets researchers visualize high-dimensional neural network loss landscapes and reveals that simpler topology correlates with better model performance.
Reference graph
Works this paper leans on
-
[1]
An overview of the T opology T ool K it
Talha Bin Masood, Joseph Budin, Martin Falk, Guillaume Favelier, Christoph Garth, Charles Gueunet, Pierre Guillou, Lutz Hofmann, Petar Hristov, Adhitya Kamakshidasan, Christopher Kappe, Pavol Klacansky, Patrick Laurin, Joshua Levine, Jonas Lukasczyk, Daisuke Sakurai, Maxime Soler, Peter Steneteg, Julien Tierny, Will Usher, Jules Vidal, and Michal Wozniak....
work page 2021
-
[2]
Computing contour trees in all dimensions
Hamish Carr, Jack Snoeyink, and Ulrike Axen. Computing contour trees in all dimensions. Computational Geometry, 24 0 (2): 0 75--94, 2003. ISSN 0925-7721. Special Issue on the Fourth CGC Workshop on Computational Geometry
work page 2003
-
[3]
Swad: Domain generalization by seeking flat minima
Junbum Cha, Sanghyuk Chun, Kyungjae Lee, Han-Cheol Cho, Seunghyun Park, Yunsung Lee, and Sungrae Park. Swad: Domain generalization by seeking flat minima. Advances in Neural Information Processing Systems, 34: 0 22405--22418, 2021
2021
-
[4]
On robustness and transferability of convolutional neural networks
Josip Djolonga, Jessica Yung, Michael Tschannen, Rob Romijnders, Lucas Beyer, Alexander Kolesnikov, Joan Puigcerver, Matthias Minderer, Alexander D'Amour, Dan Moldovan, et al. On robustness and transferability of convolutional neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16458--16468, 2021
work page 2021
-
[5]
Efficient k-nearest neighbor graph construction for generic similarity measures
Wei Dong, Charikar Moses, and Kai Li. Efficient k-nearest neighbor graph construction for generic similarity measures. In Proceedings of the 20th international conference on World wide web, pages 577--586, 2011
work page 2011
-
[6]
Persistent H omology-a S urvey
Herbert Edelsbrunner and John Harer. Persistent H omology-a S urvey. Contemporary mathematics, 453 0 (26): 0 257--282, 2008
work page 2008
-
[7]
Qualitatively characterizing neural network optimization problems
Ian J Goodfellow, Oriol Vinyals, and Andrew M Saxe. Qualitatively characterizing neural network optimization problems. arXiv preprint arXiv:1412.6544, 2014
arXiv 2014
-
[8]
A S urvey of T opology-based M ethods in V isualization
Christian Heine, Heike Leitte, Mario Hlawitschka, Federico Iuricich, Leila De Floriani, Gerik Scheuermann, Hans Hagen, and Christoph Garth. A S urvey of T opology-based M ethods in V isualization. Computer Graphics Forum, 35 0 (3): 0 643--667, 2016
work page 2016
Show all 18 references
-
[9]
Characterizing possible failure modes in physics-informed neural networks
Aditi Krishnapriyan, Amir Gholami, Shandian Zhe, Robert Kirby, and Michael W Mahoney. Characterizing possible failure modes in physics-informed neural networks. Advances in Neural Information Processing Systems, 34: 0 26548--26560, 2021
2021
-
[10]
Adversarial machine learning at scale
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale. arXiv preprint arXiv:1611.01236, 2016
2016 arXiv
-
[11]
Visualizing the loss landscape of neural nets
Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss landscape of neural nets. Advances in neural information processing systems, 2018
2018
-
[12]
Implicit S elf- R egularization in D eep N eural N etworks: E vidence from R andom M atrix T heory and I mplications for L earning
Charles H Martin and Michael W Mahoney. Implicit S elf- R egularization in D eep N eural N etworks: E vidence from R andom M atrix T heory and I mplications for L earning. The Journal of Machine Learning Research, 22 0 (1): 0 7479--7551, 2021
2021
-
[13]
Predicting trends in the quality of state-of-the-art neural networks without access to training or testing data
Charles H Martin, Tongsu Peng, and Michael W Mahoney. Predicting trends in the quality of state-of-the-art neural networks without access to training or testing data. Nature Communications, 12 0 (1): 0 4122, 2021
2021
-
[14]
Generalized out-of-distribution detection: A survey, 2022 a
Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu. Generalized out-of-distribution detection: A survey, 2022 a
2022
-
[15]
Taxonomizing local versus global structure in neural network loss landscapes
Yaoqing Yang, Liam Hodgkinson, Ryan Theisen, Joe Zou, Joseph E Gonzalez, Kannan Ramchandran, and Michael W Mahoney. Taxonomizing local versus global structure in neural network loss landscapes. Advances in Neural Information Processing Systems, pages 18722--18733, 2021
2021
-
[16]
Evaluating natural language processing models with generalization metrics that do not need access to any training or testing data
Yaoqing Yang, Ryan Theisen, Liam Hodgkinson, Joseph E Gonzalez, Kannan Ramchandran, Charles H Martin, and Michael W Mahoney. Evaluating natural language processing models with generalization metrics that do not need access to any training or testing data. arXiv preprint arXiv:...
2022 arXiv
-
[17]
PyHessian : Neural networks through the lens of the hessian
Zhewei Yao, Amir Gholami, Kurt Keutzer, and Michael W Mahoney. PyHessian : Neural networks through the lens of the hessian. In 2020 IEEE international conference on big data (Big data), pages 581--590. IEEE, 2020
2020
-
[18]
A three-regime model of network pruning
Yefan Zhou, Yaoqing Yang, Arin Chang, and Michael W Mahoney. A three-regime model of network pruning. In International Conference on Machine Learning, pages 42790--42809. PMLR, 2023
2023
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.