Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Evaluating Loss Landscapes from a Topology Perspective

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Pith's one-line read By quantifying the topology of loss landscapes with merge trees and persistence diagrams, this paper shows that saddle count and average persistence track model accuracy, curvature, and the onset of optimization failure in PINNs.

desk verdict A solid exploratory application of TDA to loss landscapes with some real insights, but the quantitative claims rest on unexamined discretization choices and thin experimental evidence. read the letter →

arxiv 2411.09807 v1 pith:DW6Q2AMX submitted 2024-11-14 cs.LG cs.AI

classification cs.LGcs.AI MSC 55N3168T07
keywords losslandscapetopologicaldataanalysismergetreepersistencediagramResNetphysics-informedneuralnetworksHessiansaddlepoints
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's claim is that the shape of a neural network's loss landscape can be measured, not just visualised, using two tools from topological data analysis: the merge tree and the persistence diagram. These tools count the landscape's saddle points and measure how long its valleys persist as a loss threshold is raised, yielding quantitative descriptors that can be compared across models. On ResNet-20, the method finds that removing residual connections produces a merge tree with more saddles and lower average persistence, even where the raw loss surface looks deceptively simple. On physics-informed neural networks solving a 1D convection equation, raising the wave speed parameter increases both saddle count and persistence, in step with rising error and Hessian-based measures of curvature. The paper argues that these topological summaries offer a reproducible, scale-free window into why some architectures and problems are easier to optimize than others.

What carries the argument

The central object is the merge tree of the loss function's sublevel sets, along with its 0-dimensional persistence diagram. As a threshold on loss is raised, new connected components are born at local minima and die at saddles; the merge tree records these events as degree-three nodes (saddles) and degree-one nodes (minima), while the persistence diagram plots each feature's birth against its death and measures its lifetime by distance to the diagonal. The paper counts the degree-three saddle nodes and averages the persistence values to get two scalar descriptors. These descriptors are what connect the topology of the landscape to accuracy, Hessian eigenvalue, and Hessian trace.

What would settle it

A concrete check: compute merge trees on 3D and higher-dimensional subspaces, sampled along the top 3 to 10 Hessian eigenvectors, for the same trained ResNet-20 and PINN models. If the reported trends in saddle count and average persistence reverse or disappear as the subspace dimension grows, the 2D-projection topology is not representative of the true landscape. A complementary test is to vary the sampling density and the k-nearest-neighbor connectivity of the loss grid; if the relationships between saddle count, persistence, and accuracy or Hessian metrics change sign or vanish, the descriptors are artifacts of the discretization.

Watch

Extended reading notes

Core claim

The central discovery is that the 0-dimensional persistent homology of a loss landscape, encoded in a merge tree and persistence diagram, provides quantitative, reproducible descriptors that track model performance and learning dynamics. For ResNet-20, the paper reports that deleting residual connections increases the number of saddle points and decreases the average persistence, and these topological changes move oppositely to the model's accuracy, top Hessian eigenvalue, and Hessian trace. For PINNs, raising the convection coefficient from 1 to 9 makes the loss landscape more complex in both the saddle-count and persistence senses, and this complexity grows together with absolute error, top Hessian eigenvalue, and Hessian trace. The paper reads these paired results as evidence that topological summary statistics capture the shape of the loss function in a way that complements, and sometimes reveals more than, visual inspection.

Load-bearing premise

The load-bearing premise is that a two-dimensional slice through parameter space, defined by two random or Hessian-based directions, preserves enough of the loss landscape's topology that merge trees and persistence diagrams computed on that slice reflect the actual high-dimensional landscape.

Editorial extensions

If this is right

  • For image classifiers like ResNet-20, saddle count and average persistence offer a quantitative, parameter-free way to measure landscape smoothness and to compare architectures without needing to inspect plots.
  • For physics-informed neural networks, the same two numbers can flag the onset of optimization failure, since they rise together with absolute error and Hessian-based curvature measures as the convection problem gets harder.
  • Because merge trees and persistence diagrams are defined for arbitrary-dimensional functions, the approach extends naturally to sampling along more than two directions, potentially revealing structure invisible in 2D slices.
  • The merge tree's scale-free nature means it captures structure across different loss scales automatically, removing the need to manually choose visualization ranges to see small-scale features.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the authors leave implicit is to use topological descriptors as training-time diagnostics: monitoring saddle count and persistence during optimization could reveal when a run is about to fail, before error metrics rise.
  • The 2D-projection caveat suggests a stress test: if the same correlations hold in 3D and higher subspaces, the method becomes a practical tool; if not, the current results are a property of 2D slices. This is testable with existing software.
  • The same pipeline could be applied to loss landscapes of other structured models, such as graph neural networks or transformers, where the parameter space is too high-dimensional for visual inspection but low-dimensional projections are still feasible.
  • Because the persistence diagram is computed from sublevel sets, it is sensitive to the exact choice of connectivity graph; checking whether the results are stable under different k values would show how much of the signal is intrinsic.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes using topological data analysis (TDA) — specifically merge trees and 0-dimensional persistence diagrams — to quantify the loss landscapes of neural networks. The authors construct 2D loss surfaces by perturbing trained model parameters along either random directions or the top two Hessian eigenvectors (Eq. 4), represent the sampled losses as an unstructured k-nearest-neighbor graph (k = 8 in Appendix A.1), and extract two scalar metrics: the number of saddle points in the merge tree and the average persistence of the 0-dimensional persistence diagram. These metrics are compared with traditional ML metrics (accuracy, absolute error, top Hessian eigenvalue, Hessian trace) for two case studies: ResNet-20 on CIFAR-10 with and without residual connections, and a physics-informed neural network (PINN) solving a 1D convection problem with increasing convection coefficient β. The central claims are that removing residual connections increases the number of saddle points and decreases average persistence, while increasing β increases both saddle count and average persistence, tracking the deterioration of PINN accuracy. The paper also highlights that merge trees reveal multiscale structure not visible in raw loss landscape visualizations.

Significance. If the empirical trends hold, the proposed TDA-based metrics would provide a useful quantitative complement to existing loss landscape visualization and Hessian-based analysis. The paper leverages well-established TDA libraries (TTK) and connects topological summaries to both performance metrics and local curvature metrics, which is a sensible and potentially reproducible framework. The use of four random seeds for the ResNet experiment is a positive feature. However, the evidence in the current manuscript is thin: the central quantitative claims rest on discrete graph summaries whose dependence on the kNN graph, grid resolution, and projection choice is not examined, and the PINN experiment uses a single seed. The claimed insights are plausible but not yet established at the level of a journal publication.

major comments (3)
  1. [Appendix A.1, Figs. 5 and 6] The saddle count and average persistence are computed on a k-nearest-neighbor graph with k = 8 over a finite 2D grid of loss samples from Eq. (4), but the paper reports no sensitivity analysis or convergence check with respect to k, grid resolution, or perturbation range. Since every subsequent comparison in Figs. 5 and 6 is based on these discrete quantities, the reported differences (e.g., more saddles without residual connections, higher persistence at β = 9) could in principle be artifacts of graph connectivity or sampling density rather than properties of the underlying continuous loss landscape. The authors should either add a sensitivity/convergence study or provide a formal argument that the chosen discretization is sufficient.
  2. [Section 3.2 and Fig. 6] The PINN experiment uses a fixed random seed (the Fig. 6 caption states seed = 0) and reports no error bars, no repeated trials, and no statistical significance tests, yet the text states as a general trend that increasing β increases saddle count and average persistence. A single optimization trajectory cannot support a claim about the relationship between β and the TDA metrics. Multiple seeds with reported variability are needed before the PINN conclusion can be considered robust.
  3. [Eq. (4) and Section 4] The TDA descriptors are computed on a 2D projection of the parameter space, and the future-work paragraph explicitly acknowledges that this is not the full space. Because the merge tree and persistence diagram of a projection need not reflect the topology of the high-dimensional loss function, the paper's framing as 'quantifying the topology of loss landscapes' is stronger than what the method supports. The authors should either justify that the chosen 2D projections preserve the relevant topological features (e.g., by comparing random and Hessian projections across repeated draws), or consistently re-frame the claims as properties of the 2D slices. As written, the paper oscillates between these two readings.
minor comments (5)
  1. [Section 2] The definition of a sublevel set writes L−(v) = {x ∈ D; x ≤ v}; this should be {x ∈ D | f(x) ≤ v} for the scalar function f under consideration.
  2. [Eq. (3)] The PINN loss in Eq. (3) is written with 1/Nu and 1/Nf prefactors but no square root or explicit weighting convention; please clarify whether the terms are mean squared errors and specify how the weights λi are set.
  3. [Fig. 6 caption] The caption states β ∈ [1..10], while the text and Fig. 2 show β = 1, 3, 5, 7, 9; please reconcile these ranges and specify exactly which values were used.
  4. [Fig. 5 caption] The caption reports four random seeds for each ResNet condition, but the figure as described does not show per-seed points or error bars; consider overlaying individual seeds or reporting standard deviations so the reader can assess the spread.
  5. [Section 2] The statement that saddle nodes in the merge tree are 'degree-three nodes (each connecting two local minima and one other saddle point)' may be too restrictive; a merge tree can have nodes where more than two components merge at the same threshold. Clarify how such degeneracies are handled in the implementation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the TDA metrics are measured directly from sampled loss values and compared, not fitted, and the load-bearing claims do not reduce to self-citation.

full rationale

The paper's quantitative claims are saddle-point counts from the merge tree and average persistence from the 0-dimensional persistence diagram, both computed by applying standard TDA software (TTK) to the sampled 2D loss landscape f(α1, α2) = L(θ + α1δ1 + α2δ2) (Eq. 4). These quantities are direct summaries of the evaluated loss values; no parameter is fitted to accuracy, error, top Hessian eigenvalue, or Hessian trace, and the comparisons with those ML metrics are correlational rather than derived from them. The choice of 2D subspace follows Li et al. (2018) and Yao et al. (2020), and the PINN setup follows Krishnapriyan et al. (2021), but those works supply the sampling methodology and known qualitative phenomena, not the TDA-based measurements or the specific quantitative trends reported here. Even where cited authors overlap with the present author list, the load-bearing content is the newly computed topology of independently evaluated loss surfaces, so the citation is not doing the argumentative work. The acknowledged limitation that only 2D projections are studied is an honest scope caveat, not a circular reduction of the high-dimensional landscape to the projection. The lack of a sensitivity analysis for grid resolution or k-nearest-neighbor connectivity is a robustness concern about discretization artifacts, but it is not evidence that the saddle counts or persistence values are equivalent to the paper's inputs by construction. No circular step can be exhibited from the paper's equations or self-citations.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central results rest on choices of projection, graph construction, and persistence dimension. These are assumptions about how well a 2D projection and a k-NN graph represent the high-dimensional loss landscape, not consequences derived from the data.

free parameters (3)
  • k in k-NN graph = 8
    Chosen by hand to approximate pixel connectivity; changes the graph topology and therefore the saddle counts and persistence values.
  • Landscape grid resolution = Not stated
    The number of sampled positions in the 2D subspace is not specified, and merge tree and persistence results depend on this discretization.
  • Perturbation range = Not stated
    The scale of alpha1 and alpha2 in Eq. 4 determines which portion of the landscape is explored and is not specified in the text.
assumptions (3)
  • domain assumption The 2D projection preserves the topology of the true loss landscape.
    All TDA measurements are made on f(alpha1, alpha2) = L(theta + alpha1*delta1 + alpha2*delta2) (Eq. 4); the paper acknowledges this limitation in the future work section.
  • domain assumption The k-NN graph with k=8 approximates the connectivity of the sampled landscape.
    The merge tree is computed on this graph; different choices of k would likely change the number of saddle points and persistence values.
  • domain assumption 0-dimensional persistent homology is sufficient to capture meaningful landscape structure.
    The authors explicitly limit analysis to 0-dimensional features (minima and saddles) and leave higher-dimensional holes for future work.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Evaluating Loss Landscapes from a Topology Perspective." pith.science (2026). https://pith.science/paper/DW6Q2AMX

@misc{pith2026241109807,
  author       = {Pith},
  title        = {Pith review of: Evaluating Loss Landscapes from a Topology Perspective},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DW6Q2AMX}},
  note         = {Machine review of arXiv:2411.09807}
}
read the original abstract

Characterizing the loss of a neural network with respect to model parameters, i.e., the loss landscape, can provide valuable insights into properties of that model. Various methods for visualizing loss landscapes have been proposed, but less emphasis has been placed on quantifying and extracting actionable and reproducible insights from these complex representations. Inspired by powerful tools from topological data analysis (TDA) for summarizing the structure of high-dimensional data, here we characterize the underlying shape (or topology) of loss landscapes, quantifying the topology to reveal new insights about neural networks. To relate our findings to the machine learning (ML) literature, we compute simple performance metrics (e.g., accuracy, error), and we characterize the local structure of loss landscapes using Hessian-based metrics (e.g., largest eigenvalue, trace, eigenvalue spectral density). Following this approach, we study established models from image pattern recognition (e.g., ResNets) and scientific ML (e.g., physics-informed neural networks), and we show how quantifying the shape of loss landscapes can provide new insights into model performance and learning dynamics.

Figures

Figures reproduced from arXiv: 2411.09807 by the authors.

Figure 1
Figure 1. Visualizing loss landscapes for ResNet-20. Compare with Yao et al. [2020]. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Visualizing the failure modes of PINNs. Compare with Krishnapriyan et al. [2021]. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Illustration of generating the two-dimensional loss landscape. In this work, to compute a [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Our loss landscape analysis pipeline. The pipeline includes six stages: (1) [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Quantifying the loss landscape for ResNet-20, with and without residual connections. We [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Quantifying the failure modes of PINNs. Here we show additional experimental results [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Visualizing Loss Functions as Topological Landscape Profiles

    cs.LG 2024-11 conditional novelty 6.0 of 10

    A new topological landscape profile representation lets researchers visualize high-dimensional neural network loss landscapes and reveals that simpler topology correlates with better model performance.

Reference graph

Works this paper leans on

18 extracted references · 14 canonical work pages · cited by 1 Pith paper

  1. [1]

    An overview of the T opology T ool K it

    Talha Bin Masood, Joseph Budin, Martin Falk, Guillaume Favelier, Christoph Garth, Charles Gueunet, Pierre Guillou, Lutz Hofmann, Petar Hristov, Adhitya Kamakshidasan, Christopher Kappe, Pavol Klacansky, Patrick Laurin, Joshua Levine, Jonas Lukasczyk, Daisuke Sakurai, Maxime Soler, Peter Steneteg, Julien Tierny, Will Usher, Jules Vidal, and Michal Wozniak....

  2. [2]

    Computing contour trees in all dimensions

    Hamish Carr, Jack Snoeyink, and Ulrike Axen. Computing contour trees in all dimensions. Computational Geometry, 24 0 (2): 0 75--94, 2003. ISSN 0925-7721. Special Issue on the Fourth CGC Workshop on Computational Geometry

  3. [3]

    Swad: Domain generalization by seeking flat minima

    Junbum Cha, Sanghyuk Chun, Kyungjae Lee, Han-Cheol Cho, Seunghyun Park, Yunsung Lee, and Sungrae Park. Swad: Domain generalization by seeking flat minima. Advances in Neural Information Processing Systems, 34: 0 22405--22418, 2021

  4. [4]

    On robustness and transferability of convolutional neural networks

    Josip Djolonga, Jessica Yung, Michael Tschannen, Rob Romijnders, Lucas Beyer, Alexander Kolesnikov, Joan Puigcerver, Matthias Minderer, Alexander D'Amour, Dan Moldovan, et al. On robustness and transferability of convolutional neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16458--16468, 2021

  5. [5]

    Efficient k-nearest neighbor graph construction for generic similarity measures

    Wei Dong, Charikar Moses, and Kai Li. Efficient k-nearest neighbor graph construction for generic similarity measures. In Proceedings of the 20th international conference on World wide web, pages 577--586, 2011

  6. [6]

    Persistent H omology-a S urvey

    Herbert Edelsbrunner and John Harer. Persistent H omology-a S urvey. Contemporary mathematics, 453 0 (26): 0 257--282, 2008

  7. [7]

    Qualitatively characterizing neural network optimization problems

    Ian J Goodfellow, Oriol Vinyals, and Andrew M Saxe. Qualitatively characterizing neural network optimization problems. arXiv preprint arXiv:1412.6544, 2014

  8. [8]

    A S urvey of T opology-based M ethods in V isualization

    Christian Heine, Heike Leitte, Mario Hlawitschka, Federico Iuricich, Leila De Floriani, Gerik Scheuermann, Hans Hagen, and Christoph Garth. A S urvey of T opology-based M ethods in V isualization. Computer Graphics Forum, 35 0 (3): 0 643--667, 2016

Show all 18 references
  1. [9]

    Characterizing possible failure modes in physics-informed neural networks

    Aditi Krishnapriyan, Amir Gholami, Shandian Zhe, Robert Kirby, and Michael W Mahoney. Characterizing possible failure modes in physics-informed neural networks. Advances in Neural Information Processing Systems, 34: 0 26548--26560, 2021

  2. [10]

    Adversarial machine learning at scale

    Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale. arXiv preprint arXiv:1611.01236, 2016

  3. [11]

    Visualizing the loss landscape of neural nets

    Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss landscape of neural nets. Advances in neural information processing systems, 2018

  4. [12]

    Implicit S elf- R egularization in D eep N eural N etworks: E vidence from R andom M atrix T heory and I mplications for L earning

    Charles H Martin and Michael W Mahoney. Implicit S elf- R egularization in D eep N eural N etworks: E vidence from R andom M atrix T heory and I mplications for L earning. The Journal of Machine Learning Research, 22 0 (1): 0 7479--7551, 2021

  5. [13]

    Predicting trends in the quality of state-of-the-art neural networks without access to training or testing data

    Charles H Martin, Tongsu Peng, and Michael W Mahoney. Predicting trends in the quality of state-of-the-art neural networks without access to training or testing data. Nature Communications, 12 0 (1): 0 4122, 2021

  6. [14]

    Generalized out-of-distribution detection: A survey, 2022 a

    Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu. Generalized out-of-distribution detection: A survey, 2022 a

  7. [15]

    Taxonomizing local versus global structure in neural network loss landscapes

    Yaoqing Yang, Liam Hodgkinson, Ryan Theisen, Joe Zou, Joseph E Gonzalez, Kannan Ramchandran, and Michael W Mahoney. Taxonomizing local versus global structure in neural network loss landscapes. Advances in Neural Information Processing Systems, pages 18722--18733, 2021

  8. [16]

    Evaluating natural language processing models with generalization metrics that do not need access to any training or testing data

    Yaoqing Yang, Ryan Theisen, Liam Hodgkinson, Joseph E Gonzalez, Kannan Ramchandran, Charles H Martin, and Michael W Mahoney. Evaluating natural language processing models with generalization metrics that do not need access to any training or testing data. arXiv preprint arXiv:...

  9. [17]

    PyHessian : Neural networks through the lens of the hessian

    Zhewei Yao, Amir Gholami, Kurt Keutzer, and Michael W Mahoney. PyHessian : Neural networks through the lens of the hessian. In 2020 IEEE international conference on big data (Big data), pages 581--590. IEEE, 2020

  10. [18]

    A three-regime model of network pruning

    Yefan Zhou, Yaoqing Yang, Arin Chang, and Michael W Mahoney. A three-regime model of network pruning. In International Conference on Machine Learning, pages 42790--42809. PMLR, 2023

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.