{"id":"84c4099c-2d45-4c76-babb-2ae42644bfea","arxiv_id":"2411.12136","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A new topological landscape profile representation lets researchers visualize high-dimensional neural network loss landscapes and reveals that simpler topology correlates with better model performance.","lead":"The paper introduces a way to visualize the high-dimensional loss landscape of a neural network using topological data analysis, turning complex 3D and 4D landscapes into 2D profile plots. It uses these profiles to show that better-performing models have simpler loss landscape topology, and that topology varies most near performance transitions.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The merge tree is computed on a sparse k-nearest-neighbor graph (k=4n, §3.1), not on a faithful triangulation of the sampled grid; for n=3, k=12 omits most of the 26-neighbor cubical neighborhood, so spurious critical points may drive the reported 'simpler topology for better models' claim.","rationale":"The reader's weakest assumption identifies the same load-bearing issue: the merge tree is computed on a k-NN graph of a coarse grid, not on the continuous loss function. I agree and would not change the conditional verdict. The central claim that 'the topology of the loss landscape is simpler for better-performing models' depends entirely on the critical points and merge events extracted by the merge tree. Because the graph used to compute that tree is not a faithful cell complex of the sampled domain, there is a concrete mechanism for spurious local minima and missing saddle connections. The paper's analogy to image pixels (k=8 in 2D) does not justify k=4n in 3D or 4D, and no sensitivity analysis is provided. The paper does have independent strengths: it builds on a published TDA library (TTK), limits claims to 3D/4D subspaces, and acknowledges sampling costs. But without a check that the discrete graph reproduces the topology of the sampled function, the qualitative empirical conclusions are not secured. A recomputation with exact grid connectivity, or a synthetic quadratic test showing the k=12 graph introduces false minima, would settle this directly. Therefore the appropriate verdict remains CONDITIONAL, pending that validation.","tokens_in":9880,"tokens_out":8941,"duration_ms":110601,"concrete_test":"Recompute the merge trees and landscape profiles for the PINN experiments in Section 4.1 from the same stored loss samples, but replace the k=4n approximate k-NN graph with the exact connectivity of the underlying cubical grid: for the 3D case use 26-neighbor connectivity (or TTK's native regular-grid loader), and for the 4D case use the corresponding hypercubical connectivity. Then compare, per beta and learning rate, the number of merge-tree leaves/branches and the visual profile shapes against the published figures. If the low-error funnel vs high-error bowl distinction and the transition-region variability survive exact connectivity, the concern is refuted; if spurious critical points disappear or the ordering reverses, the central claim is a graph artifact.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing assumption is that the merge tree computed on the k-nearest-neighbor graph of Section 3.1 is the merge tree of the sampled loss function. The paper uses k=4n: for the 3D PINN landscapes this is k=12 on a 41^3 grid, and for 4D k=16 on a 41^4 grid. A k-NN graph is not a triangulation or cubical complex of the sampled grid: it contains only a subset of the near-neighbor edges (a full 3D cubical neighborhood has 26 neighbors), and the symmetric k-NN pruning can further remove edges. Merge-tree critical points on a graph are defined by graph adjacency: a vertex is a local minimum only if no adjacent vertex has lower loss, and components merge only through graph paths. Omitting an edge to a lower-valued diagonal neighbor can therefore create a spurious local minimum, and missing paths can postpone or eliminate legitimate merges. The landscape profiles are direct drawings of this merge tree, so the reported contrast between 'funnel-like' low-error landscapes and 'rougher, bowl-like' high-error landscapes, and especially the conclusion that better models have simpler topology, could be produced by the choice of k rather than by the loss landscape itself. The paper provides no sensitivity analysis over k or r, and no code or data, so this alternative explanation is not ruled out. This is not an objection to TDA in general; it is a request to validate that the discrete graph faithfully represents the continuous function sampled on the grid.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces a topological landscape profile representation for loss landscapes, built on a merge tree computed over a k-nearest-neighbor graph of a grid-sampled loss function in the subspace spanned by the top Hessian eigenvectors. The authors claim this representation enables visualization of three- and four-dimensional loss landscapes, and they report two empirical findings from PINN and UNet models: (1) better-performing models have topologically simpler loss landscapes (e.g., funnel-like versus bowl-like), and (2) there is greater landscape-shape variability near transitions from low to high model performance. The method is a standard TDA pipeline (sample grid, construct neighborhood graph, compute merge tree, render landscape profile), adapted from Oesterling et al. (2013) with a valley-oriented interpretation.","tokens_in":10266,"tokens_out":3933,"duration_ms":40110,"significance":"If validated, the proposed representation would be a useful diagnostic tool for comparing higher-dimensional loss landscapes, going beyond one- and two-dimensional slices. A clear strength is that the paper uses established TDA machinery (merge trees, landscape profiles) on genuinely trained models (PINNs and UNets), and it identifies concrete, falsifiable-looking qualitative contrasts between low- and high-error regimes. However, the central empirical claims are not yet supported at the level expected for a rigorous contribution: 'simpler topology' is never measured, the graph-construction choice is not validated with sensitivity analysis, and the second use case (UNet training) does not actually use the proposed profile representation. The paper also does not provide code or data, which limits reproducibility. With quantitative simplicity metrics, a sensitivity analysis of k and grid resolution, and a demonstration of the actual profile on the UNet setting, the contribution could be solid; in its current form the evidence is mostly visual and the claims outrun the analysis.","major_comments":[{"comment":"The merge tree is computed on a symmetric k-nearest-neighbor graph with k=4n (k=12 for the 3D PINN landscapes, k=16 for 4D). On a regular 3D grid the full cubical neighborhood has 26 neighbors, so k=12 omits most of the diagonal near-neighbor edges. Merge-tree critical points and component merges are defined by graph adjacency; omitted edges can create spurious local minima and postpone legitimate merges, directly changing the resulting landscape profile. Since the 'simpler topology for better models' finding is drawn from these profiles, the result may be an artifact of the graph-construction choice rather than a property of the loss function. The paper reports no sensitivity analysis over k or the grid resolution r=41, and provides no code or data to rule out this alternative explanation. Please validate the pipeline using, for example, the full cubical complex or a range of k values, and report how the profiles and the qualitative conclusions change.","section":"3.1"},{"comment":"The central claim that 'the topology of the loss landscape is simpler for better-performing models' is not operationalized. 'Simpler' is never defined or measured; the paper provides no quantity such as number of persistent minima, total persistence, merge-tree leaf count, or any statistical comparison across random seeds and hyperparameters. The visual evidence in Figure 3 is suggestive but does not support a general conclusion absent a quantitative summary. Please define a simplicity metric, compute it across the hyperparameter grid, and test whether differences between low- and high-error regimes are statistically significant.","section":"Abstract; Section 5"},{"comment":"The UNet study is presented as a use case for the proposed method, but Section 4.2 states that two-dimensional loss landscapes were computed using the top two Hessian eigenvectors, and Figure 4 shows standard 2D loss heat maps, not topological landscape profiles. Consequently, this section does not demonstrate the new representation's ability to visualize higher-dimensional landscapes or to extract topology information for this model. Please either apply the proposed profile construction to the UNet setting (even at 2D or 3D resolution) to show the versatility claim, or explicitly reframe Section 4.2 as a baseline comparison using classical visualization.","section":"4.2; Figure 4"}],"minor_comments":[{"comment":"The text states that each α_i is an equally spaced integer between 0 and r, with r=41 and the center corresponding to the original model, i.e., Σ α_i δ_i = 0. If α_i ranges from 0 to 41, the center is 20.5, not 0; please clarify the coordinate centering (e.g., α_i ∈ [−20, 20] or a shifted indexing).","section":"3.1"},{"comment":"Since the samples lie on a regular grid, the connectivity is already known; the use of an approximate k-NN graph and its symmetric pruning is unusual. Please justify why a direct cubical complex or structured grid adjacency was not used, and note that the algorithm of Dong et al. (2011) is approximate, introducing another potential source of variability.","section":"3.1"},{"comment":"The discussion of Figure 3 states that landscapes are 'stable across seeds' for low and high β, but this is asserted from five visualizations with no quantitative measure of similarity. Consider reporting a simple metric (e.g., persistence, branch count, or profile distance) across seeds to substantiate the stability claim.","section":"4.1"},{"comment":"The concluding sentence says 'our approach opens up the door to visualizing higher-dimensional loss landscapes,' but the experiments stop at four dimensions and the sampling cost is acknowledged as a limitation. This is fine, but the wording could be softened to avoid overclaiming practical scalability.","section":"5"},{"comment":"The paper does not include a data or code availability statement. Given the empirical and visual nature of the work, making the sampling and topological pipeline available would substantially improve reproducibility.","section":"Global"}],"recommendation":"major_revision","confidential_remarks":"The paper is submitted to a workshop (Symmetry and Geometry in Neural Representations) and follows closely the authors' prior work Xie et al. (2024), which is cited as the motivation but is only an arXiv preprint. The relationship between the two papers (shared methodology, division of new content) should be clarified. The core idea is promising and the major concerns above are addressable within the standard revision scope, provided the authors add sensitivity analysis, a quantitative definition of simplicity, and a genuine demonstration of the profile on the UNet case. If those additions are made, the paper would likely be a useful contribution to the TDA-for-ML visualization community."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take. The paper is a solid, well-scoped visualization contribution. It adapts the topological landscape profile of Oesterling et al. (2013) from density maxima to loss minima, and shows that merge trees can be rendered as 2D valley-like profiles for 3D and 4D Hessian subspaces. That is new and useful: the loss-landscape community has mostly stuck to 1D/2D slices, so a principled way to see higher-dimensional structure is a real step forward. The PINN and UNet examples are illustrative, and the qualitative observations—funnel-like profiles for low-error models, bowl-like for high-error, greater variability near the transition—are plausible and match prior work like Krishnapriyan et al. (2021).\n\nThe soft spot is the load-bearing assumption that the kNN graph faithfully represents the sampled loss function. The paper uses k=4n, which gives k=12 for the 3D grids and k=16 for 4D. For a regular grid that is only a fraction of the 26 (or 80) neighbor cubical neighborhood. Missing diagonal edges can create spurious local minima and alter merge events, and the authors do not report any sensitivity analysis over k or r. So the claim that better models have simpler topology could be an artifact of the graph construction rather than the landscape itself. This is not a fatal flaw—with code and a sensitivity study it would be easy to check—but as it stands the empirical conclusion is more fragile than the paper suggests.\n\nA second, related issue is that 'simpler' is never defined quantitatively. The paper shows examples but does not count critical points, compare persistence across models, or run any significance tests. That's acceptable for a visualization-first paper, but it means the abstract's claim is a qualitative observation, not a measured result. No code or data is provided, which makes independent validation harder.\n\nFor a reader in the loss-landscape or TDA-for-ML space, this is worth engaging with. The method should be citable as a visualization technique, and the paper is a reasonable candidate for peer review. I would send it to reviewers with a request to focus on the kNN choice and to ask for a sensitivity analysis or code release. If the authors can show the profiles are stable across k and r, I'd be much more comfortable with the 'simpler topology' claim.","headline":"Useful visualization method with a suggestive but under-supported empirical claim; the kNN hyperparameter deserves sensitivity analysis before the 'simpler topology for better models' conclusion is accepted.","tokens_in":10799,"tokens_out":2556,"would_cite":false,"duration_ms":27454,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that compressing a loss function's minima and saddle points into a two-dimensional topological landscape profile makes 3D and 4D loss landscapes visible and reveals that better-performing models have simpler…","keywords":["topological landscape profile","loss landscape","merge tree","topological data analysis","Hessian eigenvectors","physics-informed neural networks","model performance diagnosis"],"falsifier":"Resample the same models on a finer grid, say $r=81$ or $r=121$, and with different values of $k$, then recompute the merge tree and landscape profiles; if the ranking of better versus worse models by topological simplicity changes, or if spurious basins appear and vanish with resolution, the reported simplicity claim is an artifact of the discretization rather than a property of the loss landscape. A complementary check is to train models with matched training loss but different test accuracy and see whether the simpler-profile association follows test performance.","tokens_in":9717,"feed_emoji":"📉","tokens_out":8318,"duration_ms":79877,"temperature":0.7,"pith_summary":"This paper introduces a two-dimensional topological landscape profile built from the merge tree of a loss function, and argues that this representation makes three- and four-dimensional Hessian-based loss landscapes visible in a way that one- or two-direction slices cannot. It applies the profile to physics-informed neural networks solving a convection problem and to a segmentation network with a CRF layer, and reports two qualitative findings: loss landscapes are topologically simpler for better-performing models, and landscape shape varies most near transitions between low and high performance. If these findings hold, the loss landscape becomes a practical visual diagnostic for model quality and training dynamics rather than an object that can only be inspected in one or two random directions.","feed_headline":"Simpler loss landscapes mark better-performing models","feed_subtitle":"New 2D profiles render 3D and 4D loss surfaces, exposing shape shifts as performance changes.","key_machinery":"The load-bearing construction is the merge tree computed from loss values sampled on a grid of resolution $r=41$ spanning the top $n$ Hessian eigenvectors, with connectivity defined by a symmetric $k$-nearest-neighbor graph using $k=4n$. As the loss threshold rises, the merge tree records when connected components of sublevel sets appear at local minima and when they merge at saddle points, so each branch ending in a minimum becomes a basin in the resulting topological landscape profile. Each basin is drawn as a stack of rectangles whose cumulative width encodes the number of sampled points along the corresponding merge-tree branch, with basins colored by average loss and annotated by their minima and saddle points. The profile construction adapts an earlier landscape representation for high-dimensional point clouds from density hills to loss basins.","core_discovery":"The paper's central claim is that the critical-point structure of an n-dimensional loss landscape—its local minima, the saddle points connecting them, and the loss thresholds at which basins merge—can be encoded in a merge tree and redrawn as a two-dimensional landscape of basins, and that this representation is faithful enough to expose properties of the underlying model. In the examples studied, models with lower error or higher test accuracy show simpler topology: fewer and more funnel-like basins, with the low-loss region dominating the profile. Near the hyperparameter settings where a model crosses from performing well to performing poorly, the paper reports the largest variation in landscape shape across random seeds, with some runs showing funnel-like profiles and others bowl-like ones. The paper also reports that moving from three sampled directions to four preserves the global shape while adding finer spiky structure, which it presents as evidence that the profile captures information beyond two-dimensional slices.","pith_inferences":["Beyond the paper's examples, the same representation could be used as a live training monitor: the observed deepening and flattening of the main basin over epochs suggests that a scalar summary such as basin count or basin depth could track learning progress.","The merge-tree construction does not depend on Hessian directions, so the profile could equally summarize random or filter-normalized subspaces; testing the simplicity claim across subspace choices would show whether the finding is tied to Hessian geometry.","A natural experiment the paper does not run is comparing models with equal training loss but different test accuracy; if the simpler-topology pattern tracks test performance, the profile would be a generalization diagnostic rather than just an optimization diagnostic.","The resolution dependence is easy to probe: recomputing one PINN profile at $r=81$ and comparing it with $r=41$ would quantify how much of the reported basin structure is an artifact of the grid."],"forward_implications":["Landscape profiles computed in three or four Hessian directions give a visual summary that can be compared across models, random seeds, and hyperparameter settings without collapsing the landscape to one or two dimensions.","If the reported pattern is general, better-performing models should consistently show fewer dominant basins and more funnel-like profiles in the subspaces spanned by their top Hessian eigenvectors.","At hyperparameter values where performance begins to degrade, landscape shape across random seeds should be less consistent, reflecting a mixture of funnel-like and bowl-like profiles.","Because the profile is annotated with the average loss of each basin, it encodes optimization state as well as topology, making the transition from a shallow high-loss basin to a deep stable basin visible over training."],"supporting_citations":[{"why":"Supplies the filter-normalized two-direction loss landscape sampling that the higher-dimensional sampling generalizes.","marker":"Li et al. (2018)"},{"why":"Provides the Hessian eigenvector computation used to define the sampled subspace.","marker":"Yao et al. (2020)"},{"why":"Introduces the landscape profile representation that the paper adapts from density hills to loss basins.","marker":"Oesterling et al. (2013)"},{"why":"Provides the merge-tree and contour-tree computation in all dimensions that underpins the critical-point encoding.","marker":"Carr et al. (2003)"},{"why":"Supplies the PINN convection benchmark and the earlier observation that failure modes are tied to loss landscape complexity.","marker":"Krishnapriyan et al. (2021)"},{"why":"Provides the merge-tree computation software used in the pipeline.","marker":"Bin Masood et al. (2021)"},{"why":"Supplies the approximate nearest-neighbor graph construction used to connect points on the sampled grid.","marker":"Dong et al. (2011)"},{"why":"Is the prior two-dimensional topology-based loss landscape analysis that this work extends into a visual representation.","marker":"Xie et al. (2024)"}],"fun_headline_variants":["Topological profiles reveal simpler loss in better models","Loss landscape topology maps to model performance","2D profiles compress high-D loss surfaces truthfully","Simpler topology of loss surfaces flags better models","Loss shapes visualized via topological merge trees"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument depends on the assumption that sampling the loss at 41 evenly spaced points along each direction and connecting nearby samples with $k=4n$ neighbors faithfully captures the true shape of the loss function rather than creating artificial bumps and valleys.","fun_headline_variants_meta":{"raw":{"variants":["Topological profiles reveal simpler loss in better models","Loss landscape topology maps to model performance","2D profiles compress high-D loss surfaces truthfully","Simpler topology of loss surfaces flags better models","Loss shapes visualized via topological merge trees"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000192,"raw_usage":{"total_tokens":1339,"prompt_tokens":927,"completion_tokens":412,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":543,"completion_tokens_details":{"reasoning_tokens":344}},"tokens_in":543,"tokens_out":412,"duration_ms":4933,"temperature":1.0,"reasoning_tokens":344,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T17:51:32.928506+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Resample the same models on a finer grid, say $r=81$ or $r=121$, and with different values of $k$, then recompute the merge tree and landscape profiles; if the ranking of better versus worse models by topological simplicity changes, or if spurious basins appear and vanish with resolution, the reported simplicity claim is an artifact of the discretization rather than a property of the loss landscape. A complementary check is to train models with matched training loss but different test accuracy and see whether the simpler-profile association follows test performance.","supporting_citations":[],"review_version":1}