Uncovering Critical Sets of Deep Neural Networks via Sample-Independent Critical Lifting

Leyang Zhang; Tao Luo; Yaoyu Zhang

REVIEW 2 major objections 5 minor 18 references

Uncovering Critical Sets of Deep Neural Networks via Sample-Independent Critical Lifting

T0 review · 2 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper establishes that, for sample sizes larger than a parameter-count threshold, widening a neural network creates sample-dependent critical points and saddles, and that previously studied critical embeddings fail to account for all…

desk verdict A careful existence theory for sample-dependent lifted critical points; the main theorems hold up, with minor clarifications needed on assumptions and threshold statements. read the letter →

arxiv 2505.13582 v1 pith:TF2I3X6Y submitted 2025-05-19 cs.LG

Leyang Zhang , Yaoyu Zhang , Tao Luo This is my paper · ORCID

classification cs.LG

keywords sample-dependentcriticalpointssample-independentliftinglosslandscapeneuralnetworkwidthsaddleembeddingsoverparameterizationrealanalyticfunctions

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper studies how the critical points of a neural network's loss surface depend on the training sample. It introduces a sample-independent critical lifting operator, which maps a parameter of a narrower network to the parameters of a wider network that preserve both the output function and criticality for every sample for which the narrower parameter is critical. The central result is that once the number of samples exceeds a threshold proportional to the narrower network's parameter count, sample-dependent lifted critical points are unavoidable, and beyond a slightly larger threshold some of these are saddles. The message is that data dependence of the critical set is not a degenerate accident: for sufficiently large datasets it is the typical situation, and the extra critical points are not local minima.

What carries the argument

The engine is the sample-independent critical lifting operator, defined as the intersection over all samples for which the narrower parameter is critical, of the wider parameters that preserve both output and criticality. For one-hidden-layer networks, the decisive object is the analytic function $\varphi(w') = \sum_{i=1}^n \partial_p \ell(H(\theta_{\mathrm{narr}},x_i),y_i)\,\sigma(w'\cdot x_i)$, whose zeros decide whether a dead output neuron with weight zero and input weight $w'$ is critical. Because the activation is non-polynomial analytic and the inputs satisfy $x_i\neq 0$ and $x_i\pm x_j\neq 0$, the functions $w'\mapsto \sigma(w'\cdot x_i)$ are linearly independent; hence $\varphi$ is nonconstant analytic, and its zero set has measure zero, making criticality of newly added neurons exceptional for a fixed sample. The threshold $1+(d+1)m$ comes from counting equations versus unknowns in the matrix whose kernel contains the vector of loss derivatives: exceeding it guarantees a nonzero kernel direction, which allows the authors to choose output values that make $\theta_{\mathrm{wide}}$ critical, and the second threshold $2+(d+1)m$ keeps a kernel direction after adding an extra constraint, yielding a nonzero-loss critical point that a gradient-flow argument converts into a saddle.

What would settle it

Fix a one-hidden-layer tanh network with $m=1$, input dimension $d=1$, and four sample inputs containing a zero, say $x_1=0$ plus three generic nonzero inputs, so the threshold $n>3$ is met but the generic-position condition fails. Vary the outputs to search for parameters of the form $(a_1,w_1,0,w')$ that are critical for one output vector but not for another; if none exist, the generic-position assumption is essential to Theorem 4.2.1.

Watch

Extended reading notes

Core claim

The paper claims that critical sets of deep neural networks have a sample-independent core and a sample-dependent shell, and that the two are separated by a lifting operator. Fixing a critical point of a narrower network, one can add zero-output neurons with freely chosen input weights; these parameters preserve the output function but, for a fixed sample, almost none of them are critical (Proposition 4.2.1). However, when the sample size is large enough, the situation reverses: Theorem 4.2.1 states that for $m$ neurons, input dimension $d$, and sample size $n>1+(d+1)m$, there exist sample-dependent lifted critical points of the form (2), and for $n>2+(d+1)m$ there exist sample-dependent lifted saddles. The multi-layer generalization, Theorem 4.2.2, gives the same phenomenon with thresholds expressed in terms of the narrower network's parameter count and the intermediate widths. The paper also proves by example that previously studied splitting, null, and compatible embeddings do not generate all sample-independent lifted critical points, so the full sample-independent structure is richer than earlier embedding operators captured.

Load-bearing premise

The proofs rely on the sample inputs being in generic position: none is zero and no two are equal or opposite ($x_i\pm x_j\neq 0$); if a dataset contains a zero input or a pair like $x$ and $-x$, the neuron functions can become linearly dependent and the existence results for sample-dependent saddles are not proved.

Editorial extensions

If this is right

For one-hidden-layer networks, every sample-independent lifted critical point obtained from a nonzero-loss critical point is a saddle, so gradient methods that avoid saddles are not trapped there.
The threshold $n>1+(d+1)m$ identifies a regime in which the loss landscape of a wider network has critical points that cannot be predicted from the narrower network alone; sample information matters exactly when the dataset is large relative to the narrow network's parameter count.
The multi-layer version gives explicit sample-size thresholds involving the widths of intermediate layers, so the phenomenon persists in depth rather than being special to one hidden layer.
Previously proposed splitting, null, and compatible embeddings are insufficient: there are sample-independent lifted critical points, such as parameters producing identical zero output functions, that no combination of those embeddings generates.
The sample-dependent saddles found are degenerate and may lack a negative eigenvalue, so standard saddle-avoidance results do not apply directly near them.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The finite-sample thresholds suggest an interpolation between two regimes: with tiny sample size, the critical sets of wide networks are largely architecture-determined, while with large sample size, sample-specific saddles proliferate; this may be observable as a change in how gradient trajectories from random initializations behave as $n$ grows.
A testable extension would replace the analytic, non-polynomial activation by a polynomial or ReLU activation: the linear-independence lemma can fail there, so the sample-dependent saddles may disappear or become nongeneric, separating the analyticity assumption from the counting argument.
The same lifting argument should apply to other symmetric architectures, such as convolutional or residual networks, whenever zero-weight neurons preserve the output function and the relevant neuron maps are linearly independent; the threshold would then be parameter-count based.
The generic-position requirement (no zero or opposite inputs) holds for generic continuous data but is systematically violated by duplicate samples, mirrored samples, or constant features, so datasets with such structure may exhibit a different critical-set geometry.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

What is actually new here is the formal separation of sample-independent from sample-dependent lifted critical points, plus the counterexample showing that the previously studied embedding operators (splitting, null, compatible) do not exhaust the sample-independent structure. That is a genuine step beyond the embedding-principle literature, which mostly assumed sample-independent embeddings. The existence theorems for sample-dependent critical points and saddles, with explicit sample-size thresholds, are also new and useful.

The proofs are the real strength. They rest on the linear independence of neurons under the generic-position assumptions (xi nonzero, xi ± xj nonzero) and on standard zero-measure properties of real-analytic functions. I checked the main chain: the kernel-construction in Proposition A.2.3 is sound, and the saddle argument via Lemma A.1.2 is correct because output-preserving perturbations give points with the same loss and nonzero gradient. The multi-layer version is dense but follows the same logic.

Soft spots are minor. First, the Section 4.1 counterexample silently requires sigma(0)=0; the appendix states it, the main text does not. That should be fixed. Second, the thresholds: the theorem statements use strict inequalities (n > 1+(d+1)m, n > 2+(d+1)m) while the proofs use non-strict ones (n >= ..., n >= ...). Since n is an integer, the proofs actually establish existence for one fewer sample than the theorems claim. So the theorems are conservative, not wrong. A stress-test note claimed this "understates sample size needed," which is backwards; the only real issue is a minor mismatch in how the thresholds are advertised. Third, the multi-layer proof could use a worked example, but that is presentation.

The paper is honest about its open questions, especially whether embedding operators fully characterize sample-independent lifted points, and the dynamics near degenerate saddles. It is a theory paper for people working on loss landscapes and the embedding principle; the toy illustration is small but does support the claims.

My verdict: worth a serious referee. The central results appear correct, the proofs are reproducible, and the flaws are easily fixed. I would engage with it.

Referee Report

2 major / 5 minor

Summary. The paper introduces a sample-independent critical lifting operator that maps a critical parameter of a narrower fully connected network to the set of parameters of a wider network that preserve both the output function and criticality for every sample for which the original parameter is critical (Definition 4.1). It then separates sample-independent from sample-dependent lifted critical points. The main results are: (i) an example showing that known embedding operators (splitting, null, and compatible embeddings) do not exhaust the sample-independent lifted critical points for three-hidden-layer networks; (ii) for one-hidden-layer networks, a zero-measure statement and a saddle statement for lifted critical points of the form (2); (iii) existence of sample-dependent lifted critical points and saddles once the sample size exceeds thresholds of the form n > 1 + (d+1)m and n > 2 + (d+1)m (Theorem 4.2.1); and (iv) analogous multi-layer statements with thresholds depending on the parameter count of the narrower network (Theorem 4.2.2). Proofs in the appendix use real analyticity, a linear-independence lemma for neurons, Mityagin's zero-measure lemma, and a kernel-membership construction to choose sample outputs. The paper is theoretical, with a small numerical illustration in Section 5.

Significance. If the results are correct, the paper provides a useful formal framework for separating architecture-driven, sample-independent structure from sample-dependent structure in neural network loss landscapes. The explicit sample-size thresholds are concrete and checkable, and the proof strategy is transparent: the linear-independence lemma for neurons is carefully stated, and the kernel arguments in the appendix are logically coherent. The paper is also honest about its limitations, including the generic-position condition on sample inputs and the restriction to analytic activations and width-wise lifting. A notable strength is that the main existence theorems are proved from the paper's own definitions, with no fitted parameters and no circular reliance on the target result. The counterexample to completeness of previously studied embeddings is simple and effective, although it needs a stated hypothesis on the activation (see major comments). Overall, the contribution is a solid incremental advance over the embedding-principle literature, and the open questions in the conclusion are appropriate.

major comments (2)

[Section 4.1 and Proposition A.2.1] The main-text example claims that all parameters in E_narr and E_wide yield the constant zero output function and are critical for arbitrary samples. This claim is true only when σ(0)=0, but that hypothesis appears only in the appendix Proposition A.2.1, not in the main-text statement of the example. Under the standing Assumption 3.1 alone, activations such as sigmoid or Gaussian satisfy the assumption yet have σ(0)≠0, and for those activations the parameters of the form (3) do not produce the zero output function and the derivative computations in Proposition A.2.1 fail. The hypothesis σ(0)=0 should be stated explicitly in Section 4.1 where the example is introduced, and in Remark 4.2 if the generalization is claimed for arbitrary L≥3.
[Proposition A.2.3 and Theorem 4.2.1] There is a threshold mismatch between the statement of Proposition A.2.3 and its proof. The proposition states that sample-dependent lifted critical points exist for n ≥ 1 + (d+1)m, but the proof constructs the outputs y'_i by choosing a nonzero v' ∈ ker M', which is guaranteed only when n ≥ 2 + (d+1)m; for n = 1 + (d+1)m, the matrix M' is square and its kernel may be trivial. The main theorem's own critical-point condition n > 1 + (d+1)m is, for integer n, exactly n ≥ 2 + (d+1)m, so the theorem is not affected. However, the appendix statement should be aligned to n ≥ 2 + (d+1)m, or the proof should be completed by the alternative construction y'_i = H(θ_narr, x_i), which puts the zero vector in ker M' and still yields a sample-dependent lifted critical point at n = 1 + (d+1)m.

minor comments (5)

[Lemma A.1.1] The statement says 'for 1 ≤ i < j ≤ m' but the index should be n; the domain is x_1,...,x_n, so the condition should read 'for 1 ≤ i < j ≤ n'.
[Appendix A.2.5(b)] The sentence 'Since n≥..., M′ and M′′ have more rows than columns, so there is some v′∈ker M′′\{0}' has the inequality backwards: a nontrivial kernel requires more columns than rows. This is a typo, but it should be corrected because the rank argument is inverted as printed.
[Section 5, Figure 1] The text defines the plotted vector field with the factor 1/a_1 multiplying ∂R/∂w_1, but the caption of Figure 1 writes the factor as 3/a_1. The factor should be made consistent.
[Theorem 4.2.2 and Proposition A.2.5] The thresholds written as 'n≥ 1+N / D' and 'n≥ 1+D+Σ_{l=2}^L m_l(m'_{l-1}-m_{l-1})+N / D' are ambiguous in the rendered text. They should be typeset as fractions (1+N)/D and (1+D+Σ + N)/D to avoid being read as 1 + N/D.
[Conclusion, Section 6] The sentence 'Recall that we have shown that all sample-dependent critical points must be of the form (2)' overstates the scope: the form-(2) statement is established in Remark 4.5 for one-hidden-layer networks under parity or related assumptions, not for the general multi-layer case. The sentence should be qualified accordingly.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the central existence and saddle theorems are proved from the paper's own definitions, linear-independence lemmas, and explicit sample constructions, not from the target results.

full rationale

The main results (Theorem 4.2.1, Propositions 4.2.1/4.2.2, and Theorem 4.2.2) are established by explicit construction. In the proof of Theorem 4.2.1 (Appendix A.2.3), the authors fix v in ker M, build an augmented matrix M', choose v' in ker M', and then use the hypothesis on the range of ∂_p ℓ to realize these vectors as loss gradients for two different sample-output choices. Sample dependence follows because θ_wide is critical for the samples (x_i, y'_i) but not for the samples (x_i, y_i), while both sample sets belong to S(θ_narr). This is a genuine construction, not a renaming or a fitted input called a prediction. Proposition 4.2.1 proves the zero-measure non-criticality and saddleness using Lemma A.1.2 and the analytic zero-measure Lemma A.1.3; Lemma A.1.1 (linear independence of neurons) is proved in the appendix rather than imported as an unverified assumption. Prior work by the same group (Zhang et al. 2021/2022; Bai et al. 2024) is cited for the Embedding Principle context and for definitions of critical embeddings, but the new existence theorems do not use those cited results as premises; the only external facts are standard analytic-function lemmas or are proven in the appendix. The reviewer-flagged threshold concern is not a circularity: the appendix proves saddleness under n ≥ 2+(d+1)m, a less restrictive condition than the theorem's n > 2+(d+1)m, so the theorem statement follows. The conclusion section explicitly lists open questions about completeness and about sample-dependent points in multi-layer networks, which further indicates that the results are not forced by definition. No step was found in which a prediction reduces by construction to an input or to a self-citation chain.

Assumptions & free parameters 0 free parameters · 6 assumptions · 0 invented entities

The central results rest on standard analyticity and generic-position assumptions about the activation, loss, and sample inputs, plus one ad hoc assumption (sigma(0)=0) in the counterexample. No parameters are fitted to data; the only free choices are mathematical definitions.

assumptions (6)

domain assumption Activation sigma is non-polynomial analytic (Assumption 3.1).
Used to ensure neurons are analytic and linearly independent under generic inputs (Lemma A.1.1). Excludes ReLU and other non-analytic activations.
domain assumption Loss ℓ is analytic, ℓ(p,q)=0 iff p=q, and ∂_p ℓ(p,q)=0 iff p=q (Assumption 3.2).
Ensures nonzero loss implies nonzero partial derivative, which makes the functions phi in the saddle proofs non-trivial.
ad hoc to paper sigma(0)=0 in the Section 4.1 counterexample (Proposition A.2.1).
Needed for zero weights at layers 1 and 2 to produce constant zero output and criticality for arbitrary samples. Not stated in the main-text example.
domain assumption Sample inputs satisfy xi != 0 and xi ± xj != 0 (stated in Propositions 4.2.1 and 4.2.2).
Generic-position condition used to establish linear independence of neurons, the backbone of the zero-measure and saddle arguments.
domain assumption Range of ∂_p ℓ(p,·) contains an open interval (one-hidden-layer) or neighborhood (multi-layer) around 0 (Theorems 4.2.1 and 4.2.2).
Allows construction of sample outputs y_i realizing a chosen vector of partial derivatives; satisfied by squared loss and cross-entropy.
standard math Zero set of a nonzero real analytic function has zero measure (Lemma A.1.3, Mityagin 2015).
Used repeatedly to show non-critical lifted parameters form a full-measure set.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Uncovering Critical Sets of Deep Neural Networks via Sample-Independent Critical Lifting." pith.science (2026). https://pith.science/paper/TF2I3X6Y

@misc{pith2026250513582,
  author       = {Pith},
  title        = {Pith review of: Uncovering Critical Sets of Deep Neural Networks via Sample-Independent Critical Lifting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TF2I3X6Y}},
  note         = {Machine review of arXiv:2505.13582}
}

read the original abstract

This paper investigates the sample dependence of critical points for neural networks. We introduce a sample-independent critical lifting operator that associates a parameter of one network with a set of parameters of another, thus defining sample-dependent and sample-independent lifted critical points. We then show by example that previously studied critical embeddings do not capture all sample-independent lifted critical points. Finally, we demonstrate the existence of sample-dependent lifted critical points for sufficiently large sample sizes and prove that saddles appear among them.

Figures

Figures reproduced from arXiv: 2505.13582 by the authors.

**Figure 1.** Plot of the vector field (a1, a2) 7→ ∂R ∂a1 (a1, w, a ¯ 2, w¯), 3 a1 ∂R ∂w1 (a1, w, a ¯ 2, w¯) for (a1, a2) ∈ (0.1, 0.9)2 with respect to (εi(−4))4 i=1 (left), (εi(0))4 i=1 (middle) and (εi(3))4 i=1. In all three figures, the vector field vanishes approximately along the line {a1 + a2 = 1}, indicating that the parameters produced by splitting embeddings are sample-independent saddles. Finally, we show the existe… view at source ↗

**Figure 2.** Contour plot of the loss function along the [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

**Figure 3.** The zero set of φ(t) = P4 i=1 εi(t)tanh(wxi) for (t, w) ∈ (−0.5, 0.5) × (−0.8, 0.8). The blue curve minus the origin, which arises when t ranges approximately from −0.05 to 0.3, is locally the graph of a non-constant function in t. This indicates that there is a sample-dependent lifted critical point for each such t. Also note that the grey curve {(0, t)} indicates a sample-independent lifted critical point (1, w, ¯… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 16 canonical work pages

[1]

Sun , author D

author R. Sun , author D. Li , author S. Liang , author T. Ding , title The global landscape of neural networks , journal Nonconvex Optimization for Signal Processing and Machine Learning volume 37 ( year 2020 ) pages 95--108

work page 2020
[2]

Zhang , author Y

author Y. Zhang , author Y. Li , author Z. Zhang , author T. Luo , author Z.-Q. J. Xu , title Embedding principle: a hierarchical structure of loss landscape of deep neural networks , journal Journal of Machine Learning volume 1 ( year 2022 ) pages 60--113

work page 2022
[3]

Zhang , author Z

author Y. Zhang , author Z. Zhang , author T. Luo , author Z.-Q. J. Xu , title Embedding principle of loss landscape of deep neural networks , journal NeurIPS volume 34 ( year 2021 ) pages 14848--14859

work page 2021
[4]

Bai , author T

author Z. Bai , author T. Luo , author Z.-Q. J. Xu , author Y. Zhang , title Embedding principle in depth for the loss landscape analysis of deep neural networks , journal CSIAM Transactions on Applied Mathematics volume 5 ( year 2024 ) pages 350--389

work page 2024
[5]

Cooper , title Global minima of overparameterized neural networks , journal SIAM Journal on Mathematics of Data Science volume 3 ( year 2021 ) pages 676--691

author Y. Cooper , title Global minima of overparameterized neural networks , journal SIAM Journal on Mathematics of Data Science volume 3 ( year 2021 ) pages 676--691

work page 2021
[6]

Geometry and Local Recovery of Global Minima of Two-layer Neural Networks at Overparameterization

author L. Zhang , author Y. Zhang , author T. Luo , title Structure and gradient dynamics near global minima of two-layer neural networks , journal arXiv:2309.00508 ( year 2023 )

work page Pith review arXiv 2023
[7]

Fukumizu , author S

author K. Fukumizu , author S. ichi Amari , title Local minima and plateaus in hierarchical structures of multilayer perceptrons , journal Neural Networks volume 13 ( year 2000 ) pages 317--327

work page 2000
[8]

Fukumizu , author S

author K. Fukumizu , author S. Yamaguchi , author Y. ichi Mototake , author M. Tanaka , title Semi-flat minima and saddle points by embedding neural networks to overparameterization , journal NeurIPS volume 32 ( year 2019 )

work page 2019

Show all 18 references

[9]

Simsek , author F

author B. Simsek , author F. Ged , author A. Jacot , author F. Spadaro , author C. Hongler , author W. Gerstner , author J. Brea , title Geometry of the loss landscape in overparametrized neural networks: Symmetry and invariances , journal Proceedings of Machine Learning Resea...

2021
[10]

Simsek , author A

author B. Simsek , author A. Bendjeddou , author W. Gerstner , author J. Brea , title Should under-parameterized student networks copy or average teacher weights? , journal NeurIPS ( year 2023 )

2023
[11]

author J. D. Lee , author I. Panageas , author G. Piliouras , author M. Simchowitz , author M. I. Jordan , author B. Recht , title First-order methods almost always avoid saddle points , journal arxiv:1710.07406 ( year 2017 )

2017 arXiv
[12]

Venturi , author A

author L. Venturi , author A. S. Bandeira , author J. Bruna , title Spurious valleys in one-hidden-layer neural network optimization landscapes , journal Journal of Machine Learning Research volume 20 ( year 2019 ) pages 1--34

2019
[13]

Li , author T

author D. Li , author T. Ding , author R. Sun , title On the benefit of width for neural networks: Disappearance of basins , journal SIAM Journal on Optimization volume 32 ( year 2022 ) pages 1728--1758

2022
[14]

Nguyen , author M

author Q. Nguyen , author M. Hein , title The loss surface of deep and wide neural networks , journal ICML volume 70 ( year 2017 ) pages 2603–2612

2017
[15]

Nguyen , title On connected sublevel sets in deep learning , journal ICML ( year 2019 ) pages 4790–4799

author Q. Nguyen , title On connected sublevel sets in deep learning , journal ICML ( year 2019 ) pages 4790–4799

2019
[16]

Kawaguchi , title Deep learning without poor local minima , journal NeurIPS ( year 2016 )

author K. Kawaguchi , title Deep learning without poor local minima , journal NeurIPS ( year 2016 )

2016
[17]

author S. G. Krantz , author H. R. Parks , title A Primer of Real Analytic Functions , Birkhäuser Advanced Texts Basler Lehrbücher, edition 2nd ed., publisher Birkhäuser Boston, MA , year 2002

2002
[18]

Mityagin , title The zero set of a real analytic function , journal arxiv:1512.07276 ( year 2015 )

author B. Mityagin , title The zero set of a real analytic function , journal arxiv:1512.07276 ( year 2015 )

2015 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Sun , author D

author R. Sun , author D. Li , author S. Liang , author T. Ding , title The global landscape of neural networks , journal Nonconvex Optimization for Signal Processing and Machine Learning volume 37 ( year 2020 ) pages 95--108

work page 2020

[2] [2]

Zhang , author Y

author Y. Zhang , author Y. Li , author Z. Zhang , author T. Luo , author Z.-Q. J. Xu , title Embedding principle: a hierarchical structure of loss landscape of deep neural networks , journal Journal of Machine Learning volume 1 ( year 2022 ) pages 60--113

work page 2022

[3] [3]

Zhang , author Z

author Y. Zhang , author Z. Zhang , author T. Luo , author Z.-Q. J. Xu , title Embedding principle of loss landscape of deep neural networks , journal NeurIPS volume 34 ( year 2021 ) pages 14848--14859

work page 2021

[4] [4]

Bai , author T

author Z. Bai , author T. Luo , author Z.-Q. J. Xu , author Y. Zhang , title Embedding principle in depth for the loss landscape analysis of deep neural networks , journal CSIAM Transactions on Applied Mathematics volume 5 ( year 2024 ) pages 350--389

work page 2024

[5] [5]

Cooper , title Global minima of overparameterized neural networks , journal SIAM Journal on Mathematics of Data Science volume 3 ( year 2021 ) pages 676--691

author Y. Cooper , title Global minima of overparameterized neural networks , journal SIAM Journal on Mathematics of Data Science volume 3 ( year 2021 ) pages 676--691

work page 2021

[6] [6]

Geometry and Local Recovery of Global Minima of Two-layer Neural Networks at Overparameterization

author L. Zhang , author Y. Zhang , author T. Luo , title Structure and gradient dynamics near global minima of two-layer neural networks , journal arXiv:2309.00508 ( year 2023 )

work page Pith review arXiv 2023

[7] [7]

Fukumizu , author S

author K. Fukumizu , author S. ichi Amari , title Local minima and plateaus in hierarchical structures of multilayer perceptrons , journal Neural Networks volume 13 ( year 2000 ) pages 317--327

work page 2000

[8] [8]

Fukumizu , author S

author K. Fukumizu , author S. Yamaguchi , author Y. ichi Mototake , author M. Tanaka , title Semi-flat minima and saddle points by embedding neural networks to overparameterization , journal NeurIPS volume 32 ( year 2019 )

work page 2019