Pith. sign in

REVIEW 2 cited by

Semi-flat minima and saddle points by embedding neural networks to overparameterization

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1906.04868 v2 pith:4LLDJZTI submitted 2019-06-12 cs.LG stat.ML

Kenji Fukumizu , Shoichiro Yamaguchi , Yoh-ichi Mototake , Mirai Tanaka This is my paper · ORCID
classification cs.LGstat.ML
keywords networkspointembeddingminimumnetworkneuraloverparameterizedresults
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We theoretically study the landscape of the training error for neural networks in overparameterized cases. We consider three basic methods for embedding a network into a wider one with more hidden units, and discuss whether a minimum point of the narrower network gives a minimum or saddle point of the wider one. Our results show that the networks with smooth and ReLU activation have different partially flat landscapes around the embedded point. We also relate these results to a difference of their generalization abilities in overparameterized realization.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Neural Network-based High-index Saddle Dynamics Method for Searching Saddle Points and Solution Landscape

    cs.LG 2024-11 conditional novelty 4.0 of 10

    A neural network surrogate replaces the explicit energy function in high-index saddle dynamics, and nearby surrogate saddle points are proven to approximate true ones when the network is accurate.

  2. Explaining Machine Learning and Memorization with Statistical Mechanics

    cs.LG 2026-06 unverdicted novelty 3.0 of 10

    Thesis uses statistical mechanics to study DAM and RBM models for understanding memorization, low-dimensional learning, and adversarial robustness in neural networks.

Pith tools