Pith. sign in

REVIEW 3 cited by

GradMax: Growing Neural Networks using Gradient Information

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2201.05125 v3 pith:YUYHDA4M submitted 2022-01-13 cs.LG cs.CV

Utku Evci , Bart van Merriënboer , Thomas Unterthiner , Max Vladymyrov , Fabian Pedregosa This is my paper
classification cs.LGcs.CV
keywords architecturecostlygradientgradmaxgrowingmaximizingnetworksneural
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

The architecture and the parameters of neural networks are often optimized independently, which requires costly retraining of the parameters whenever the architecture is modified. In this work we instead focus on growing the architecture without requiring costly retraining. We present a method that adds new neurons during training without impacting what is already learned, while improving the training dynamics. We achieve the latter by maximizing the gradients of the new weights and find the optimal initialization efficiently by means of the singular value decomposition (SVD). We call this technique Gradient Maximizing Growth (GradMax) and demonstrate its effectiveness in variety of vision tasks and architectures.

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. An optimal control approach for neural network architecture adaptation with a posteriori error estimation

    cs.LG 2026-07 conditional novelty 7.0 of 10

    The paper derives a posteriori error estimates for neural network depth adaptation by formulating training as an optimal control problem and using dual weighted residuals to insert layers where error is highest.

  2. SPARKLING: Balancing Signal Preservation and Symmetry Breaking for Width-Progressive Learning

    cs.LG 2026-02 conditional novelty 6.0 of 10

    A width-progressive training method (RMS-preserving rescaling plus asymmetric optimizer-state reset and LR rewarmup) enables mid-training 2x width expansion with up to 35% compute savings over training from scratch.

  3. Evolutionary Developmental Biology Can Serve as the Conceptual Foundation for a New Design Paradigm in Artificial Intelligence

    cs.AI 2025-06 conditional novelty 6.0 of 10

    The paper proposes regulatory connections, weak linkage, and component-level variation-selection, drawn from evo-devo, as the unifying conceptual foundation for a new AI design paradigm.

Pith tools