Training ResNets with a stage cost on intermediate outputs obtained through skip connections biases deep residual layers toward identity mappings and enables pruning with small accuracy loss on homogeneous models.
On Dissipativity of Cross-Entropy Loss in Training ResNets
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
The training of ResNets and neural ODEs can be formulated and analyzed from the perspective of optimal control. This paper proposes a dissipative formulation of the training of ResNets and neural ODEs for classification problems by including a variant of the cross-entropy as a regularization in the stage cost. Based on the dissipative formulation of the training, we prove that the trained ResNet exhibit the turnpike phenomenon. We then illustrate that the training exhibits the turnpike phenomenon by training on the two spirals and MNIST datasets. This can be used to find very shallow networks suitable for a given classification task.
citation-role summary
background 1
citation-polarity summary
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
Towards an Optimal Control Perspective of ResNet Training
Training ResNets with a stage cost on intermediate outputs obtained through skip connections biases deep residual layers toward identity mappings and enables pruning with small accuracy loss on homogeneous models.