New optimization algorithms for neural network training using operator splitting techniques

Cristian Daniel Alecsa; Imre Boros; Titus Pinta

RECORD

New optimization algorithms for neural network training using operator splitting techniques

Cristian Daniel Alecsa, Titus Pinta, Imre Boros

Machine Learning · Submitted 29 Apr 2019

abstract compiled · 2026-09-23 · no language-model review

The authors' main claim We validate the convergence of these optimizers using the results of the accuracy and of the loss function on the MNIST, MNIST-Fashion and CIFAR 10 classification datasets.

how it is stated · asserted 1.00 read the record →

Comment (0) Read on arXiv ↗

Cite ▾

arXiv ID1904.12952 (version 5)

Pith ID6XSE43DZ, Pith's permanent record of this paper and its review

CitationCristian Daniel Alecsa, Imre Boros, Titus Pinta. "New optimization algorithms for neural network training using operator splitting techniques." arXiv:1904.12952 (2019). https://pith.science/paper/6XSE43DZ

Claim paper

The compiled record

The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.

Open the compiled record

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-23: 1 request, 3,560 input tokens.

What the abstract itself states, as probabilities: claims first 0.08 · states limitation 0.20 · human participants 0.02 · controlled comparison 0.07 · randomized assignment 0.01 · states quantitative result 0.08 · compares with prior results 0.10 · mentions code or data availability 0.02.

The abstract, sentence by sentence

arXiv listing abstract v5. Results and the main claim are underlined; show every role.

In the following paper we present a new type of optimization algorithms adapted for neural network training. These algorithms are based upon sequential operator splitting technique for some associated dynamical systems. Furthermore, we investigate through numerical simulations the empirical rate of convergence of these iterative schemes toward a local minimum of the loss function, with some suitable choices of the underlying hyper-parameters. We validate the convergence of these optimizers using the results of the accuracy and of the loss function on the MNIST, MNIST-Fashion and CIFAR 10 classification datasets.

background contribution method result interpretation main claim

What the classifier reports for each sentence. The number is the probability it gave the label.
#	sentence	role	the authors' own work	states the main conclusion	strength
s0	In the following paper we present a new type of optimization algorithms adapted for neural network training.	contribution 1.00	0.84	0.08	asserted 1.00
s1	These algorithms are based upon sequential operator splitting technique for some associated dynamical systems.	method 0.93	0.49	0.05	asserted 1.00
s2	Furthermore, we investigate through numerical simulations the empirical rate of convergence of these iterative schemes toward a local minimum of the loss function, with some suitable choices of the underlying hyper-parameters.	method 0.59	0.95	0.12	asserted 0.90
s3	We validate the convergence of these optimizers using the results of the accuracy and of the loss function on the MNIST, MNIST-Fashion and CIFAR 10 classification datasets.	method 0.57	0.95	0.27	asserted 1.00

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from simulation or computation 0.99.

What was examined, and what was not

sources read

arXiv listing abstract v5

not examined

full text
figures and tables
references and citing sentences
relations to other papers

levels run

P abstract

not run

S sections
T body sentences
C citations
R cross-paper relations

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. Every number in the measurements table is the authors', checked by code. Record as JSON

The paper

Original abstract

In the following paper we present a new type of optimization algorithms adapted for neural network training. These algorithms are based upon sequential operator splitting technique for some associated dynamical systems. Furthermore, we investigate through numerical simulations the empirical rate of convergence of these iterative schemes toward a local minimum of the loss function, with some suitable choices of the underlying hyper-parameters. We validate the convergence of these optimizers using the results of the accuracy and of the loss function on the MNIST, MNIST-Fashion and CIFAR 10 classification datasets.

Human review

How Pith checks papers → · Full audit of this paper →

How the literature uses it

1 arXiv paper cite this in 2 sentences

Each citing sentence is taken from the citing paper's LaTeX source and typed by Pith's classifier. A label appears at probability 0.8 or more (0.95 for support), a contest at 0.7 or more; the quoted words are the citing authors'. How each label was checked · JSON

Papers citing it · 1 sentence without a confident label

Compares with it 1 paper

The analysis is in contrast with the present article by the fact that in [ref] , the optimization algorithms were developed in an heuristic manner based upon algebraic manipulations.

The long time behavior and the rate of convergence of symplectic convex algorithms obtained via splitting discretizations of inertial damping systems · Some notes on splitting operator techniques · 0.92

What it cites · 12 sentences without a confident label

Uses the method or tool of 2 papers

Last but not least, we reassert that instead of using adaptive learning rate algorithms, one can use [ref] using learning rate updates that depend on the current epoch training(for more details, follow [ref] ).

Cyclical Learning Rates for Training Neural Networks · A gentle introduction to machine learning algorithms · 0.85
In our computations, we have used the model proposed in [ref] by Szegedy et. al.

Going Deeper with Convolutions · Description of the neural network · 1.00

Compares with 3 papers

Elseways, the comparison of our splitting-based algorithms were done with some classical optimizers, like in the paper [ref] , [ref] , [ref] , [ref] and [ref] .

Adam: A Method for Stochastic Optimization · Discussions · 0.99
Elseways, the comparison of our splitting-based algorithms were done with some classical optimizers, like in the paper [ref] , [ref] , [ref] , [ref] and [ref] .

ADADELTA: An Adaptive Learning Rate Method · Discussions · 0.99
Elseways, the comparison of our splitting-based algorithms were done with some classical optimizers, like in the paper [ref] , [ref] , [ref] , [ref] and [ref] .

The Marginal Value of Adaptive Gradient Methods in Machine Learning · Discussions · 0.99

Cites as background 7 papers

For some references regarding the choice of the optimal learning rate, the optimizers for neural networks and for an empirical analysis of the training and classification problems, we refer to [ref] , [ref] , [ref] and [ref] .

Understanding deep learning requires rethinking generalization · A gentle introduction to machine learning algorithms · 0.97
For some references regarding the choice of the optimal learning rate, the optimizers for neural networks and for an empirical analysis of the training and classification problems, we refer to [ref] , [ref] , [ref] and [ref] .

Practical recommendations for gradient-based training of deep architectures · A gentle introduction to machine learning algorithms · 0.97
Last but not least, we recall the optimization algorithm Adam (see [ref] on Page 2), that can be considered as a modified gradient descent using two biased corrected moments, i.e.

Adam: A Method for Stochastic Optimization · A gentle introduction to machine learning algorithms · 0.86

Discussion

Know something about this paper? Leave a comment, or highlight any passage above to comment on it.