Training neural networks without backpropagation using particles

Deepak Kumar

REVIEW 4 major objections 5 minor 43 references

Training neural networks without backpropagation using particles

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper reports that a multi-layer perceptron can be trained without backpropagation by giving each neuron its own particle swarm, reaching accuracies on Rice and Dry Bean datasets comparable to a backpropagation-trained MLP.

desk verdict The per-neuron PSO idea is clear and implementable, but the validation-loss gate in Algorithm 1 may be selecting on the same held-out fold whose accuracy is reported, which means the central 'similar to MLP' claim is not yet established. read the letter →

arxiv 2412.05667 v3 pith:PBKZCNZ4 submitted 2024-12-07 cs.NE cs.AI

Deepak Kumar This is my paper · ORCID

classification cs.NEcs.AI

keywords particleswarmoptimizationbackpropagation-freetrainingper-neuronforward-onlylearningmulti-layerperceptronderivative-freevalidationgatingcrazinessterm

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a multi-layer perceptron can be trained without backpropagation by treating each neuron as an independent search problem. For every neuron, a small swarm of particles explores candidate weight vectors while all other weights are held fixed, and the best particle's weights are selected; a validation-loss gate decides whether the combined new weights are kept. On synthetic classification tasks and on the Rice and Dry Bean datasets, the author reports loss curves and accuracies comparable to a backpropagation-trained MLP. If the claim holds, it offers a derivative-free, forward-only training alternative for small neural networks, with per-neuron updates that can be parallelized.

What carries the argument

The central object is the per-neuron particle swarm. Each neuron node hosts k PSO particles, each particle being a candidate weight vector for that node; the velocity update includes the standard PSO terms plus a craziness term $c_3 r_3$ to keep particles exploring. While one node is updated, all other network weights are fixed, so the node's loss subproblem depends only on its own particle positions. The per-node best particles are combined into the network weights, and a validation-loss gate (Equation 5) decides whether the new combined weights are kept, which restricts runaway behavior. This machinery replaces the backward pass with k forward loss evaluations per node, repeated batchwise across epochs.

What would settle it

Train the method on a synthetic classification task whose solution requires two hidden neurons to change their weights simultaneously, so that no single-neuron update reduces the loss; if per-neuron PSO with validation gating cannot match a backpropagation-trained baseline on that task, the central claim fails.

Watch

Extended reading notes

Core claim

The central claim is that individual neuron nodes can be trained independently with particle swarm optimization and that the collective behavior of these separately trained neurons solves classification problems as effectively as backpropagation. The author splits the network into sub-problems: for each node, k particles carry candidate weight vectors; the loss is computed for each particle while all other node weights are frozen, and the minimum-loss particle is chosen. A validation-loss comparison after each batch decides whether the new combined weights replace the previous global best, which the author says contains the runaway problem of unconstrained per-node updates. The reported result is that the proposed method follows the same loss-reduction path as gradient descent and reaches accuracies within the spread of the MLP baseline on the Rice and Dry Bean datasets, while avoiding backpropagation entirely.

Load-bearing premise

The load-bearing assumption is that updating one neuron at a time while freezing every other neuron, and accepting the new weights only when validation loss improves, is enough to navigate the network's joint loss landscape to a good solution.

Editorial extensions

If this is right

Training requires no backward pass, so errors do not need to be transported backward through the network; weight updates come from forward loss evaluations alone.
Each neuron's weight update is independent, so node updates across the network can be computed in parallel without waiting for other neurons.
The method is insensitive to the choice of activation function in the reported tests: replacing ReLU with Tanh left the loss-reduction behavior unchanged.
Because updates are separated per node, the approach opens the door to arbitrary, non-standard connectivity between neurons without needing a global backpropagation rule.
On the Rice and Dry Bean datasets, the reported accuracy, specificity, and F1-scores land within one standard deviation of the MLP baseline in Tables 1 and 2.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A comparison the paper does not report is against plain coordinate-wise random search without the PSO velocity memory; without it, the contribution of the swarm dynamics versus the per-neuron decomposition alone is untested.
The validation-loss gate repeatedly selects network states on the same validation data, which can bias the reported metrics; a clean test would hold out a third split for final evaluation.
The method's cost is k forward passes per neuron per batch, which the paper itself flags as redundant computation; scaling to larger networks would likely require sharing or approximating per-neuron loss evaluations.
If the approach scales, the per-neuron scalar reward resembles local learning rules in neuroscience, where each neuron aligns to a global performance score instead of receiving a backpropagated gradient.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Worth a look, but the headline claim is not yet established. The novel bit is real: instead of running PSO over the whole weight vector, you give each neuron its own sub-swarm, fix everything else, update that neuron, and gate the whole-network update on validation loss. That is block coordinate descent with PSO as the inner solver, and the paper is candid about the runaway instability and the redundant-computation cost. That honesty counts. The synthetic curves do what they claim: PSO without the craziness term stalls, and the proposed method reaches loss comparable to an SGD-trained MLP. On Rice and Dry Bean, the accuracy numbers sit within about a point of the MLP baseline.

The soft spots are serious. First, the validation gate in Algorithm 1 can undermine the accuracy tables. Step 14 keeps a network update only if the new validation loss is lower, and the method needs 'independent validation data.' The experimental section only mentions k-fold cross-validation; it never says a separate test split was reserved inside each fold for final evaluation. If the holdout fold used for the gating is the same one whose accuracy is reported in Tables 1 and 2, those numbers are optimistically biased, and the 'similar to MLP' claim is unproven. That has to be fixed before the empirical result can be trusted. Second, there is no convergence or guarantee argument, and the per-neuron decomposition is motivated only by intuition. The abstract's language about 'overcoming' gradient descent and PSO limitations is too strong. Third, error bars exist only for the two PSO variants, not for the MLP or classical baselines, and hyperparameters are under-reported. The code is linked, but no commit hash or version is given, so I cannot pin down what exactly produced the tables.

Proportionately, this is a legitimate, simple idea that a competent referee can evaluate quickly. It is not a major breakthrough; the contribution is a modest extension of coordinate descent and cooperative PSO. But it is implementable, and the stability issue is acknowledged in the text. I would send it to peer review, with the referee instructed to require a clean train/validation/test split and to rerun the tables. The paper is not ready as is; a revised version with a proper split and full hyperparameters would be credible. The audience is people working on derivative-free or forward-only training methods, not those looking for large-scale results.

Referee Report

4 major / 5 minor

Summary. The paper proposes a derivative-free training method for multi-layer perceptrons in which each neuron is equipped with its own set of PSO particles and is updated coordinate-wise while all other weights are held fixed. A validation-loss gate in Algorithm 1 accepts or rejects whole-network updates, and the method is evaluated on four synthetic problems and two real tabular datasets (Rice and Dry Bean). The authors claim that this per-neuron PSO approach overcomes both local-minima problems of gradient descent and the search-space limitation of whole-network PSO, achieving accuracies and loss curves comparable to backpropagation-trained MLPs.

Significance. If the empirical claims hold, the paper offers a simple, forward-only, derivative-free training scheme with per-neuron parallelism that is easy to implement and that the authors have released as open-source code. Those are genuine strengths. However, the evidence is limited to two small tabular datasets and synthetic examples, and the central comparability claim is weakened by the evaluation-protocol ambiguity discussed below, by missing baseline error bars, and by an unproven premise about decomposing the joint loss into per-neuron subproblems. The contribution is incremental relative to the large literature on evolutionary and derivative-free neural-network training, but the per-neuron decomposition is a distinct and potentially useful variation.

major comments (4)

[Algorithm 1, Steps 6 and 14; Sections 6.5 and 6.6] The final network is selected based on validation loss (Steps 6 and 14), but the paper never states that a separate test split, not used for this selection, is reserved inside each cross-validation fold. Sections 6.5 and 6.6 only say '4-fold cross-validation' and '13-fold cross-validation.' If the validation set used in Step 14 is the same held-out fold whose accuracy is reported in Tables 1 and 2, the reported accuracies are optimistically biased because the network has effectively been selected on the test set. Please clarify the exact data split (e.g., train/validation/test within each fold) and, if the validation gate used the test fold, re-run the experiments with nested splits before the comparability claim can be assessed.
[Sections 4.1 and 5; Eq. (5)] The method's core premise is that fixing all other weights turns each neuron into an independent subproblem, but the joint loss is not separable into per-neuron losses. Updating each neuron by its own best particle does not necessarily decrease the joint loss; the paper itself notes in Section 4 that 'the nodes together perform runaway operations' and relies on the validation-loss gate in Algorithm 1 for stability. No convergence or quality guarantee is provided for this coordinate-wise PSO procedure. Please either provide a formal statement of what the per-neuron updates optimize (e.g., a descent or monotone-improvement condition on the joint loss) or substantially temper the Abstract and Introduction claims that the method 'overcome[s] the problem of gradient descent and the limitation of the PSO algorithm.'
[Tables 1 and 2] The comparability claim rests on the last three rows of Tables 1 and 2, but standard deviations are reported only for the proposed PSO rows, not for the MLP or other baselines. Overlapping or comparable error bars are necessary to support 'performance is similar to MLP.' Please report means and standard deviations (or confidence intervals) for all methods and specify the MLP architecture, optimizer, learning rate, epochs, and any regularization used for the baselines.
[Section 6 (all subsections)] The free PSO parameters—inertia weight w, acceleration constants c1 and c2, craziness constant c3, number of particles per neuron, batch size, weight initialization range, and stopping criterion—are not reported anywhere in Section 6 or in Algorithm 1. Without these values, the experiments are not reproducible from the text alone; the GitHub link is helpful but does not replace a hyperparameter table. Please add a table listing the values used for each dataset and synthetic experiment.

minor comments (5)

[Figure 3] The flowchart text is garbled and largely unreadable in the PDF; please replace it with a clear vector figure with legible font.
[Section 6.2] The MLP and the proposed method use different output-layer configurations (one output node with MSE loss vs. two output nodes with BCE loss), so the loss curves in Figures 4–7 are not directly comparable. Please explain how the comparison is intended despite the differing losses.
[Sections 6.1–6.4 and 7] The phrase 'follows the gradient descent' is misleading because the proposed method is derivative-free; it appears to mean only that the loss curve decreases similarly. Please rephrase to avoid implying gradient-based behavior.
[Algorithm 1, Step 7 and Eq. (5)] Step 7 says loss values are set to zero after each epoch, but Eq. (5) accumulates batch losses into f_j_ik. Please clarify whether the accumulated loss is reset per epoch and how the personal-best comparison in Steps 9–11 interacts with this reset.
[Section 3.1] The statement that 'the original PSO paper had removed the craziness term' is a historical claim without a citation; please either cite the source or rephrase to describe only the version used here.

Circularity Check

0 steps flagged · score 2.0 of 10

Central training claim is empirically self-contained; only minor self-cited craziness-term heuristic appears, and it is not load-bearing.

full rationale

The paper's central claim is that per-neuron PSO weight updates can train MLPs to accuracies comparable to backpropagation on Rice and Dry Bean datasets. This is an external empirical benchmark, not a derivation from a fitted quantity. No equation in the paper defines the target result as an input: Eq. (4) is a per-node min-selection over particles, Eq. (5) is a batch-wise loss accumulation rule, and Algorithm 1 is a coordinate-wise search controlled by a validation-loss gate. The only author self-citation of note is the craziness term from [24,25] used in Eq. (3); it is a heuristic component of the search, not a theorem from which the central claim is deduced, so it does not make the training result circular. The validation-loss gate in Algorithm 1 could bias results if the validation data coincided with the scored test fold, but the manuscript does not state that they coincide, and k-fold evaluation is described; this is an experimental-design risk, not an exhibited circular reduction. The paper is self-contained against external datasets, so no significant circularity is found.

Assumptions & free parameters 7 free parameters · 3 assumptions · 0 invented entities

The method introduces no invented physical or conceptual entities and derives no new constants. Its load-bearing numerical inputs are PSO hyperparameters and training schedule choices, most of which are unreported. The key auxiliary assumption is that per-neuron coordinate-wise optimization can solve the joint non-convex problem, which is stated rather than proven.

free parameters (7)

PSO inertia weight w = not reported
Appears in the velocity update Eq. 1; the value is chosen by hand and never stated.
PSO acceleration constants c1, c2 = not reported
Appear in Eq. 1 and control the balance between personal and global best; values are omitted.
Craziness constant c3 = not reported
Appears in Eq. 3 and is inherited from the authors' prior work; no value is reported.
Number of particles k per neuron = not reported
Algorithm 1 and Figure 2 depend on k, the number of PSO particles per node; no value or sweep is given.
Number of epochs = 20 for synthetic datasets, 100 for real datasets
Section 6 sets epoch counts, so the reported convergence depends on this schedule choice.
Batch size = not reported
Batch-wise training is central to Algorithm 1, but the batch size is never specified.
Weight initialization range = not reported
Algorithm 1 says random values are initialized for each particle, but the distribution or range is not given.

assumptions (3)

domain assumption Optimizing each neuron independently, with other weights fixed, leads to a good solution of the joint loss.
Section 4.1 defines per-neuron loss minimization without a proof that coordinate-wise updates decrease the global loss; Section 5 adds a validation-loss gate to compensate.
domain assumption PSO converges to an acceptable local optimum for each neuron within the chosen number of epochs.
The paper uses PSO as a black-box optimizer (Eqs. 1 and 3) and provides no convergence or approximation guarantee for the per-neuron subproblems.
domain assumption Repeated use of validation loss for accepting or rejecting weight updates does not materially bias the reported test performance.
Section 5 uses validation loss as the gate for global best weights, so the validation set is part of model selection rather than a pure test set.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Training neural networks without backpropagation using particles." pith.science (2026). https://pith.science/paper/PBKZCNZ4

@misc{pith2026241205667,
  author       = {Pith},
  title        = {Pith review of: Training neural networks without backpropagation using particles},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PBKZCNZ4}},
  note         = {Machine review of arXiv:2412.05667}
}

read the original abstract

Neural networks are a group of neurons stacked together in multiple layers to mimic the biological neurons in a human brain. Neural networks have been trained using the backpropagation algorithm based on gradient descent strategy for several decades. Several variants have been developed to improve the backpropagation algorithm. The loss function for the neural network is optimized through backpropagation, but several local minima exist in the manifold of the constructed neural network. We obtain several solutions matching the minima. The gradient descent strategy cannot avoid the problem of local minima and gets stuck in the minima due to the initialization. Particle swarm optimization (PSO) was proposed to select the best local minima among the search space of the loss function. The search space is limited to the instantiated particles in the PSO algorithm, and sometimes it cannot select the best solution. In the proposed approach, we overcome the problem of gradient descent and the limitation of the PSO algorithm by training individual neurons separately, capable of collectively solving the problem as a group of neurons forming a network. Our code and data are available at https://github.com/dipkmr/train-nn-wobp/

Figures

Figures reproduced from arXiv: 2412.05667 by the authors.

**Figure 2.** A three layer neural network with each node showing [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** The flowchart of the proposed method including a bat [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: The synthetic data with linearly separable classe [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

**Figure 5.** Figure 5: The synthetic data with nonlinearly separable cla [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

**Figure 6.** Figure 6: The synthetic data with increased nonlinearity. T [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

**Figure 7.** Figure 7: The synthetic data with increased number of classe [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]

**Figure 8.** Figure 8: The normalized loss function after each epoch is sh [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 27 canonical work pages

[1]

Gradient Followi ng Without Back-Propagation in Layered Networks

Andrew G. Barto and Michael I. Jordan. “Gradient Followi ng Without Back-Propagation in Layered Networks”. In: International Conference on Neural Networks . 1987, pp. 629–636

work page 1987
[2]

Gradients without Backprop agation

Atılım Güne¸ s Baydin et al. “Gradients without Backprop agation”. In: (2022). arXiv: 2202.08587 [cs.LG]

arXiv 2022
[3]

C. M. Bishop. Neural networks for pattern recognition . Oxford: Oxford university press, 1995

work page 1995
[4]

C. M. Bishop. Pattern recognition and machine learning. Berlin: Springer, 2006

work page 2006
[5]

How to Train Your Wide Neural Network Without Backprop: An Input-Weight Alignment Perspective

Akhilan Boopathy and Ila Fiete. “How to Train Y our Wide Ne ural Network Without Back- prop: An Input-Weight Alignment Perspective”. In: (2021). arXiv: 2106.08453 [cs.LG]

work page Pith review arXiv 2021
[6]

Supervised Learning in Ne ural Networks without Feed- back Networks

Robert D. Brandt and Feng Lin. “Supervised Learning in Ne ural Networks without Feed- back Networks”. In: IEEE International Symposium on Intelligent Control . 1996. DOI : 10.1109/ISIC.1996.556182

work page arXiv 1996
[7]

Classiﬁcation of Rice V ari eties Using Artiﬁcial Intelligence Methods

Ilkay Cinar and Murat Koklu. “Classiﬁcation of Rice V ari eties Using Artiﬁcial Intelligence Methods”. In: International Journal of Intelligent Systems and Applicat ions in Engineering 7.3 (2019), pp. 188–194. DOI : 10.18201/ijisae.2019355381

work page doi:10.18201/ijisae.2019355381 2019
[8]

Adaptive Subg radient Methods for Online Learning and Stochastic Optimization

John Duchi, Elad Hazan, and Y oram Singer. “Adaptive Subg radient Methods for Online Learning and Stochastic Optimization”. In: Journal of Machine Learning Research 12 (2011), pp. 2121–2159

work page 2011

Show all 43 references

[9]

R. O. Duda, P . E. Hart, and D. G. Stork. Pattern classiﬁcation . Wiley, 2001

2001
[10]

Neuroevolution in Deep Neural Networks: Current Trends and Future Challenges

Edgar Galván and Peter Mooney. “Neuroevolution in Deep Neural Networks: Current Trends and Future Challenges”. In: IEEE Transactions on Artiﬁcial Intelligence 2.6 (2021), pp. 476–

2021
[11]

Exponential natural evolut ion strategies

Tobias Glasmachers et al. “Exponential natural evolut ion strategies”. In: Proceedings of the 12th Annual Conference on Genetic and Evolutionary Comp utation. GECCO ’10. Association for Computing Machinery, 2010, pp. 393–400. ISBN : 9781450300728. DOI : 10.1145/1830483.1830557...

2010
[12]

An introduction to neural networks

Kevin Gurney. An introduction to neural networks . London: UCL Press, 1997

1997
[13]

Never loo k back - A modiﬁed EnKF method and its application to the training of neural networks witho ut back propagation

Eldad Haber, Felix Lucka, and Lars Ruthotto. “Never loo k back - A modiﬁed EnKF method and its application to the training of neural networks witho ut back propagation”. In: (2018). arXiv: 1805.08034 [math.NA]

2018 arXiv
[14]

Non-Linear Back-propagation: Doing Back-Propagation without Deriva- tives of the Activation Function

John Hertz et al. “Non-Linear Back-propagation: Doing Back-Propagation without Deriva- tives of the Activation Function”. In: IEEE Transactions on Neural Networks 8.6 (1997), pp. 1321–1327

1997
[15]

The Forward-Forward Algorithm: Som e Preliminary Investigations

Geoffrey Hinton. “The Forward-Forward Algorithm: Som e Preliminary Investigations”. In: (2022). arXiv: 2212.13345 [cs.LG]. 13

2022 arXiv
[16]

Decoupled Neural Interfaces usin g Synthetic Gradients

Max Jaderberg et al. “Decoupled Neural Interfaces usin g Synthetic Gradients”. In: (2016). arXiv: 1608.05343 [cs.LG]

2016 arXiv
[17]

One forward is enough for neural ne twork training via likelihood ratio method

Jinyang Jiang et al. “One forward is enough for neural ne twork training via likelihood ratio method”. In: (2023). arXiv: 2305.08960 [cs.LG]

2023 arXiv
[18]

Particle swarm optimization

James Kennedy and Russell C. Eberhart. “Particle swarm optimization”. In: Proceedings of the IEEE International Conference on Neural Networks . 1995, pp. 1942–1948

1995
[19]

Eberhart, and Shi Y

James Kennedy, Russell C. Eberhart, and Shi Y. Swarm intelligence. San Francisco: Morgan Kaufmann Publishers, 2001

2001
[20]

Adam: a Method for S tochastic Optimization

Diederik P . Kingma and Jimmy Lei Ba. “Adam: a Method for S tochastic Optimization”. In: International Conference on Learning Representations . 2015, pp. 1–13

2015
[21]

Multiclass Classiﬁca tion of Dry Beans Using Computer Vision and Machine Learning Techniques

Murat Koklu and Ilker Ali Ozkan. “Multiclass Classiﬁca tion of Dry Beans Using Computer Vision and Machine Learning Techniques”. In: Computers and Electronics in Agriculture 174.105507 (2020). DOI : 10.1016/j.compag.2020.105507

2020
[22]

Back-Propagation Without Weight Trans- port

John F. Kolen and Jordan B. Pollack. “Back-Propagation Without Weight Trans- port”. In: IEEE International Conference on Neural Networks (ICNN’94 ). 1994. DOI : 10.1109/ICNN.1994.374486

1994
[23]

Ensemble Kalman inversion: a derivative-free technique for machine learning tasks

Nikola B Kovachki and Andrew M Stuart. “Ensemble Kalman inversion: a derivative-free technique for machine learning tasks”. In: Inverse Problems 35.095005 (2019)

2019
[24]

Binary classiﬁca tion posed as a quadratically con- strained quadratic programming and solved using particle s warm optimization

Deepak Kumar and A. G. Ramakrishnan. “Binary classiﬁca tion posed as a quadratically con- strained quadratic programming and solved using particle s warm optimization”. In: S¯ adhan¯ a 41.3 (2016), pp. 289–298. DOI : 10.1007/s12046-016-0466-y

2016 doi
[25]

Quadratically co nstrained quadratic pro- gramming for classiﬁcation using particle swarms and appli cations

Deepak Kumar and A. G. Ramakrishnan. “Quadratically co nstrained quadratic pro- gramming for classiﬁcation using particle swarms and appli cations”. In: (2014). arXiv: 1407.6315 [cs.AI]

2014 arXiv
[26]

Backpropagation and the br ain

Timothy P . Lillicrap et al. “Backpropagation and the br ain”. In: Nature Reviews Neuroscience 21 (2020), pp. 335–346. DOI : 10.1038/s41583-020-0277-3

2020 doi
[27]

Random feedback weights su pport learning in deep neural net- works

Timothy P . Lillicrap et al. “Random feedback weights su pport learning in deep neural net- works”. In: (2014). arXiv: 1411.0247 [q-bio.NC]

2014 arXiv
[28]

Random synaptic feedback w eights support error back- propagation for deep learning

Timothy P . Lillicrap et al. “Random synaptic feedback w eights support error back- propagation for deep learning”. In: Nature Communications 7.13276 (2016). DOI : 10.1038/ncomms13276

2016 doi
[29]

Th e HSIC Bottleneck: Deep Learning without Back-Propagation

Wan-Duo Kurt Ma, J.P . Lewis, and W . Bastiaan Kleijn. “Th e HSIC Bottleneck: Deep Learning without Back-Propagation”. In: AAAI Conference on Artiﬁcial Intelligence . 2020, pp. 5085–5092

2020
[30]

Simple Evolutio nary Optimization Can Rival Stochastic Gradient Descent in Neural Networks

Gregory Morse and Kenneth O. Stanley. “Simple Evolutio nary Optimization Can Rival Stochastic Gradient Descent in Neural Networks”. In: Proceedings of the Genetic and Evolu- tionary Computation Conference 2016 . GECCO ’16. Association for Computing Machin- ery, 2016, pp. 477–4...

2016
[31]

Random Gradient-Free Min imization of Convex Func- tions

Y . Nesterov and V . Spokoiny. “Random Gradient-Free Min imization of Convex Func- tions”. In: F oundations of Computational Mathematics 17 (2017), pp. 527–566. DOI : 10.1007/s10208-015-9296-2

2017 doi
[32]

Derivativ e-free optimization: a review of algo- rithms and comparison of software implementations

Luis Miguel Rios and Nikolaos V . Sahinidis. “Derivativ e-free optimization: a review of algo- rithms and comparison of software implementations”. In: Journal of Global Optimization 56 (2013), pp. 1247–1293. DOI : 10.1007/s10898-012-9951-y

2013 doi
[33]

Learning representations by back-propagating errors

David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Wi lliams. “Learning representations by back-propagating errors”. In: Nature 323 (1986), pp. 533–536

1986
[34]

Evolution Strategies as a Scalable Alternative to Reinforcement Learning

Tim Salimans et al. Evolution Strategies as a Scalable Alternative to Reinforcement Learning
[35]

Schneider et al

Andreas C. Schneider et al. What should a neuron aim for? Designing local objec- tive functions based on information theory . 2024. arXiv: 2412.02482 [cs.IT]. URL : https://arxiv.org/abs/2412.02482

2024
[36]

Deep Neuroevolution: Gene tic Algorithms Are a Competitive Al- ternative for Training Deep Neural Networks for Reinforcem ent Learning

Felipe Petroski Such et al. “Deep Neuroevolution: Gene tic Algorithms Are a Competitive Al- ternative for Training Deep Neural Networks for Reinforcem ent Learning”. In: (2017). arXiv: 1712.06567 [cs.NE]. URL : http://arxiv.org/abs/1712.06567. 14

2017 arXiv
[37]

Efﬁcient natural evolution strategies

Yi Sun et al. “Efﬁcient natural evolution strategies”. In: Proceedings of the 11th Annual Con- ference on Genetic and Evolutionary Computation . GECCO ’09. Association for Computing Machinery, 2009, 539–546. ISBN : 9781605583259. DOI : 10.1145/1569901.1569976. URL : https://do...

2009
[38]

An Evolut ionary Algorithm of Lin- ear complexity: Application to Training of Deep Neural Netw orks

S. Ivvan V aldez and Alfonso Rojas Domínguez. “An Evolut ionary Algorithm of Lin- ear complexity: Application to Training of Deep Neural Netw orks”. In: (2019). arXiv: 1907.05951 [cs.NE]. URL : http://arxiv.org/abs/1907.05951

2019 arXiv
[39]

Natural evolution strategies

Daan Wierstra et al. “Natural evolution strategies”. I n: J. Mach. Learn. Res. 15.1 (2014), pp. 949–980. ISSN : 1532-4435

2014
[40]

Neural Network Learni ng without Backpropagation

Bogdan M. Wilamowski and Hao Y u. “Neural Network Learni ng without Backpropagation”. In: IEEE Transactions on Neural Networks 21.11 (2010), pp. 1793–1803

2010
[41]

A Gradient-Guided Evolutionar y Approach to Training Deep Neural Networks

Shangshang Y ang et al. “A Gradient-Guided Evolutionar y Approach to Training Deep Neural Networks”. In: IEEE Transactions on Neural Networks and Learning Systems 33.9 (2022), pp. 4861–4875. DOI : 10.1109/TNNLS.2021.3061630. 15

2022
[493]

DOI : 10.1109/TAI.2021.3067574

2021
[2017]

URL : https://arxiv.org/abs/1703.03864

arXiv: 1703.03864 [stat.ML]. URL : https://arxiv.org/abs/1703.03864

arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Gradient Followi ng Without Back-Propagation in Layered Networks

Andrew G. Barto and Michael I. Jordan. “Gradient Followi ng Without Back-Propagation in Layered Networks”. In: International Conference on Neural Networks . 1987, pp. 629–636

work page 1987

[2] [2]

Gradients without Backprop agation

Atılım Güne¸ s Baydin et al. “Gradients without Backprop agation”. In: (2022). arXiv: 2202.08587 [cs.LG]

arXiv 2022

[3] [3]

C. M. Bishop. Neural networks for pattern recognition . Oxford: Oxford university press, 1995

work page 1995

[4] [4]

C. M. Bishop. Pattern recognition and machine learning. Berlin: Springer, 2006

work page 2006

[5] [5]

How to Train Your Wide Neural Network Without Backprop: An Input-Weight Alignment Perspective

Akhilan Boopathy and Ila Fiete. “How to Train Y our Wide Ne ural Network Without Back- prop: An Input-Weight Alignment Perspective”. In: (2021). arXiv: 2106.08453 [cs.LG]

work page Pith review arXiv 2021

[6] [6]

Supervised Learning in Ne ural Networks without Feed- back Networks

Robert D. Brandt and Feng Lin. “Supervised Learning in Ne ural Networks without Feed- back Networks”. In: IEEE International Symposium on Intelligent Control . 1996. DOI : 10.1109/ISIC.1996.556182

work page arXiv 1996

[7] [7]

Classiﬁcation of Rice V ari eties Using Artiﬁcial Intelligence Methods

Ilkay Cinar and Murat Koklu. “Classiﬁcation of Rice V ari eties Using Artiﬁcial Intelligence Methods”. In: International Journal of Intelligent Systems and Applicat ions in Engineering 7.3 (2019), pp. 188–194. DOI : 10.18201/ijisae.2019355381

work page doi:10.18201/ijisae.2019355381 2019

[8] [8]

Adaptive Subg radient Methods for Online Learning and Stochastic Optimization

John Duchi, Elad Hazan, and Y oram Singer. “Adaptive Subg radient Methods for Online Learning and Stochastic Optimization”. In: Journal of Machine Learning Research 12 (2011), pp. 2121–2159

work page 2011