Pith. sign in

REVIEW 40 cited by

The Forward-Forward Algorithm: Some Preliminary Investigations

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2212.13345 v1 pith:MBB4B7VM submitted 2022-12-27 cs.LG

Geoffrey Hinton This is my paper · ORCID
classification cs.LG
keywords datanegativepassespositiveactivitiesgoodnessalgorithmforward
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

The aim of this paper is to introduce a new learning procedure for neural networks and to demonstrate that it works well enough on a few small problems to be worth further investigation. The Forward-Forward algorithm replaces the forward and backward passes of backpropagation by two forward passes, one with positive (i.e. real) data and the other with negative data which could be generated by the network itself. Each layer has its own objective function which is simply to have high goodness for positive data and low goodness for negative data. The sum of the squared activities in a layer can be used as the goodness but there are many other possibilities, including minus the sum of the squared activities. If the positive and negative passes could be separated in time, the negative passes could be done offline, which would make the learning much simpler in the positive pass and allow video to be pipelined through the network without ever storing activities or stopping to propagate derivatives.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 40 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 118 citations worldwide. Full citation record

  1. Learning in Deep Networks under Dale's Constraint

    cs.AI 2026-08 reject novelty 7.0 of 10

    An on-off two-channel network with fixed-sign synapses and local Hebbian learning is claimed to recover backpropagation exactly under symmetric weights and to beat comparable vanilla networks on Tiny ImageNet.

  2. Gauge-Fixing the Forward-Forward Objective: A Whitened Goodness Derived from a Likelihood-Ratio Account

    cs.LG 2026-07 conditional novelty 7.0 of 10

    Squared Forward-Forward goodness is the likelihood-ratio statistic for zero-mean populations differing in scale; anisotropic and heavy-tailed cases yield Mahalanobis and saturating (divisive-normalization) forms.

  3. Forget, Anticipate and Adapt: Test Time Training for Long Videos

    cs.CV 2026-06 unverdicted novelty 7.0 of 10

    FFN performs TTT on multi-hour videos by restricting updates to three frames and using a surprise metric for adaptive window sizing, plus a new EpicTours dataset.

  4. Can Local Learning Match Self-Supervised Backpropagation?

    cs.LG 2026-01 conditional novelty 7.0 of 10

    With orthonormal linear networks and optimized layer-wise projections, local-SSL updates equal global BP-SSL updates; adding top-down and 2D-spatial structure to CLAPP then nearly matches BP-SSL on CIFAR-10, STL-10, a...

  5. Programmable Photonic Unitary Processor Enables Parametrized Differentiable Long-Haul Spatial Division Multiplexed Transmission

    physics.optics 2025-05 conditional novelty 7.0 of 10

    A programmable photonic unitary processor, optimized through a differentiable fiber model, is shown in a 1300-km three-mode fiber experiment to cut modal dispersion by more than half.

  6. From Local Learning to Global Prediction Through Layered Surprise Cascades

    q-bio.NC 2026-08 conditional novelty 6.0 of 10

    An inverted Forward-Forward rule makes layered networks cancel expected activity and amplify surprise, producing brain-like bottom-up cascades.

  7. LoCA: Forward-Only LLM Tuning after One-Shot Calibration with Local Credit Assignment

    cs.AI 2026-08 conditional novelty 6.0 of 10

    LoCA tunes adapters with one calibrated map per layer and closed-form ridge solves, avoiding repeated backpropagation after calibration and roughly matching LoRA quality on tested tasks.

  8. Versatile On-device Adaptation at the Edge by Unifying Few-shot, Zero-shot, Continual, and In-context Learning

    cs.LG 2026-07 conditional novelty 6.0 of 10

    ECL unifies few-shot, continual, zero-shot, and in-context learning on a single TCN-based edge chip, with first hardware baselines on several tasks.

  9. Breaking the Curse with BAND: Nonparametric Distribution Estimation in High Dimensions

    stat.ML 2026-07 conditional novelty 6.0 of 10

    Sparse Bayesian-network factorization plus sparsity-aware regression yields polynomial TV rates for high-dimensional mixed-type distribution estimation, beating classical histogram rates under sparsity.

  10. Reconstructing Backpropagation from Forward Fluctuations in Noise-modulated Neural Networks

    cs.NE 2026-07 conditional novelty 6.0 of 10

    In a noise-modulated neural network, backpropagation's transposed-weight multiplications can be replaced by covariance estimates from forward fluctuations, matching backpropagation accuracy on small tasks.

  11. Conditioned Direct Feedback Alignment via Activity and Error Geometry

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Conditioning the activity and error factors of the direct feedback alignment update with damped inverse second moments improves DFA accuracy in nuisance-dominated regimes and clean confirmations.

  12. Backpropagation-Free Trunk Training via the Split Forward Gradients

    stat.ML 2026-07 conditional novelty 6.0 of 10

    Split-FG splits a network into an exactly trained head and a forward-gradient-estimated trunk, reducing variance and reaching 387 perplexity on WikiText-103 with a 16M GPT-2-style model.

  13. Local Synaptic Rules Can Implement a SIGReg Gradient Without Backpropagation

    cs.NE 2026-07 conditional novelty 6.0 of 10

    STDP+ and homeostatic plasticity can compute the exact gradient of a SIGReg-like loss using only local signals, verified in a clustering task and on MNIST.

  14. Shunting Inhibition and Dendritic Branching Shape Local Credit Assignment

    q-bio.NC 2026-07 conditional novelty 6.0 of 10

    Exact dendritic gradients factor into local eligibility times path-transported compartment error, and shunting inhibition can improve restricted-feedback local learning by reshaping that error field.

  15. Decentralised AI Training and Inference with BlockTrain

    cs.AI 2026-06 unverdicted novelty 6.0 of 10

    BlockTrain partitions models into blocks trained on local objectives, reaching CE 1.359 on WikiText within 0.04 of end-to-end baseline while enabling distributed training and inference over TCP for up to 75B-parameter models.

  16. Self-Motivated Growing Neural Network for Adaptive Architecture via Local Structural Plasticity

    cs.NE 2025-12 conditional novelty 6.0 of 10

    A gradient-trained control network whose size adjusts online through a local structural plasticity module matches or beats fixed-size MLPs on three control benchmarks.

  17. Decoupling Search and Learning in Neural Net Training

    cs.LG 2025-09 conditional novelty 6.0 of 10

    Training can be split into evolutionary search over intermediate activations and gradient-based regression to those activations, achieving near-SGD accuracy on three benchmarks.

  18. Forward-Only Continual Learning

    cs.LG 2025-09 conditional novelty 6.0 of 10

    FoRo achieves strong continual learning accuracy and low forgetting on CIFAR-100, ImageNet-R, and CUB-200 using only forward updates, via CMA-ES prompt tuning and a recursive knowledge encoding matrix.

  19. Eigen Neural Network: Unlocking Generalizable Vision with Eigenbasis

    cs.CV 2025-08 unverdicted novelty 6.0 of 10

    Reparameterizing layer weights in a learned orthonormal eigenbasis is claimed to improve ImageNet classification, cross-modal retrieval, and enable a faster backpropagation-free variant that surpasses standard backpro...

  20. FFGAF-SNN: The Forward-Forward Based Gradient Approximation Free Training Framework for Spiking Neural Networks

    cs.CV 2025-07 conditional novelty 6.0 of 10

    A Forward-Forward training framework that freezes spiking layers as black-box encoders and allocates channels by inter-class difficulty achieves 99.58% on MNIST, 92.13% on Fashion-MNIST, and 75.64% on CIFAR-10, the be...

  21. Can Biologically Plausible Temporal Credit Assignment Rules Match BPTT for Neural Similarity? E-prop as an Example

    cs.NE 2025-06 conditional novelty 6.0 of 10

    At matched task accuracy, e-prop trained RNNs reach neural data similarity comparable to BPTT trained RNNs on Mante 2013 and Sussillo 2015 datasets, with initialization and architecture influencing similarity more tha...

  22. Backpropagation-Free Metropolis-Adjusted Langevin Algorithm

    cs.LG 2025-05 conditional novelty 6.0 of 10

    Forward-mode automatic differentiation can replace backpropagation inside the Metropolis-Adjusted Langevin Algorithm, yielding four MCMC samplers that are sometimes faster and lower-memory than standard MALA.

  23. Forward Target Propagation: A Forward-Only Approach to Global Error Credit Assignment via Local Losses

    cs.LG 2025-05 conditional novelty 6.0 of 10

    FTP trains neural networks using only forward passes, propagating random-projection target signals through the network, and achieves near-backpropagation accuracy on small shallow benchmarks.

  24. An In-depth Investigation of Sparse Rate Reduction in Transformer-like Models

    cs.LG 2024-11 conditional novelty 6.0 of 10

    In CRATE-family transformers, the attention-like MSSA update with skip connection raises the coding rate it was designed to compress, yet layer-averaged SRR still correlates positively with the generalization gap (tau...

  25. Train Often, Deploy Selectively: Forward-Gated Model Replacement in Crypto Markets

    cs.CE 2026-07 conditional novelty 5.5 of 10

    Shadow Before Swap promotes warm-refit challengers only after a paired one-week NLL edge, beating calendar, blind, and continuous-maintenance baselines while cutting deployments 78%.

  26. Beyond Backpropagation: Monte Carlo Method Can Train Deep Neural Networks

    cs.LG 2026-07 conditional novelty 5.0 of 10

    Single-parameter Monte Carlo mutation-selection trains deep networks and a simple Transformer on MNIST and Tiny Shakespeare without backpropagation.

  27. Steering the Noise: Turning Random Perturbations into Effective Descent for Memory-Efficient LLM Fine-Tuning

    cs.CL 2026-01 conditional novelty 5.0 of 10

    Selecting or combining the lowest-loss random perturbations before each update makes zeroth-order LLM fine-tuning converge faster, reportedly beating gradient-based fine-tuning on 9 of 11 tasks at a fraction of the memory.

  28. Reshaping the Forward-Forward Algorithm with a Similarity-Based Objective

    cs.LG 2025-08 conditional novelty 5.0 of 10

    FAUST replaces the Forward-Forward goodness score with triplet/tuplet similarity losses and achieves near-backpropagation accuracy on MNIST, Fashion-MNIST, and CIFAR-10 with single-pass inference.

  29. FF-INT8: Efficient Forward-Forward DNN Training on Edge Devices with INT8 Precision

    cs.LG 2025-06 reject novelty 5.0 of 10

    INT8 quantized training using the Forward-Forward algorithm with a look-ahead loss achieves competitive accuracy and modest resource savings on an edge device.

  30. Taxonomic Networks: A Representation for Neuro-Symbolic Pairing

    cs.AI 2025-05 conditional novelty 5.0 of 10

    A Cobweb-style symbolic concept learner and a neural soft decision tree are presented as interchangeable 'neuro-symbolic pairs' over taxonomic networks, with complementary data and compute tradeoffs.

  31. Scaling of hardware-compatible perturbative training algorithms

    cs.LG 2025-01 conditional novelty 5.0 of 10

    Training time to a fixed accuracy for perturbative gradient methods grows far slower than linearly with network size, challenging a long-standing scaling objection.

  32. Dendritic Localized Learning: Toward Biologically Plausible Algorithm

    cs.NE 2025-01 conditional novelty 5.0 of 10

    DLL trains MLPs, CNNs, and RNNs with local errors and trainable asymmetric feedback, achieving the best accuracy among algorithms satisfying the paper's three biological plausibility criteria.

  33. Hardware-In-The-Loop Training of a 4f Optical Correlator with Logarithmic Complexity Reduction for CNNs

    cs.NE 2025-01 conditional novelty 5.0 of 10

    Hardware-in-the-loop training of a 4f optical correlator with the PEPITA forward-only algorithm reaches MNIST accuracy comparable to backpropagation while removing the need for software gradients and claiming a log-fa...

  34. Scalable Forward-Forward Algorithm

    cs.LG 2025-01 conditional novelty 5.0 of 10

    SFF trains convolutional networks layer-by-layer with auxiliary class-goodness layers and block-wise local backpropagation, reaching backprop-comparable accuracy on small benchmarks in the reported runs.

  35. ZeroFlow: Overcoming Catastrophic Forgetting is Easier than You Think

    cs.CV 2025-01 conditional novelty 5.0 of 10

    ZeroFlow is a benchmark showing zeroth-order, forward-pass-only optimizers can match backpropagation-based continual learning on several datasets with about five times lower memory, plus three modest enhancements.

  36. Detection of Breast Cancer Lumpectomy Margin with SAM-incorporated Forward-Forward Contrastive Learning

    cs.CV 2025-06 reject novelty 4.0 of 10

    FFCL-SAM, a patch-level classifier plus SAM-based refinement, reports AUC 0.8455 and improved margin segmentation on intraoperative breast radiographs, but the test set excludes negative patients.

  37. Training neural networks without backpropagation using particles

    cs.NE 2024-12 conditional novelty 4.0 of 10

    Training each neuron of a multilayer perceptron with its own particle-swarm search, instead of using backpropagation, reaches accuracy comparable to gradient-based training on two real datasets and several synthetic ones.

  38. A Neural Network Training Method Based on Distributed PID Control

    cs.LG 2024-11 reject novelty 4.0 of 10

    A differential-equation-based training rule with distributed PID control is proposed for a symmetric Wuxing neural network and tested on MNIST.

  39. Bootstrapping Life-Inspired Machine Intelligence: The Biological Route from Chemistry to Cognition and Creativity

    q-bio.NC 2026-02 unverdicted novelty 3.0 of 10

    Flexible, scalable intelligence in biology is attributed to five design principles—autonomy, multiscale self-assembly, continuous rebuilding, embodied constraints, and pervasive signaling—offering a roadmap for life-i...

  40. Current Opinions on Memristor-Accelerated Machine Learning Hardware

    cs.ET 2025-01 conditional novelty 1.0 of 10

    A review of memristor accelerators for machine learning, covering prototype chips, device/circuit/system challenges, and future directions.

Pith tools