Forward gradient framework for PQCs unifies SPSA and parameter-shift as limits, introduces QUIVER adaptive optimizer with closed-form measurement allocation, and demonstrates efficient training of 60-qubit circuits on ECG5000 and MNIST.
hub
The forward-forward algorithm: Some preliminary investigations
33 Pith papers cite this work, alongside 118 external citations. Polarity classification is still indexing.
33
Pith papers citing it
118
external citations · Pith
abstract
The aim of this paper is to introduce a new learning procedure for neural networks and to demonstrate that it works well enough on a few small problems to be worth further investigation. The Forward-Forward algorithm replaces the forward and backward passes of backpropagation by two forward passes, one with positive (i.e. real) data and the other with negative data which could be generated by the network itself. Each layer has its own objective function which is simply to have high goodness for positive data and low goodness for negative data. The sum of the squared activities in a layer can be used as the goodness but there are many other possibilities, including minus the sum of the squared activities. If the positive and negative passes could be separated in time, the negative passes could be done offline, which would make the learning much simpler in the positive pass and allow video to be pipelined through the network without ever storing activities or stopping to propagate derivatives.
hub tools
citation-role summary
background 3
citation-polarity summary
roles
background 3polarities
background 3representative citing papers
An argument paper reframes LLM explainability as an embodied, situated practice based on Dourish and enactivist cognition, identifying ontological obstacles in internal explanations and advocating affordance-based designs.
DTG-FF reaches 91.8% on CIFAR-10 and 49.4% on ImageNet-100 224x224 but BP baselines beat it by 2.4-5.93 pp with gaps widening by class count on real data while reversing the synthetic trend.
FFR adapts Forward-Forward learning to regression via ordinal competitive goodness, stratified ladder layers, and hierarchical uncertainty-aware prediction, recovering 98.6% of backpropagation accuracy with substantially lower peak memory.
SBD unifies broadcast credit assignment for losses including cross-entropy via output-score orthogonality to hidden activations and reports performance gains on CIFAR-10 and Tiny ImageNet.
A taxonomy of SNN training algorithms is presented with the release of NeuroTrain, an open benchmarking framework for reproducible comparisons across datasets and architectures.
Shape- and peak-sensitive goodness functions for Forward-Forward deliver up to 72pp gains over sum-of-squares, reaching 98.2% on MNIST and 89% on Fashion-MNIST.
A hierarchical ranking-filter architecture learns classifiers entirely in permutation space via Spearman's footrule and non-gradient updates, claiming competitive accuracy without core floating-point parameters.
FFN performs efficient test-time training on multi-hour videos by forgetting the exiting frame, anticipating the next, and adapting only when a surprise metric exceeds a dynamic threshold.
TL++ recovers centralized mini-batch gradients via virtual batches in split learning and adds secret sharing for cut-layer tensors, achieving 91.41% accuracy on CIFAR-10 with 13x lower communication than full-model sync.
BlockTrain partitions models into block-local diffusion objectives that train near end-to-end WikiText quality with one-block worker memory, real WAN transport, and one-sweep distributed serving.
Standard accuracy and aggregate cosine metrics for feedback alignment are insufficient because they miss measurement degeneracy in terminal layers and aggregation collapse across depths; a new protocol using scale stability, reference validity, and depth utility plus per-layer reporting detects thes
Learnable channel-class assignment and adaptive layer weighting allow forward-only CNNs to reach new state-of-the-art results among FF models on CIFAR-10, CIFAR-100, and Tiny-ImageNet.
Intrinsic computational functionalism uses system-intrinsic instantiation (C1) and causal-dynamical organisation under intervention (C2) to identify observer-independent computational structures for consciousness via a three-tier decomposition of identification work.
Synthetic gradients can achieve arbitrarily lower gradient-estimation MSE than backpropagation under characterized conditions in a new vectorized feedback framework for computational graphs.
HCL-FF augments the Forward-Forward algorithm with hierarchical learning and contrastive objectives to reach new state-of-the-art accuracies among FF methods on CIFAR-10 (+5.46%), CIFAR-100 (+17.00%), and Tiny-ImageNet (+12.51%).
Layerwise self-supervised local rules learn the hierarchical structure of the Random Hierarchy Model as data-efficiently as supervised backpropagation, while direct feedback approximations fail due to missing masking nonlinearities.
AMSGA extends Forward-Forward learning via multi-scale goodness aggregation, curriculum-guided hard negative mining, and adaptive thresholds, reporting up to 1.5% accuracy gains on MNIST and Fashion-MNIST.
Cumulative-goodness Forward-Forward networks exhibit layer free-riding where discrimination gradients decay exponentially with prior positive margins; per-block, hardness-gated, and depth-scaled remedies yield 4-45x better layer separation but <1% accuracy change on CIFAR and Tiny ImageNet.
A complete RTL substrate executes discrete-time predictive coding dynamics directly in hardware with fixed local rules and adjacent-layer communication only.
A framework combining stochastic zeroth-order optimization and dynamic low-rank surrogate modeling with an implicit projector-splitting integrator enables end-to-end training of hybrid neural networks containing black-box physical layers and reaches near-digital accuracy on vision, audio, and text任务
MemFlow uses forward-only memorization via randomly connected neurons and spiking signals on a frozen backbone for unsupervised domain adaptation, claiming up to 10% gains at under 1% of traditional compute costs.
Single-parameter Monte Carlo mutation-selection trains deep networks and a simple Transformer on MNIST and Tiny Shakespeare without backpropagation.
AI agents lack the persistent identity and feedback mechanisms needed for consequence reception, requiring new architectures or continued human accountability.
citing papers explorer
-
Adaptive directional gradients for parameterised quantum circuits
Forward gradient framework for PQCs unifies SPSA and parameter-shift as limits, introduces QUIVER adaptive optimizer with closed-form measurement allocation, and demonstrates efficient training of 60-qubit circuits on ECG5000 and MNIST.
-
Embodied Explainability and Ontological Obstacles: Why We Struggle to Explain the Answers of Large Language Models (LLMs)
An argument paper reframes LLM explainability as an embodied, situated practice based on Dourish and enactivist cognition, identifying ontological obstacles in internal explanations and advocating affordance-based designs.
-
Synthetic Benchmarks Overstate Forward-Forward Scaling: Real-Data Limits of Layer-Local Training
DTG-FF reaches 91.8% on CIFAR-10 and 49.4% on ImageNet-100 224x224 but BP baselines beat it by 2.4-5.93 pp with gaps widening by class count on real data while reversing the synthetic trend.
-
FFR: Forward-Forward Learning for Regression
FFR adapts Forward-Forward learning to regression via ordinal competitive goodness, stratified ladder layers, and hierarchical uncertainty-aware prediction, recovering 98.6% of backpropagation accuracy with substantially lower peak memory.
-
Score Broadcast and Decorrelation: A General Framework for Broadcast-Based Credit Assignment
SBD unifies broadcast credit assignment for losses including cross-entropy via output-score orthogonality to hidden activations and reports performance gains on CIFAR-10 and Tiny ImageNet.
-
NeuroTrain: Surveying Local Learning Rules for Spiking Neural Networks with an Open Benchmarking Framework
A taxonomy of SNN training algorithms is presented with the release of NeuroTrain, an open benchmarking framework for reproducible comparisons across datasets and architectures.
-
Selectivity and Shape in the Design of Forward-Forward Goodness Functions
Shape- and peak-sensitive goodness functions for Forward-Forward deliver up to 72pp gains over sum-of-squares, reaching 98.2% on MNIST and 89% on Fashion-MNIST.
-
ArrowFlow: Hierarchical Machine Learning in the Space of Permutations
A hierarchical ranking-filter architecture learns classifiers entirely in permutation space via Spearman's footrule and non-gradient updates, claiming competitive accuracy without core floating-point parameters.
-
Forget, Anticipate and Adapt: Test Time Training for Long Videos
FFN performs efficient test-time training on multi-hour videos by forgetting the exiting frame, anticipating the next, and adapting only when a surprise metric exceeds a dynamic threshold.
-
TL++: Accuracy and Privacy Preserving Traversal Learning for Distributed Intelligent Systems
TL++ recovers centralized mini-batch gradients via virtual batches in split learning and adds secret sharing for cut-layer tensors, achieving 91.41% accuracy on CIFAR-10 with 13x lower communication than full-model sync.
-
Decentralised AI Training and Inference with BlockTrain
BlockTrain partitions models into block-local diffusion objectives that train near end-to-end WikiText quality with one-block worker memory, real WAN transport, and one-sweep distributed serving.
-
What Accuracy and Gradient Cosine Miss: Evaluating Feedback Alignment via Scale Stability, Reference Validity, and Depth Utility
Standard accuracy and aggregate cosine metrics for feedback alignment are insufficient because they miss measurement degeneracy in terminal layers and aggregation collapse across depths; a new protocol using scale stability, reference validity, and depth utility plus per-layer reporting detects thes
-
Forward-Only Convolutional Neural Networks with Learnable Channel-Class Assignment
Learnable channel-class assignment and adaptive layer weighting allow forward-only CNNs to reach new state-of-the-art results among FF models on CIFAR-10, CIFAR-100, and Tiny-ImageNet.
-
Intrinsic Computational Functionalism: From Observer-Relative Maps to Observer-Independent Structures
Intrinsic computational functionalism uses system-intrinsic instantiation (C1) and causal-dynamical organisation under intervention (C2) to identify observer-independent computational structures for consciousness via a three-tier decomposition of identification work.
-
Is Backpropagation Optimal? When Synthetic Gradients Improve Sample Efficiency
Synthetic gradients can achieve arbitrarily lower gradient-estimation MSE than backpropagation under characterized conditions in a new vectorized feedback framework for computational graphs.
-
HCL-FF: Hierarchical and Contrastive Learning for Forward-Forward Algorithm
HCL-FF augments the Forward-Forward algorithm with hierarchical learning and contrastive objectives to reach new state-of-the-art accuracies among FF methods on CIFAR-10 (+5.46%), CIFAR-100 (+17.00%), and Tiny-ImageNet (+12.51%).
-
Self-supervised local learning rules learn the hidden hierarchical structure of high-dimensional data
Layerwise self-supervised local rules learn the hierarchical structure of the Random Hierarchy Model as data-efficiently as supervised backpropagation, while direct feedback approximations fail due to missing masking nonlinearities.
-
Adaptive Multi-Scale Goodness Aggregation for Forward-Forward Learning
AMSGA extends Forward-Forward learning via multi-scale goodness aggregation, curriculum-guided hard negative mining, and adaptive thresholds, reporting up to 1.5% accuracy gains on MNIST and Fashion-MNIST.
-
Cumulative-Goodness Free-Riding in Forward-Forward Networks: Real, Repairable, but Not Accuracy-Dominant
Cumulative-goodness Forward-Forward networks exhibit layer free-riding where discrimination gradients decay exponentially with prior positive margins; per-block, hardness-gated, and depth-scaled remedies yield 4-45x better layer separation but <1% accuracy change on CIFAR and Tiny ImageNet.
-
A Synthesizable RTL Implementation of Predictive Coding Networks
A complete RTL substrate executes discrete-time predictive coding dynamics directly in hardware with fixed local rules and adjacent-layer communication only.
-
Low-rank surrogate modeling and stochastic zero-order optimization for training of neural networks with black-box layers
A framework combining stochastic zeroth-order optimization and dynamic low-rank surrogate modeling with an implicit projector-splitting integrator enables end-to-end training of hybrid neural networks containing black-box physical layers and reaches near-digital accuracy on vision, audio, and text任务
-
MemFlow: A Lightweight Forward Memorizing Framework for Quick Domain Adaptive Feature Mapping
MemFlow uses forward-only memorization via randomly connected neurons and spiking signals on a frozen backbone for unsupervised domain adaptation, claiming up to 10% gains at under 1% of traditional compute costs.
-
Beyond Backpropagation: Monte Carlo Method Can Train Deep Neural Networks
Single-parameter Monte Carlo mutation-selection trains deep networks and a simple Transformer on MNIST and Tiny Shakespeare without backpropagation.
-
Some[Body] Must Receive That Pain for Agent Accountability
AI agents lack the persistent identity and feedback mechanisms needed for consequence reception, requiring new architectures or continued human accountability.
-
Position: Zeroth-Order Optimization in Deep Learning Is Underexplored, Not Underpowered
Zeroth-order optimization is underexplored rather than underpowered in deep learning, with limitations stemming from full-space designs that can be addressed via subspace, spectral, and systems-aware approaches.
-
FAAST: Forward-Only Associative Learning via Closed-Form Fast Weights for Test-Time Supervised Adaptation
FAAST performs test-time supervised adaptation by analytically deriving fast weights from examples in one forward pass, matching backprop performance with over 90% less adaptation time and up to 95% memory savings versus memory-based methods.
-
Physical Foundation Models: Fixed hardware implementations of large-scale neural networks
Physical Foundation Models are fixed physical hardware realizations of foundation-scale neural networks that compute via inherent material dynamics, potentially delivering orders-of-magnitude gains in energy efficiency, speed, and density over digital systems.
-
Introducing Echo Networks for Computational Neuroevolution
Echo Networks are recurrent networks defined by a single connection matrix with no layers, enabling matrix-based mutation and recombination in neuroevolution, and demonstrated on ECG signal classification.
-
LightTune: Lightweight Forward-Only Online Fine-Tuning with Applications to Link Adaptation
LightTune is a backpropagation-free online fine-tuning framework that reduces BLER prediction error by up to 48.8% and improves throughput by 15.5% in 6G link adaptation.
-
Cultivating Machine Intelligence: The OMEGA Shift from Top-Down Optimization to Autopoietic Cognitive Ecologies
The paper introduces the RECLAIM framework and OMEGA shift as a transition from top-down optimization to autopoietic cognitive ecologies for cultivating machine intelligence.
-
Grid-SD2E: A General Grid-Feedback in a System for Cognitive Learning
Proposes Grid-SD2E, a theoretical grid-feedback cognitive learning system combining grid-cell inspiration with Bayesian reasoning for self-reinforcing interaction.
- Breaking Chains with Trees: Model-Parallel Deep Learning with $\mathcal{O}(\log N)$ Time Complexity
- Training Non-Differentiable Networks via Optimal Transport