Predictive Coding: a Theoretical and Experimental Review
read the original abstract
Predictive coding offers a potentially unifying account of cortical function -- postulating that the core function of the brain is to minimize prediction errors with respect to a generative model of the world. The theory is closely related to the Bayesian brain framework and, over the last two decades, has gained substantial influence in the fields of theoretical and cognitive neuroscience. A large body of research has arisen based on both empirically testing improved and extended theoretical and mathematical models of predictive coding, as well as in evaluating their potential biological plausibility for implementation in the brain and the concrete neurophysiological and psychological predictions made by the theory. Despite this enduring popularity, however, no comprehensive review of predictive coding theory, and especially of recent developments in this field, exists. Here, we provide a comprehensive review both of the core mathematical structure and logic of predictive coding, thus complementing recent tutorials in the literature. We also review a wide range of classic and recent work within the framework, ranging from the neurobiologically realistic microcircuits that could implement predictive coding, to the close relationship between predictive coding and the widely-used backpropagation of error algorithm, as well as surveying the close relationships between predictive coding and modern machine learning techniques.
This paper has not been read by Pith yet.
Forward citations
Cited by 10 Pith papers
-
Predictive Coding with Bayesian Priors via Proximal Gradients
Predictive coding equals proximal gradient descent on MAP problems, with priors setting nonlinearities via proximal operators and yielding leaky firing-rate networks plus hierarchical MRFs.
-
Training a Predictive Coding Network on ImageNet using Equilibrium Propagation
A VGG10 predictive coding network is trained on ImageNet via equilibrium propagation to 13.23% top-5 error, close to the 12.2% backpropagation baseline, marking the first such demonstration at this scale.
-
Augmented Lagrangian Predictive Coding
PC-ALM uses dual ascent on an augmented Lagrangian to achieve exact backpropagation gradients via layer-local updates in linear networks and matching performance in nonlinear networks up to depth 128.
-
Learn from your own latents and not from tokens: A sample-complexity theory
Latent prediction SSL recovers latent trees from PCFG data with sample complexity constant in hierarchy depth L (up to logs), unlike exponential for token-level or supervised methods.
-
Equilibrium Propagation and Hamiltonian Inference in the Diffusive Fitzhugh-Nagumo Model
Extends equilibrium propagation to skew-gradient Fitzhugh-Nagumo systems and derives an explicit layer-wise Hamiltonian recurrence for inference in deep residual topologies.
-
Retrieval-Augmented Personalization with Foundation Models for Wearable Stress Detection
Retrieval from out-of-domain foundation models enables personalization of a lightweight transformer for stress detection, yielding +3.92% accuracy and +4.76% F1 gains on WESAD without user labels.
-
Closed-form predictive coding via hierarchical Gaussian filters
Predictive coding is recast as deep hierarchical Gaussian filters to restore precision-weighted message passing, yielding closed-form inference and online precision learning that matches backpropagation speed on Fashi...
-
Spike-based alignment learning solves the weight transport problem
SAL is a spike-timing-based local learning rule that aligns feedback weights to forward weights in spiking networks by exploiting noise and Hebbian/anti-Hebbian plasticity to recover the true gradient.
-
Exploring Temporal Representation in Neural Processes for Multimodal Action Prediction
A revised DMBN with positional time encoding improves temporal representation and generalization in neural processes for multimodal robotic action prediction.
-
Quantitative Promise Theory: Intentionality and Inference in Autonomous Agents
Promise Theory is augmented with Bayesian methods and information optimization to model intentionality, inference, and swarm formation in autonomous agents.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.