Pith. sign in

REVIEW 3 major objections 3 minor 6 references

Bio-Inspired Artificial Neural Networks based on Predictive Coding

T0 review · 3 major / 3 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A tutorial introduction to Predictive Coding derives it from the free-energy principle, shows it can reproduce backpropagation with local updates, connects it to Kalman filtering, and benchmarks its real-world cost.

desk verdict Read it as lecture notes, not as a research contribution; the tutorial has genuine pedagogical value, but its accuracy depends on an equation-by-equation check that the excerpt doesn't allow. read the letter →

arxiv 2508.08762 v1 pith:XMFAFI75 submitted 2025-08-12 stat.ML cs.LG

Davide Casnici , Charlotte Frenkel , Justin Dauwels This is my paper · ORCID
classification stat.MLcs.LG
keywords predictivecodingbackpropagationfreeenergyprinciplebiologicallyplausiblelearninglocalrulesKalmanfilterneuralnetworkstutorial
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This lecture-notes paper argues that Predictive Coding (PC) is a concrete learning algorithm, not just a metaphor: minimizing a free-energy objective over prediction errors, with an inference phase and a learning phase, yields weight updates that are local, biologically plausible, and that can exactly reproduce backpropagation in the right limit. The tutorial derives this chain from first principles, connects PC to Kalman filtering for temporal settings, and provides working Python implementations of classification and compression experiments. Its experiments show PC matching backpropagation on small and medium datasets, requiring roughly half the generative parameters in compression at comparable error, at the price of more computation. On deeper and larger-scale tasks, PC loses to backpropagation, and the paper states this scalability limit as open. A careful reader should come away able to implement PC and understand both its appeal and its cost.

What carries the argument

The load-bearing object is the variational (negative) free energy, defined as the sum of precision-weighted squared prediction errors between value nodes and prediction nodes. Training alternates an inference phase, in which neural activities are adjusted to minimize this energy, and a learning phase, in which weights are updated by the local product of pre-synaptic activity and post-synaptic error. This single objective is what generates the equivalence to backpropagation, the connection to Kalman filtering, and the uncertainty-based gradient scaling that PC offers.

What would settle it

Implement PC on a small multilayer perceptron with squared error, run the inference phase to equilibrium, and compare the learning-phase weight updates to the exact backpropagation gradient; any disagreement beyond numerical precision would falsify the central equivalence. A second check is to rerun the paper's compression experiment on MNIST and verify that at matched MSE the PC model really uses about half of BP's generative parameters.

Watch

Extended reading notes

Core claim

The paper's core claim is that Predictive Coding networks are a fully specified alternative training rule whose mathematical backbone is the same free-energy functional used in Bayesian inference of latent variables. Starting from the negative free energy, the tutorial shows that the learning rule for each synapse uses only the activity of the pre-synaptic neuron and the local prediction error, giving automatic uncertainty scaling through precision terms. It then shows that when the squared-error loss is used and inference is run to equilibrium, PC's updates match those of backpropagation; when temporal dynamics are introduced, the same formalism becomes a neuronal Kalman filter. The accompa

Load-bearing premise

The tutorial's chain of equivalences holds only if its rendering of the free-energy principle and the cited PC results is faithful; if the mathematical transcription of those sources is wrong, readers would be misled even when the Python code runs.

Editorial extensions

If this is right

  • A reader can implement PC from the tutorial and train small networks to accuracy comparable with BP, with every weight update local.
  • In compression experiments, PC needs about half the generative parameters of BP at similar MSE, pointing to a structural efficiency in the representation, not a tuning detail.
  • PC's inference loop is the main practical bottleneck: more FLOPs and longer training times are the expected price of locality.
  • The Kalman-filter connection extends PC to temporal prediction tasks, where the same local rule can track changing inputs.
  • For deep and large-scale architectures, BP remains the stronger algorithm; the paper's own benchmark comparison establishes that limit.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because precision weights scale errors by uncertainty, one testable extension is that PC should be more robust than BP to heteroscedastic noise; a dataset with known per-sample noise would reveal this without any algorithmic change.
  • The iterative inference loop could be shortened by learning an amortized guess for the initial activity state, converting PC from many steps to a single forward pass; the paper does not explore this, but its formulation permits it.
  • The compression-friendly parameter count suggests PC may be performing a rate-distortion trade-off intrinsically, so comparing PC-trained compressed representations to standard autoencoders on identical data would test whether the efficiency is generic.
  • If the backpropagation equivalence is exact at equilibrium, PC can serve as a local surrogate for BP in settings where global error signals are unavailable, such as on-chip training; this is a direct consequence the tutorial leaves implicit.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. This manuscript is a tutorial-style introduction to Predictive Coding (PC) as a biologically plausible alternative to backpropagation (BP) for training neural networks. It derives PC from the free-energy principle, connects PC to BP and the Kalman filter, and reports experimental comparisons on MNIST, FashionMNIST, and CIFAR-10, including a FLOPs analysis. The central claim is that the exposition enables readers to understand and implement PC networks from first principles, with PyTorch-based examples.

Significance. If the derivations are faithful to the cited literature and the Python examples are correct, this tutorial could be a valuable pedagogical resource, filling a gap between scattered papers on predictive coding and practical implementations. The paper also honestly discusses the scalability limitations of PC relative to BP. However, the presented text does not allow independent verification of the mathematical fidelity or the empirical claims, and no code or data are provided in the manuscript excerpt. These issues directly bear on the central pedagogical and comparative claims.

major comments (3)
  1. [Derivation of the free-energy surrogate and PC update rules] The central derivation is not fully present in the manuscript text provided for review. The paper promises an exact translation of predictive coding from Song et al. (2020) and Millidge et al. (2021), but the provided excerpt does not include the numbered equations for the free-energy surrogate, the inference dynamics, or the weight-update rules. In particular, the exact stopping condition under which PC reproduces BP, including sign conventions and any scaling factors, must be stated explicitly. Without these equations, a reader cannot implement the method, and the core pedagogical claim is not verifiable.
  2. [Table 1 and the experimental section] The empirical comparison in Table 1 reports FLOPs, training times, and accuracy/MSE for PC and BP on three datasets, but no code, data, hyperparameters, or experimental details are included in the provided text. The statement that FLOPs estimates were obtained with the PyTorch Profiler is insufficient to reproduce the numbers. Since the paper claims that PC requires only half the generative parameters in the compression task and quantifies the computational overhead, these claims need a reproducibility package or a clear experimental protocol before they can be accepted.
  3. [Python examples / implementation] The abstract and introduction state that the tutorial includes practical implementation using PyTorch, but no code listings, pseudocode, or repository link appear in the provided manuscript. If the code is intended to be online, it must be cited and made available; if it is in an appendix, that appendix needs to be included. This is a load-bearing component of the 'practical implementation' claim.
minor comments (3)
  1. [Final limitation paragraph] Typo: 'improving its efficiency and performance on large-scale tasks remains an open and actively researched challenge.' The phrase 'the the latter' appears earlier in the same paragraph; should be 'the latter.'
  2. [Experimental tasks] In the text, 'CIF AR10' should be written as 'CIFAR-10' for consistency and clarity.
  3. [Table 1] The table lacks confidence intervals or standard deviations for accuracy/MSE, which is important for small-scale comparisons. Please add these or state that the numbers are single runs.

Circularity Check

0 steps flagged · score 0.0 of 10

Tutorial is expository and self-contained; no circular derivation or fitted prediction identified.

full rationale

The paper is a lecture-notes tutorial that restates predictive coding from the free-energy principle and links it to backpropagation and Kalman filtering using prior external work (Song et al. 2020, Millidge et al. 2021, etc.). No parameter is fitted to data and then renamed a prediction; no central claim reduces by construction to its inputs. The derivation chain—free energy as a variational bound, alternating inference and parameter updates—is presented as a restatement of established formulations, and the cited works are external, not the authors' own. The self-referential limitation paragraph (scalability relative to BP) is a frank assessment and does not function as evidence for any derivation. The skeptic's concern that the excerpt does not permit independent verification of sign conventions and scalings is a correctness/verifiability risk, not a circularity: incorrect reproduction of an external source would be an accuracy error, not a reduction of the conclusion into its premise. Accordingly, no circular step meeting the quoted-evidence standard can be exhibited.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper is an exposition and therefore relies entirely on assumptions imported from the cited literature. The free energy principle and Gaussian noise assumptions are standard for PC, and the biological plausibility claim is a domain assumption from neuroscience.

assumptions (3)
  • domain assumption Free Energy Principle
    The paper builds PC on the minimization of variational free energy, a framework from Friston's neuroscience literature, used as a starting point without proving it.
  • domain assumption Gaussian likelihood assumption
    Used to derive squared-error loss terms in PC objectives, assuming Gaussian noise in the generative model.
  • domain assumption Local updates are biologically plausible
    The motivation that Hebbian-like local updates are what brains use, which is an assumption about neuroscience.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Bio-Inspired Artificial Neural Networks based on Predictive Coding." pith.science (2026). https://pith.science/paper/XMFAFI75

@misc{pith2026250808762,
  author       = {Pith},
  title        = {Pith review of: Bio-Inspired Artificial Neural Networks based on Predictive Coding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XMFAFI75}},
  note         = {Machine review of arXiv:2508.08762}
}
read the original abstract

Backpropagation (BP) of errors is the backbone training algorithm for artificial neural networks (ANNs). It updates network weights through gradient descent to minimize a loss function representing the mismatch between predictions and desired outputs. BP uses the chain rule to propagate the loss gradient backward through the network hierarchy, allowing efficient weight updates. However, this process requires weight updates at every layer to rely on a global error signal generated at the network's output. In contrast, the Hebbian model of synaptic plasticity states that weight updates are local, depending only on the activity of pre- and post-synaptic neurons. This suggests biological brains likely do not implement BP directly. Recently, Predictive Coding (PC) has gained interest as a biologically plausible alternative that updates weights using only local information. Originating from 1950s work on signal compression, PC was later proposed as a model of the visual cortex and formalized under the free energy principle, linking it to Bayesian inference and dynamical systems. PC weight updates rely solely on local information and provide theoretical advantages such as automatic scaling of gradients based on uncertainty. This lecture notes column offers a novel, tutorial-style introduction to PC, focusing on its formulation, derivation, and connections to well-known optimization and signal processing algorithms such as BP and the Kalman Filter (KF). It aims to support existing literature by guiding readers from the mathematical foundations of PC to practical implementation, including Python examples using PyTorch.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

6 extracted references · 5 canonical work pages

  1. [7]

    Y. Song, T. Lukasiewicz, Z. Xu, and R. Bogacz. Can the brain do backpropagation?—exact implementation of backpropagation in predictive coding networks. Advances in neural information processing systems, 33:22566–22579, 2020

    work page 2020

  2. [8]

    Neural Kalman Filtering

    B. Millidge, A. Tschantz, Anil Seth, and C. Buckley. Neural kalman filtering. arXiv preprint arXiv:2102.10021, 2021

    work page Pith review arXiv 2021

  3. [9]

    Millidge, M

    B. Millidge, M. Tang, M. Osanlouy, N. S. Harper, and R. Bogacz. Predictive coding networks for temporal prediction. PLOS Computational Biology, 20(4):e1011183, 2024

    work page 2024

  4. [10]

    C. Marsh. Introduction to continuous entropy. Department of Computer Science, Princeton University, 1034, 2013

    work page 2013

  5. [11]

    Zahid, Q

    U. Zahid, Q. Guo, and Z. Fountas. Predictive coding as a neuromorphic alternative to backpropagation: A critical evaluation. Neural Computation, 35(12):1881–1909, 2023

    work page 1909

  6. [12]

    Bench- marking predictive coding networks–made simple

    Luca Pinchetti, Chang Qi, Oleh Lokshyn, Gaspard Olivers, Cornelius Emde, Mufeng Tang, Amine M’Charrak, Simon Frieder, Bayar Menzat, Rafal Bogacz, et al. Bench- marking predictive coding networks–made simple. arXiv preprint arXiv:2407.01163, 2024. 38

    arXiv 2024

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.