Supervised Training Rapidly Degrades Early Visual Cortex Alignment Across Biologically Plausible Learning Rules

Nils Leutenegger

REVIEW 2 major objections 1 minor 13 references

Supervised training reduces alignment between neural network representations and human early visual cortex after just one epoch.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-06-29 08:43 UTC pith:TTHTQOSG

load-bearing objection One epoch of training cuts V1 alignment 25-90% depending on the rule, BP worst, but N=3 subjects makes the deltas hard to trust. the 2 major comments →

arxiv 2605.30556 v1 pith:TTHTQOSG submitted 2026-05-28 cs.LG q-bio.NC

Supervised Training Rapidly Degrades Early Visual Cortex Alignment Across Biologically Plausible Learning Rules

Nils Leutenegger This is my paper

classification cs.LG q-bio.NC

keywords representational similarity analysisvisual cortex alignmentlearning rulesbackpropagationpredictive codingSTDPfMRIneural network training

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper measures how representational similarity between model activations and human fMRI data changes over training epochs. It compares four learning rules on 720 object images and finds that alignment with V1 drops sharply right away, with backpropagation causing the biggest loss and local rules like predictive coding and STDP causing less. Untrained networks already match brain data well because of their built-in structure. The same training produces a small gain in alignment for higher visual areas instead. These patterns indicate that standard learning moves early representations away from the statistics captured by the initial architecture.

Core claim

Random untrained networks match or exceed trained networks in representational similarity to early visual cortex. Tracking Spearman correlations between model and brain representational dissimilarity matrices across eight checkpoints shows that a single epoch of training reduces V1 alignment by 25-90 percent depending on the rule, with backpropagation producing the largest drop and predictive coding and STDP the smallest. A weaker opposite pattern holds in object-selective cortex where backpropagation yields the largest gain.

What carries the argument

Representational similarity analysis that computes Spearman correlations between model and brain representational dissimilarity matrices on the same set of images and fMRI regions of interest.

Load-bearing premise

Spearman correlations between model and brain representational dissimilarity matrices on the THINGS images and fMRI from three subjects provide a stable measure of early visual cortex alignment not driven by dataset-specific artifacts or subject variability.

What would settle it

Re-running the same RSA pipeline on a new image set or additional fMRI subjects and observing no drop in V1 alignment after the first training epoch would falsify the central result.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

Untrained network architectures already capture low-level visual statistics through inductive biases alone.
Global error signals reshape early representations more aggressively than local learning rules.
Local rules such as predictive coding and STDP preserve more of the initial brain-like structure in V1.
Training produces a small increase in alignment for higher areas like LOC rather than early cortex.
The initial random state of the network is closer to brain data than the state after supervised updates.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Models intended to match early visual cortex may need to avoid standard end-to-end supervised objectives.
Hybrid training that combines local rules with limited global signals could maintain alignment longer.
The result raises the question of whether different image statistics or unsupervised pre-training would reverse the drop.
Architectural choices that set the untrained baseline may matter more for biological fidelity than the choice of optimizer.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

One epoch of training cuts V1 alignment 25-90% depending on the rule, BP worst, but N=3 subjects makes the deltas hard to trust.

read the letter

The main point is that supervised training, even for one epoch, reduces representational alignment with V1 more than it helps, and backprop does this faster and harder than predictive coding or STDP. They show this by running RSA on 720 THINGS images against fMRI from three subjects in six ROIs, tracking Spearman correlations at multiple checkpoints for four learning rules.

What stands out as new is the dynamic tracking: they move past the static untrained-vs-trained comparisons that already exist and give numbers on how fast the drop happens and how the rules differ. The observation that local rules preserve more alignment than global error signals is a clean extension of the inductive-bias story.

The soft spot is the brain data. Three subjects is thin for RDM correlations; without per-subject plots, bootstrap intervals, or leave-one-out checks, the reported deltas (BP at -0.080, others around -0.04) could easily reflect subject idiosyncrasies or averaging noise rather than a stable effect of the learning rule. The abstract also skips architecture sizes, exact hyperparameters, and any statistical tests, so the central claim rests on choices that are not visible.

This is for people working on biologically plausible learning and brain alignment models. A reader who wants concrete numbers on how different rules affect early visual representations will find the checkpoints useful, even if the result stays preliminary.

It deserves a serious referee because the question is timely and the comparison across rules is straightforward, but any review should focus on whether the fMRI side holds up under more scrutiny. I would send it to review with a request for robustness checks on the subject level.

Referee Report

2 major / 1 minor

Summary. The paper claims that a single epoch of training on 720 THINGS images rapidly degrades representational alignment (measured via Spearman RSA) between neural networks and human fMRI data from three subjects in six visual ROIs, with the drop in V1 ranging from 25-90% depending on the learning rule; backpropagation produces the largest reduction (delta r = -0.080) while predictive coding and STDP produce smaller drops (~ -0.04), and a weaker opposite trend appears in LOC.

Significance. If robust, the result would indicate that inductive biases in random networks already capture low-level visual statistics better than trained representations, and that global error signals reshape early cortex more aggressively than local rules. This would shift emphasis in model-brain alignment studies toward architecture and initialization rather than supervised optimization.

major comments (2)

[Abstract] Abstract and results: the headline deltas (e.g., delta r = -0.080 for BP in V1) are computed from Spearman correlations between model and brain RDMs using only three fMRI subjects. No per-subject RDMs, bootstrap CIs, or leave-one-subject-out analyses are described, so it is impossible to determine whether the reported ordering across learning rules survives subject-level noise.
[Abstract] Abstract: the numerical claims rest on unstated choices for model depth/width, optimizer settings, number of random seeds, and exact ROI definitions; without these, the 25-90% range and the BP vs. PC/STDP contrast cannot be reproduced or stress-tested.

minor comments (1)

[Abstract] Abstract: the statement that alignment is tracked at eight checkpoints (epochs 0-40) does not indicate whether the degradation plateaus after epoch 1 or continues, which is needed to interpret the 'rapidly degrades' claim.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive comments. We address each major point below and indicate the revisions we will incorporate.

read point-by-point responses

Referee: [Abstract] Abstract and results: the headline deltas (e.g., delta r = -0.080 for BP in V1) are computed from Spearman correlations between model and brain RDMs using only three fMRI subjects. No per-subject RDMs, bootstrap CIs, or leave-one-subject-out analyses are described, so it is impossible to determine whether the reported ordering across learning rules survives subject-level noise.

Authors: We agree that the small number of subjects (n=3) limits the ability to quantify inter-subject variability and that additional statistical reporting is warranted. In the revised manuscript we will add per-subject RSA correlations, bootstrap confidence intervals computed by resampling over the 720 images, and the range of deltas observed across the three subjects. Leave-one-subject-out analysis is not informative with n=3, but the ordering of learning rules is preserved in each individual subject. These additions will allow readers to evaluate robustness directly. revision: yes
Referee: [Abstract] Abstract: the numerical claims rest on unstated choices for model depth/width, optimizer settings, number of random seeds, and exact ROI definitions; without these, the 25-90% range and the BP vs. PC/STDP contrast cannot be reproduced or stress-tested.

Authors: The full Methods section specifies the model architecture, optimizer settings, number of random seeds, and ROI definitions from the THINGS fMRI dataset. To improve accessibility we will add a concise summary of these parameters to the abstract (or a methods summary box) in the revision. revision: yes

Circularity Check

0 steps flagged

No circularity: results are direct empirical measurements against independent fMRI data

full rationale

The paper reports experimental outcomes from training neural networks under four learning rules and computing Spearman correlations between model representational dissimilarity matrices and separate human fMRI RDMs on the THINGS image set. No derivation chain exists that reduces a claimed result to its own inputs by definition, fitted parameters renamed as predictions, or load-bearing self-citations. The central findings (alignment drops after one epoch, differential effects by rule) are obtained by applying standard RSA to held-out brain data and are therefore self-contained against external benchmarks rather than internally forced.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Empirical study whose central measurements rest on standard domain assumptions in representational similarity analysis and fMRI; no new entities are introduced and no parameters are explicitly fitted to produce the reported deltas.

axioms (1)

domain assumption Spearman correlation of representational dissimilarity matrices computed on the chosen image set and fMRI ROIs accurately captures alignment between model and early visual cortex representations
This is the measurement used to obtain all reported alignment changes in the abstract.

pith-pipeline@v0.9.1-grok · 5775 in / 1443 out tokens · 60751 ms · 2026-06-29T08:43:00.907815+00:00 · methodology

0 comments

read the original abstract

Random, untrained neural networks consistently match or exceed trained networks in representational similarity to early visual cortex. This puzzling finding challenges the assumption that learning improves brain alignment. We investigate it by tracking representational similarity analysis (RSA) alignment to human fMRI data across training for four learning rules: backpropagation (BP), feedback alignment (FA), predictive coding (PC), and spike-timing-dependent plasticity (STDP). Using 720 object images from the THINGS database and fMRI data from three subjects across six visual ROIs, we measure Spearman correlations between model and brain representational dissimilarity matrices at eight training checkpoints (epochs 0-40). We find that (1) a single epoch of training reduces V1 alignment by 25-90%, depending on the learning rule; (2) backpropagation reduces V1 alignment most severely (delta r = -0.080), while predictive coding and STDP preserve substantially more (delta r ~ -0.04); and (3) a weaker, opposite tendency appears in object-selective cortex (LOC), where BP shows the largest increase in alignment during training, although the absolute change is small. These results suggest that untrained architectures capture low-level visual statistics through inductive biases alone, and that global error signals (BP) reshape early representations more aggressively than local learning rules (PC, STDP), which better preserve brain-like structure.

Figures

Figures reproduced from arXiv: 2605.30556 by Nils Leutenegger.

**Figure 1.** Figure 1: V1 brain alignment across training. (A) Spearman r between model RDMs (best layer) and V1 fMRI RDMs across training epochs. Shaded regions: ±1 SD across 5 seeds. Dashed grey line: untrained baseline. (B) Change in alignment relative to epoch 0. BP (blue) drops most steeply; PC (green) and STDP (orange) degrade least. (C) Final alignment at epoch 40. Stars indicate significant difference from untrained base… view at source ↗

**Figure 2.** Figure 2: fMRI alignment across training for all six ROIs. Same conventions as Figure 1A. The degradation pattern is strongest in early visual areas (V1–V3) and absent in higher areas (LOC, IT). untrained networks capture low-level visual statistics through their architectural inductive biases (convolutional filters, pooling, normalisation) rather than through learning. Training then reshapes these representations… view at source ↗

**Figure 3.** Figure 3: Opposing trends in V1 and LOC. (A) V1 alignment decreases during training for all rules. (B) LOC alignment increases for BP only, while local rules (PC, STDP) show no change. Same y-axis scale highlights the magnitude difference. 4.3 Relation to Prior Work Our results complement and extend Leutenegger [2026], which compared trained learning rules at a single endpoint and found that random weights outperfo… view at source ↗

**Figure 4.** Figure 4: Seed variability. (A) V1 alignment at epoch 0 (untrained). All rules cluster near r ≈ 0.10. (B) V1 alignment at epoch 40 (trained). BP and FA collapse to r ≈ 0.02; PC and STDP retain r ≈ 0.06. Box plots with individual seed values (circles). checkpoints, and RSA results at all milestones) will be released upon publication. Acknowledgements This work was conducted independently. Compute was provided by Moda… view at source ↗

discussion (0)

Reference graph

Works this paper leans on

13 extracted references · 1 canonical work pages · 1 internal anchor

[1]

and Poo, M.-m

Bi, G.-q. and Poo, M.-m. (1998). Synaptic modifications inculturedhippocampalneurons: dependenceonspike timing, synaptic strength, and postsynaptic cell type. Journal of Neuroscience, 18(24):10464–10472

1998
[2]

Oliva, A. (2016). Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence. Scientific Reports, 6:27755

2016
[3]

Corriveau, A., Van Wicklin, C., and Baker, C. I. (2019). THINGS: A database of 1,854 object con- cepts and more than 26,000 naturalistic object images. PLOS ONE, 14(10):e0223792

2019
[4]

Pashkam, M., and Baker, C. I. (2023). THINGS-data, a multimodal collection of large-scale datasets for in- vestigating object representations in human brain and behavior.eLife, 12:e82580

2023
[5]

and Kriegeskorte, N

Khaligh-Razavi, S.-M. and Kriegeskorte, N. (2014). Deep supervised, but not unsupervised, models may explain IT cortical representation.PLOS Computational Bi- ology, 10(11):e1003915

2014
[6]

Leutenegger, N. (2026). Untrained CNNs match back- propagation at V1: A systematic RSA comparison of four learning rules against human fMRI.arXiv preprint arXiv:2604.16875

work page internal anchor Pith review Pith/arXiv arXiv 2026
[7]

P., Cownden, D., Tweed, D

Lillicrap, T. P., Cownden, D., Tweed, D. B., and Aker- man, C. J. (2016). Random synaptic feedback weights support error backpropagation for deep learning.Na- ture Communications, 7:13276

2016
[8]

and Thorpe, S

Masquelier, T. and Thorpe, S. J. (2007). Unsuper- vised learning of visual features through spike timing dependent plasticity.PLOS Computational Biology, 3(2):e31

2007
[9]

A., and Naud, R

Payeur, A., Guerguiev, J., Zenke, F., Richards, B. A., and Naud, R. (2021). Burst-dependent synaptic plas- ticity can coordinate learning in hierarchical circuits. Nature Neuroscience, 24(7):1010–1019

2021
[10]

Rao, R. P. and Ballard, D. H. (1999). Predictive cod- ing in the visual cortex: a functional interpretation of some extra-classical receptive-field effects.Nature Neuroscience, 2(1):79–87

1999
[11]

A., Restivo, K., Denfield, G., Tolias, A

Safarani, S., Nix, A., Willeke, K., Cadena, S. A., Restivo, K., Denfield, G., Tolias, A. S., and Sinz, F. H. (2021). 6 Towards robust vision by multi-task learning on mon- key visual cortex.Advances in Neural Information Processing Systems, 34:739–751

2021
[12]

Whittington, J. C. and Bogacz, R. (2017). An approx- imation of the error backpropagation algorithm in a predictive coding network with local Hebbian synap- tic plasticity.Neural Computation, 29(5):1229–1262

2017
[13]

Seibert, D., and DiCarlo, J. J. (2014). Performance- optimized hierarchical models predict neural responses in higher visual cortex.Proceedings of the National Academy of Sciences, 111(23):8619–8624. 7

2014

[1] [1]

and Poo, M.-m

Bi, G.-q. and Poo, M.-m. (1998). Synaptic modifications inculturedhippocampalneurons: dependenceonspike timing, synaptic strength, and postsynaptic cell type. Journal of Neuroscience, 18(24):10464–10472

1998

[2] [2]

Oliva, A. (2016). Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence. Scientific Reports, 6:27755

2016

[3] [3]

Corriveau, A., Van Wicklin, C., and Baker, C. I. (2019). THINGS: A database of 1,854 object con- cepts and more than 26,000 naturalistic object images. PLOS ONE, 14(10):e0223792

2019

[4] [4]

Pashkam, M., and Baker, C. I. (2023). THINGS-data, a multimodal collection of large-scale datasets for in- vestigating object representations in human brain and behavior.eLife, 12:e82580

2023

[5] [5]

and Kriegeskorte, N

Khaligh-Razavi, S.-M. and Kriegeskorte, N. (2014). Deep supervised, but not unsupervised, models may explain IT cortical representation.PLOS Computational Bi- ology, 10(11):e1003915

2014

[6] [6]

Leutenegger, N. (2026). Untrained CNNs match back- propagation at V1: A systematic RSA comparison of four learning rules against human fMRI.arXiv preprint arXiv:2604.16875

work page internal anchor Pith review Pith/arXiv arXiv 2026

[7] [7]

P., Cownden, D., Tweed, D

Lillicrap, T. P., Cownden, D., Tweed, D. B., and Aker- man, C. J. (2016). Random synaptic feedback weights support error backpropagation for deep learning.Na- ture Communications, 7:13276

2016

[8] [8]

and Thorpe, S

Masquelier, T. and Thorpe, S. J. (2007). Unsuper- vised learning of visual features through spike timing dependent plasticity.PLOS Computational Biology, 3(2):e31

2007

[9] [9]

A., and Naud, R

Payeur, A., Guerguiev, J., Zenke, F., Richards, B. A., and Naud, R. (2021). Burst-dependent synaptic plas- ticity can coordinate learning in hierarchical circuits. Nature Neuroscience, 24(7):1010–1019

2021

[10] [10]

Rao, R. P. and Ballard, D. H. (1999). Predictive cod- ing in the visual cortex: a functional interpretation of some extra-classical receptive-field effects.Nature Neuroscience, 2(1):79–87

1999

[11] [11]

A., Restivo, K., Denfield, G., Tolias, A

Safarani, S., Nix, A., Willeke, K., Cadena, S. A., Restivo, K., Denfield, G., Tolias, A. S., and Sinz, F. H. (2021). 6 Towards robust vision by multi-task learning on mon- key visual cortex.Advances in Neural Information Processing Systems, 34:739–751

2021

[12] [12]

Whittington, J. C. and Bogacz, R. (2017). An approx- imation of the error backpropagation algorithm in a predictive coding network with local Hebbian synap- tic plasticity.Neural Computation, 29(5):1229–1262

2017

[13] [13]

Seibert, D., and DiCarlo, J. J. (2014). Performance- optimized hierarchical models predict neural responses in higher visual cortex.Proceedings of the National Academy of Sciences, 111(23):8619–8624. 7

2014