On the Adversarial Robustness of Neural Networks without Weight Transport

Mohamed Akrout

REVIEW 3 major objections 4 minor 9 references

On the Adversarial Robustness of Neural Networks without Weight Transport

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Networks trained without weight transport resist adversarial examples made from their own approximate gradients, but not from true gradients.

desk verdict A clear empirical observation about FA gradients and non-transferability, but the headline robustness claim is an artifact of attacking FA networks with FA's own approximate gradients. read the letter →

arxiv 1908.03560 v2 pith:AHOV2C7M submitted 2019-08-09 cs.LG cs.CRstat.ML

Mohamed Akrout This is my paper · ORCID

classification cs.LGcs.CRstat.ML

keywords feedbackalignmentadversarialrobustnessweighttransportgradient-basedattacksexampletransferabilityMNISTCIFAR-10

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether a biologically motivated alternative to backpropagation, called feedback alignment (FA), changes how easily a neural network is fooled by gradient-based adversarial attacks. The central claim is that on MNIST, an FA-trained LeNet keeps about 97–98% accuracy under FGSM, BIM, and MI-FGSM attacks for $\epsilon \in [0,1]$, while a backprop-trained LeNet falls to about 0.03%. The paper also claims that adversarial examples generated by an FA network do not transfer to a backprop network, whereas backprop-generated examples do fool the FA network. On CIFAR-10, the gap in accuracy shrinks but remains noticeable for small perturbations, and FA-to-backprop transfer becomes significant at larger perturbation magnitudes.

What carries the argument

The central object is the fixed random feedback matrix $B$ that replaces the transposed forward weight matrix $W^T$ in the error-propagation equation, a mechanism called feedback alignment. This matrix makes the weight-update gradients only loosely aligned with true backpropagation gradients—the paper cites evidence that the angle between FA and BP updates stays above 80 degrees in deep architectures. The attack machinery consists of gradient-sign methods (FGSM, BIM, MI-FGSM) applied through those same approximate gradients, so the key mechanism is the mismatch between the gradient the attacker uses and the gradient that would actually maximize the loss.

What would settle it

Run FGSM and BIM against an FA-trained LeNet on MNIST using the exact input gradient, computed as in backpropagation, and measure accuracy across perturbation magnitudes from 0 to 1; if the accuracy falls toward zero, the claimed robustness is an artifact of which gradient the attacker is allowed to use.

Watch

Extended reading notes

Core claim

The paper's discovery is that the training rule shapes the attack surface: adversarial examples are built from a gradient, and feedback alignment's random fixed feedback matrices make that gradient a poor instrument for fooling the network that produced it. According to the paper, an FA-trained LeNet on MNIST stays near 97–98% accuracy under fast gradient sign, basic iterative, and momentum iterative attacks for perturbation magnitudes up to 1, while a BP-trained LeNet collapses to near 0.03%. The paper also finds that FA-generated adversarial examples are not transferable to BP networks on MNIST, but BP-generated examples are transferable to FA networks, and that this asymmetry weakens on CIFAR-10.

Load-bearing premise

The load-bearing premise is that the attacker is limited to the network's own approximate feedback-alignment gradients; if the attacker uses the true input gradient instead, the paper's own transfer experiment shows the claimed protection disappears.

Editorial extensions

If this is right

On MNIST, an FA-trained LeNet maintains about 97–98% accuracy against FGSM, BIM, and MI-FGSM for $\epsilon$ up to 1, while a BP-trained LeNet drops to about 0.03%.
Adversarial examples generated with FA gradients do not fool a BP network on MNIST, meaning they are not transferable in that direction.
Adversarial examples generated with BP gradients do fool FA networks, so FA's protection is not a general immunity to adversarial examples.
On CIFAR-10, FA networks still lose accuracy more slowly than BP networks under gradient attacks, especially for perturbation magnitudes below 1/2, but the transferability gap closes as the perturbation grows.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

An attacker who estimates the true input gradient, for example through black-box queries, should be able to fool FA networks on MNIST even though FA-gradient attacks fail; the paper's BP-to-FA transfer experiment already demonstrates that exact-gradient examples work.
The apparent defense may function as a form of gradient obfuscation: FA's random feedback path makes the model's own gradient a poor attack direction, but the underlying decision boundary remains vulnerable to directions discovered by other means.
If confirmed across more datasets, this result would imply that biologically motivated learning rules carry an unintended security property, and that the choice of training algorithm, not just architecture, shapes adversarial vulnerability.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things to know. The paper is the first to run FGSM/BIM/MI-FGSM against feedback-alignment-trained networks, and it reports a real, narrow phenomenon: adversarial examples generated through the FA feedback path do fool FA networks on MNIST, and those examples don't transfer to BP networks. That part is reproducible in the narrow sense and worth a footnote.

The broad claim, however, doesn't hold. The authors call it "actual robustness," but the protocol attacks FA networks with gradients computed through the same random feedback matrices used during training, while BP networks are attacked with exact gradients. That is not a white-box comparison under any standard threat model. An adversary with access to the trained FA network can compute the true input gradient by ordinary backpropagation through the forward graph—training rule doesn't matter. And the paper's own transfer experiment (BP→FA) shows exactly that: exact-gradient adversarial examples fool the FA network. So the FA network is not robust to standard white-box attacks; it is robust only to attacks that use FA's poor gradient estimates. That is closer to gradient masking than to "actual robustness." The MNIST accuracy gap is therefore forced by the asymmetric protocol, not discovered.

The CIFAR-10 results weaken the story further: the gap shrinks, and FA→BP transfer attacks succeed at larger epsilons. The paper also reports no seeds, no error bars, no code, and no training details, so the main quantitative claims (98% vs 0.03%) are hard to verify. At 10 pages it reads like a workshop note, not a full paper.

Credit where due: the non-transferability observation is novel and could be a useful data point for the adversarial robustness literature, especially as a caution about gradient-based attack assumptions. The writing is clear, and the authors correctly cite [7] showing FA gradients are misaligned with BP gradients; they just don't follow that fact to its logical conclusion about their own threat model.

Who is this for? A reader interested in biologically plausible learning and adversarial robustness might take the narrow result as motivation for studying defense-aware threat models. A reader looking for evidence that FA training confers intrinsic robustness should not rely on this paper. It deserves a referee—the claims are checkable and the flaw is fixable—but it should not be accepted in its current form. The authors should re-cast the result as "FA networks resist attacks that use FA estimates" or add the true-gradient white-box baseline and report the actual (asymmetric) robustness honestly.

Recommendation: send to peer review only if the authors commit to the corrected protocol; as is, reject.

Referee Report

3 major / 4 minor

Summary. The paper tests whether convolutional networks (LeNet) trained with feedback alignment (FA), a biologically motivated alternative to backpropagation (BP) that avoids weight transport, are less vulnerable to gradient-based adversarial attacks. Using FGSM, BIM, and MI-FGSM on MNIST and CIFAR-10, the authors report that FA-trained MNIST networks keep about 97% adversarial accuracy across perturbation magnitudes epsilon in [0,1], while BP-trained networks drop to near 0%. They also report that adversarial examples generated on FA networks do not transfer to BP networks, while BP-generated examples do fool FA networks. The paper interprets these results as evidence of 'actual robustness' from removing weight transport.

Significance. If the headline finding were correct, it would be a surprising and consequential result: a biologically plausible learning rule would confer intrinsic adversarial robustness, with implications for both neuroscience and robust machine learning. The paper has clear strengths: it addresses a novel question, compares three standard attacks on a fixed architecture, includes a transferability analysis, and checks the effect on a second dataset. These are appropriate empirical ingredients. However, the central claim as stated does not survive scrutiny of the attack protocol: FA networks are attacked with FA's approximate gradients while BP networks are attacked with exact gradients, and the paper's own BP-to-FA transfer result shows that exact-gradient attacks fool the FA network. The reported robustness is therefore an artifact of the chosen threat model, not a property of the trained model.

major comments (3)

[§3.1, Eq. (3), Fig. 1] The main comparison is not apples-to-apples. Section 3.1 explicitly permits the FGSM perturbation to be computed either 'through transposed forward-path synaptic weights like in backpropagation or through random synaptic weights like in feedback alignment,' and the results in Figure 1a can only be reproduced if FA networks are attacked with gradients computed through the random FA feedback matrices B, while BP networks are attacked with exact backpropagation gradients. Under the standard white-box threat model, an adversary with access to the trained network can compute the exact input gradient nabla_x J by ordinary backpropagation through the forward computation graph, regardless of how the network was trained. The paper's own BP-to-FA transfer result in Figure 1b shows that such exact-gradient adversarial examples do fool the FA network, so the FA network is not robust in the standard threat model.
[§4.1, §5] The headline claim of 'actual robustness' is therefore unsupported. The high accuracy of FA networks in Figure 1a is a consequence of attacking them with inaccurate gradients: the author's own reference [7] reports that FA weight updates are mostly misaligned with true gradients (angles >80 degrees for deep residual networks), so a gradient computed through B is a poor estimate of the loss gradient. This pattern, in which apparent robustness disappears when exact gradients are used, is the standard signature of gradient masking or obfuscated gradients rather than intrinsic robustness. Section 5's conclusion (1) should be restated as 'FA networks are robust to adversarial examples generated with FA gradients,' which is a much weaker and less surprising claim.
[§4.2, Fig. 2] The CIFAR-10 results further weaken the robustness claim. Figure 2a shows the accuracy of FA networks decreasing with epsilon for all three attacks, albeit more slowly than for BP networks, so the phenomenon is not a general property of FA training. The abstract's qualification 'still significant particularly for small perturbations of magnitude less than 1/2' is welcome, but it cannot repair the MNIST protocol problem. A proper test would attack the FA networks with exact input gradients and report those accuracy curves alongside the FA-gradient curves; without that, the central comparative claim is not established.

minor comments (4)

[§1] The phrase 'hardly perceptible perturbation' should be made grammatical, for example 'hardly perceptible perturbations' or 'hardly perceptible perturbation magnitudes.'
[Abstract, §4.1] The abstract states an adversarial accuracy of 98% for FA networks on MNIST, while §4.1 says the accuracy 'stays around 97%.' Please clarify whether 98% is the clean accuracy or the accuracy at the smallest epsilon, and report the exact values in Figure 1a.
[§4] The experiments section does not specify training hyperparameters (learning rate, number of epochs, batch size, FA feedback matrix initialization) or the BIM step size alpha; these details are needed for reproducibility.
[§4.1, Figs. 1 and 2] The notation 'BP→FA' and 'FA→BP' is defined in the figure captions but used before the definition in the main text; consider defining it explicitly in a sentence near the first mention.

Circularity Check

1 steps flagged · score 6.0 of 10

The MNIST 'actual robustness' result is obtained by attacking FA networks with FA's own approximate gradients; BP-to-FA transfer shows exact-gradient attacks fool them, so the headline claim is largely built into the attack definition.

self definitional [Section 3.1 (Equation 3) and Section 4.1 (Figure 1a and 1b)]
"This perturbation can be computed through transposed forward-path synaptic weights like in backpropagation or through random synaptic weights like in feedback alignment. ... We find that when performing the three gradient-based adversarial attacks (FGSM, BIM and MI-FGSM) on a FA neural network, the accuracy does not decrease and stays around 97%. ... The mutual conclusion is not true since adversarial examples generated by the BP network can fool the FA network."

The paper's central robustness claim is produced by attacking FA networks with gradients computed through FA's random feedback matrices, i.e. with the very approximate gradient whose failure is the stated motivation. Under the standard white-box threat model, an adversary with access to the trained network can compute the exact input gradient by ordinary backpropagation through the forward computation graph, regardless of the training rule. The paper's own BP-to-FA transfer result (Figure 1b) shows that such exact-gradient adversarial examples do fool the FA network. Therefore the reported 'actual robustness' is not an intrinsic property of the FA network but follows from defining the attack gradient to be the FA gradient for FA networks while using the exact gradient for BP networks.

full rationale

The paper is not circular in the sense of deriving a quantity from itself through a mathematical equivalence; it reports an empirical comparison with an explicitly stated attack protocol. However, the headline claim of 'actual robustness' rests on a definitional choice that heavily favors the FA network. Section 3.1 explicitly allows the FGSM perturbation to be computed either through the true backpropagation gradient or through FA's random feedback matrices. The MNIST result that FA networks stay near 97% accuracy is meaningful only under the FA-gradient attack; the same section's transfer experiment admits that BP-generated adversarial examples fool FA networks, which demonstrates that FA networks are not robust to the standard white-box attack using exact gradients. Thus the main quantitative result reduces, to a substantial degree, to the choice of threat model: the paper attacks FA networks with an inaccurate gradient estimator and then presents the resulting lack of fooling as evidence of robustness. This is a partial circularity in the evaluation protocol rather than a self-citation chain or a fitted prediction. The self-citation [7] about gradient angles is used only as motivating background and is not the load-bearing step that forces the experimental conclusion. Overall, the central claim is overstated and partially built into the attack definition, but the paper does contain an honest empirical observation about FA-gradient attacks and their non-transferability.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central result rests on a protocol choice: attacking FA networks with FA's random feedback gradients rather than the exact input gradient. It also relies on the background assumption that FA networks are otherwise usable classifiers. No new physical or theoretical entities are introduced, and no model parameters are fitted to data beyond hand-chosen attack settings.

free parameters (2)

epsilon sweep range = 0 to 1 in steps of 0.1
Chosen by hand; the headline MNIST robustness and the CIFAR-10 'less than 1/2' claim are stated relative to this range.
BIM and MI-FGSM attack settings = n=10 iterations, mu=0.8
Chosen by hand; they determine attack strength and thus the reported accuracy curves.

assumptions (3)

domain assumption FA-trained LeNet achieves test accuracy comparable to BP-trained LeNet on the datasets used.
The paper does not report clean test accuracy; the robustness comparison is only meaningful if FA networks are otherwise usable classifiers.
ad hoc to paper Attacks on FA networks may use the random feedback matrices B instead of the exact input gradient.
Section 3.1 permits either perturbation path; the main robustness curves for FA networks appear to use the FA path, and this choice produces the high accuracy numbers.
domain assumption The angle between FA and BP weight updates is larger than 80 degrees on ResNet-18/50, as cited in [7], so FA gradients are poor approximations of true gradients.
Section 2 uses this cited result to motivate the hypothesis that FA networks are harder to attack.

how reviews work

0 comments

Cite this review

Pith. "Pith review of On the Adversarial Robustness of Neural Networks without Weight Transport." pith.science (2026). https://pith.science/paper/AHOV2C7M

@misc{pith2026190803560,
  author       = {Pith},
  title        = {Pith review of: On the Adversarial Robustness of Neural Networks without Weight Transport},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AHOV2C7M}},
  note         = {Machine review of arXiv:1908.03560}
}

read the original abstract

Neural networks trained with backpropagation, the standard algorithm of deep learning which uses weight transport, are easily fooled by existing gradient-based adversarial attacks. This class of attacks are based on certain small perturbations of the inputs to make networks misclassify them. We show that less biologically implausible deep neural networks trained with feedback alignment, which do not use weight transport, can be harder to fool, providing actual robustness. Tested on MNIST, deep neural networks trained without weight transport (1) have an adversarial accuracy of 98% compared to 0.03% for neural networks trained with backpropagation and (2) generate non-transferable adversarial examples. However, this gap decreases on CIFAR-10 but is still significant particularly for small perturbation magnitude less than 1/2.

Figures

Figures reproduced from arXiv: 1908.03560 by the authors.

**Figure 2.** Results on CIFAR-10 (a) Adversarial accuracy against three attacks (FGSM, BIM and MI-FGSM) on CIFAR-10 for FA and BP [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

9 extracted references · 4 canonical work pages

[7]

Deep learning without weight transport

Mohamed Akrout, Collin Wilson, Peter C Humphreys, Timothy P Lillicrap, and Douglas B Tweed. Deep learning without weight transport. CoRR, abs/1904.05391, 2019

arXiv 1904
[1]

Intriguing properties of neural networks

Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013

arXiv 2013
[2]

Random synaptic feedback weights support error backpropagation for deep learning

Timothy P Lillicrap, Daniel Cownden, Douglas B Tweed, and Colin J Akerman. Random synaptic feedback weights support error backpropagation for deep learning. Nature Communications, 7:13276, 2016

work page 2016
[3]

Explaining and harnessing adversarial examples

Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014

arXiv 2014
[4]

Boosting adversarial attacks with momentum

Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial attacks with momentum. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 9185–9193, 2018

work page 2018
[5]

Competitive learning: From interactive activation to adaptive resonance

Stephen Grossberg. Competitive learning: From interactive activation to adaptive resonance. Cognitive Science, 11(1):23–63, 1987

work page 1987
[6]

The recent excitement about neural networks

Francis Crick. The recent excitement about neural networks. Nature, 337(6203):129–132, 1989

1989
[8]

Adversarial machine learning at scale

Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale. arXiv preprint arXiv:1611.01236, 2016

arXiv 2016

Show all 9 references

[9]

Gradient-based learning applied to document recognition

Yann LeCun, Léon Bottou, Yoshua Bengio, Patrick Haffner, et al. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998. 4

1998

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [7]

Deep learning without weight transport

Mohamed Akrout, Collin Wilson, Peter C Humphreys, Timothy P Lillicrap, and Douglas B Tweed. Deep learning without weight transport. CoRR, abs/1904.05391, 2019

arXiv 1904

[2] [1]

Intriguing properties of neural networks

Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013

arXiv 2013

[3] [2]

Random synaptic feedback weights support error backpropagation for deep learning

Timothy P Lillicrap, Daniel Cownden, Douglas B Tweed, and Colin J Akerman. Random synaptic feedback weights support error backpropagation for deep learning. Nature Communications, 7:13276, 2016

work page 2016

[4] [3]

Explaining and harnessing adversarial examples

Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014

arXiv 2014

[5] [4]

Boosting adversarial attacks with momentum

Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial attacks with momentum. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 9185–9193, 2018

work page 2018

[6] [5]

Competitive learning: From interactive activation to adaptive resonance

Stephen Grossberg. Competitive learning: From interactive activation to adaptive resonance. Cognitive Science, 11(1):23–63, 1987

work page 1987

[7] [6]

The recent excitement about neural networks

Francis Crick. The recent excitement about neural networks. Nature, 337(6203):129–132, 1989

1989

[8] [8]

Adversarial machine learning at scale

Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale. arXiv preprint arXiv:1611.01236, 2016

arXiv 2016