{"id":"3f6f7643-0b41-4d55-921a-cc384600ad28","arxiv_id":"1908.03560","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"On MNIST, networks trained with feedback alignment resist adversarial examples generated with feedback-alignment gradients, while backprop-trained networks collapse to near-zero accuracy; the effect weakens on CIFAR-10.","lead":"This paper tests whether neural networks trained with feedback alignment, a learning rule that avoids copying forward weights for error propagation, are harder to fool with gradient-based adversarial attacks. It reports near-perfect adversarial accuracy on MNIST for these networks and non-transferable attacks, but the effect shrinks on CIFAR-10.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'actual robustness' claim rests on attacking FA networks with FA's own approximate gradients; the paper's BP-to-FA transfer results show exact-gradient attacks fool them, so the central claim depends on a nonstandard threat model.","rationale":"The reader's weakest assumption correctly identifies that the main comparison uses FA gradients for FA networks and BP gradients for BP networks, while the paper's transfer result shows exact-gradient examples fool FA networks. My analysis agrees: the load-bearing issue is the threat model. The paper's Section 3.1 explicitly allows both gradient paths, and the empirical protocol appears to select the FA path for attacking FA networks. Under the standard white-box threat model, an attacker can always compute the true input gradient, so the reported 98% MNIST robustness is an artifact of attacking with a deliberately wrong gradient. The transfer result BP→FA in Figure 1b is direct evidence: examples generated with exact gradients transfer to the FA network, meaning the FA network is foolable when the attacker uses the correct gradient. This is a correctness risk, not merely a disagreement with consensus; it is internally inconsistent to claim 'actual robustness' while showing that exact-gradient attacks succeed. The paper's secondary observation about FA→BP non-transferability remains interesting and potentially reproducible, and the CIFAR-10 results are more cautious, but the abstract's unqualified robustness claim is not supported. No additional independent test is needed beyond the exact-gradient attack to settle the matter, though error bars and training details would also improve reproducibility.","tokens_in":3496,"tokens_out":2225,"duration_ms":26547,"concrete_test":"Retrain or reuse the FA-trained LeNet on MNIST and run a standard white-box PGD/FGSM attack using the exact input gradient ∇_x J computed by backpropagation through the network's forward graph, with the same epsilon values [0, 1] and the same number of iterations as in the paper. If the FA network's accuracy drops to near zero (consistent with the paper's BP→FA transfer results), the headline robustness claim fails. For completeness, also report the same attack on the BP-trained network and include error bars over multiple seeds.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central robustness claim is not established under the standard white-box threat model. Section 3.1 explicitly permits the FGSM perturbation to be computed 'through transposed forward-path synaptic weights like in backpropagation or through random synaptic weights like in feedback alignment.' The MNIST results in Figure 1a, which report FA accuracy near 97%, are obtained by attacking FA networks with gradients computed through the FA feedback path. But an adversary with access to the trained network can compute the true input gradient ∇_x J by ordinary backpropagation through the forward computation graph, regardless of how the network was trained. The paper's own transfer experiment (Figure 1b, BP→FA) shows that adversarial examples generated with exact BP gradients do fool the FA network. Therefore, the FA network is not robust to the standard white-box attack; it is only robust to a weaker attack that uses an inaccurate gradient estimator. The observed FA→BP non-transferability is better described as gradient masking or obfuscated gradients than as 'actual robustness.' The robustness claim as stated in the abstract and strongest result is therefore unsupported, and the reported accuracy gap depends on an asymmetric attack protocol rather than on intrinsic robustness.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper tests whether convolutional networks (LeNet) trained with feedback alignment (FA), a biologically motivated alternative to backpropagation (BP) that avoids weight transport, are less vulnerable to gradient-based adversarial attacks. Using FGSM, BIM, and MI-FGSM on MNIST and CIFAR-10, the authors report that FA-trained MNIST networks keep about 97% adversarial accuracy across perturbation magnitudes epsilon in [0,1], while BP-trained networks drop to near 0%. They also report that adversarial examples generated on FA networks do not transfer to BP networks, while BP-generated examples do fool FA networks. The paper interprets these results as evidence of 'actual robustness' from removing weight transport.","tokens_in":3719,"tokens_out":4536,"duration_ms":48028,"significance":"If the headline finding were correct, it would be a surprising and consequential result: a biologically plausible learning rule would confer intrinsic adversarial robustness, with implications for both neuroscience and robust machine learning. The paper has clear strengths: it addresses a novel question, compares three standard attacks on a fixed architecture, includes a transferability analysis, and checks the effect on a second dataset. These are appropriate empirical ingredients. However, the central claim as stated does not survive scrutiny of the attack protocol: FA networks are attacked with FA's approximate gradients while BP networks are attacked with exact gradients, and the paper's own BP-to-FA transfer result shows that exact-gradient attacks fool the FA network. The reported robustness is therefore an artifact of the chosen threat model, not a property of the trained model.","major_comments":[{"comment":"The main comparison is not apples-to-apples. Section 3.1 explicitly permits the FGSM perturbation to be computed either 'through transposed forward-path synaptic weights like in backpropagation or through random synaptic weights like in feedback alignment,' and the results in Figure 1a can only be reproduced if FA networks are attacked with gradients computed through the random FA feedback matrices B, while BP networks are attacked with exact backpropagation gradients. Under the standard white-box threat model, an adversary with access to the trained network can compute the exact input gradient nabla_x J by ordinary backpropagation through the forward computation graph, regardless of how the network was trained. The paper's own BP-to-FA transfer result in Figure 1b shows that such exact-gradient adversarial examples do fool the FA network, so the FA network is not robust in the standard threat model.","section":"§3.1, Eq. (3), Fig. 1"},{"comment":"The headline claim of 'actual robustness' is therefore unsupported. The high accuracy of FA networks in Figure 1a is a consequence of attacking them with inaccurate gradients: the author's own reference [7] reports that FA weight updates are mostly misaligned with true gradients (angles >80 degrees for deep residual networks), so a gradient computed through B is a poor estimate of the loss gradient. This pattern, in which apparent robustness disappears when exact gradients are used, is the standard signature of gradient masking or obfuscated gradients rather than intrinsic robustness. Section 5's conclusion (1) should be restated as 'FA networks are robust to adversarial examples generated with FA gradients,' which is a much weaker and less surprising claim.","section":"§4.1, §5"},{"comment":"The CIFAR-10 results further weaken the robustness claim. Figure 2a shows the accuracy of FA networks decreasing with epsilon for all three attacks, albeit more slowly than for BP networks, so the phenomenon is not a general property of FA training. The abstract's qualification 'still significant particularly for small perturbations of magnitude less than 1/2' is welcome, but it cannot repair the MNIST protocol problem. A proper test would attack the FA networks with exact input gradients and report those accuracy curves alongside the FA-gradient curves; without that, the central comparative claim is not established.","section":"§4.2, Fig. 2"}],"minor_comments":[{"comment":"The phrase 'hardly perceptible perturbation' should be made grammatical, for example 'hardly perceptible perturbations' or 'hardly perceptible perturbation magnitudes.'","section":"§1"},{"comment":"The abstract states an adversarial accuracy of 98% for FA networks on MNIST, while §4.1 says the accuracy 'stays around 97%.' Please clarify whether 98% is the clean accuracy or the accuracy at the smallest epsilon, and report the exact values in Figure 1a.","section":"Abstract, §4.1"},{"comment":"The experiments section does not specify training hyperparameters (learning rate, number of epochs, batch size, FA feedback matrix initialization) or the BIM step size alpha; these details are needed for reproducibility.","section":"§4"},{"comment":"The notation 'BP→FA' and 'FA→BP' is defined in the figure captions but used before the definition in the main text; consider defining it explicitly in a sentence near the first mention.","section":"§4.1, Figs. 1 and 2"}],"recommendation":"reject","confidential_remarks":"The central issue is not a presentation problem; the main experimental protocol contradicts the threat model implied by the abstract. Because the paper's own BP-to-FA transfer experiment provides a clean falsification of the 'actual robustness' claim, I do not see a path to acceptance without redefining the contribution. Should the authors resubmit a version that attacks FA networks with exact gradients and/or explicitly frames the contribution as robustness against FA-gradient attacks, the transfer and CIFAR analyses would be a useful empirical basis for a different paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. The paper is the first to run FGSM/BIM/MI-FGSM against feedback-alignment-trained networks, and it reports a real, narrow phenomenon: adversarial examples generated through the FA feedback path do fool FA networks on MNIST, and those examples don't transfer to BP networks. That part is reproducible in the narrow sense and worth a footnote.\n\nThe broad claim, however, doesn't hold. The authors call it \"actual robustness,\" but the protocol attacks FA networks with gradients computed through the same random feedback matrices used during training, while BP networks are attacked with exact gradients. That is not a white-box comparison under any standard threat model. An adversary with access to the trained FA network can compute the true input gradient by ordinary backpropagation through the forward graph—training rule doesn't matter. And the paper's own transfer experiment (BP→FA) shows exactly that: exact-gradient adversarial examples fool the FA network. So the FA network is not robust to standard white-box attacks; it is robust only to attacks that use FA's poor gradient estimates. That is closer to gradient masking than to \"actual robustness.\" The MNIST accuracy gap is therefore forced by the asymmetric protocol, not discovered.\n\nThe CIFAR-10 results weaken the story further: the gap shrinks, and FA→BP transfer attacks succeed at larger epsilons. The paper also reports no seeds, no error bars, no code, and no training details, so the main quantitative claims (98% vs 0.03%) are hard to verify. At 10 pages it reads like a workshop note, not a full paper.\n\nCredit where due: the non-transferability observation is novel and could be a useful data point for the adversarial robustness literature, especially as a caution about gradient-based attack assumptions. The writing is clear, and the authors correctly cite [7] showing FA gradients are misaligned with BP gradients; they just don't follow that fact to its logical conclusion about their own threat model.\n\nWho is this for? A reader interested in biologically plausible learning and adversarial robustness might take the narrow result as motivation for studying defense-aware threat models. A reader looking for evidence that FA training confers intrinsic robustness should not rely on this paper. It deserves a referee—the claims are checkable and the flaw is fixable—but it should not be accepted in its current form. The authors should re-cast the result as \"FA networks resist attacks that use FA estimates\" or add the true-gradient white-box baseline and report the actual (asymmetric) robustness honestly.\n\nRecommendation: send to peer review only if the authors commit to the corrected protocol; as is, reject.","headline":"A clear empirical observation about FA gradients and non-transferability, but the headline robustness claim is an artifact of attacking FA networks with FA's own approximate gradients.","tokens_in":4239,"tokens_out":2378,"would_cite":false,"duration_ms":23831,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Networks trained without weight transport resist adversarial examples made from their own approximate gradients, but not from true gradients.","keywords":["feedback alignment","adversarial robustness","weight transport","gradient-based attacks","adversarial example transferability","MNIST","CIFAR-10"],"falsifier":"Run FGSM and BIM against an FA-trained LeNet on MNIST using the exact input gradient, computed as in backpropagation, and measure accuracy across perturbation magnitudes from 0 to 1; if the accuracy falls toward zero, the claimed robustness is an artifact of which gradient the attacker is allowed to use.","tokens_in":3266,"feed_emoji":"🛡️","tokens_out":6695,"duration_ms":65904,"temperature":0.7,"pith_summary":"This paper asks whether a biologically motivated alternative to backpropagation, called feedback alignment (FA), changes how easily a neural network is fooled by gradient-based adversarial attacks. The central claim is that on MNIST, an FA-trained LeNet keeps about 97–98% accuracy under FGSM, BIM, and MI-FGSM attacks for $\\epsilon \\in [0,1]$, while a backprop-trained LeNet falls to about 0.03%. The paper also claims that adversarial examples generated by an FA network do not transfer to a backprop network, whereas backprop-generated examples do fool the FA network. On CIFAR-10, the gap in accuracy shrinks but remains noticeable for small perturbations, and FA-to-backprop transfer becomes significant at larger perturbation magnitudes.","feed_headline":"Feedback-aligned nets keep 98% accuracy under gradient attacks","feed_subtitle":"Backprop nets fall to 0.03% on the same MNIST attacks; the defense depends on which gradient the attacker uses.","key_machinery":"The central object is the fixed random feedback matrix $B$ that replaces the transposed forward weight matrix $W^T$ in the error-propagation equation, a mechanism called feedback alignment. This matrix makes the weight-update gradients only loosely aligned with true backpropagation gradients—the paper cites evidence that the angle between FA and BP updates stays above 80 degrees in deep architectures. The attack machinery consists of gradient-sign methods (FGSM, BIM, MI-FGSM) applied through those same approximate gradients, so the key mechanism is the mismatch between the gradient the attacker uses and the gradient that would actually maximize the loss.","core_discovery":"The paper's discovery is that the training rule shapes the attack surface: adversarial examples are built from a gradient, and feedback alignment's random fixed feedback matrices make that gradient a poor instrument for fooling the network that produced it. According to the paper, an FA-trained LeNet on MNIST stays near 97–98% accuracy under fast gradient sign, basic iterative, and momentum iterative attacks for perturbation magnitudes up to 1, while a BP-trained LeNet collapses to near 0.03%. The paper also finds that FA-generated adversarial examples are not transferable to BP networks on MNIST, but BP-generated examples are transferable to FA networks, and that this asymmetry weakens on CIFAR-10.","pith_inferences":["An attacker who estimates the true input gradient, for example through black-box queries, should be able to fool FA networks on MNIST even though FA-gradient attacks fail; the paper's BP-to-FA transfer experiment already demonstrates that exact-gradient examples work.","The apparent defense may function as a form of gradient obfuscation: FA's random feedback path makes the model's own gradient a poor attack direction, but the underlying decision boundary remains vulnerable to directions discovered by other means.","If confirmed across more datasets, this result would imply that biologically motivated learning rules carry an unintended security property, and that the choice of training algorithm, not just architecture, shapes adversarial vulnerability."],"forward_implications":["On MNIST, an FA-trained LeNet maintains about 97–98% accuracy against FGSM, BIM, and MI-FGSM for $\\epsilon$ up to 1, while a BP-trained LeNet drops to about 0.03%.","Adversarial examples generated with FA gradients do not fool a BP network on MNIST, meaning they are not transferable in that direction.","Adversarial examples generated with BP gradients do fool FA networks, so FA's protection is not a general immunity to adversarial examples.","On CIFAR-10, FA networks still lose accuracy more slowly than BP networks under gradient attacks, especially for perturbation magnitudes below 1/2, but the transferability gap closes as the perturbation grows."],"supporting_citations":[{"why":"Introduces feedback alignment with random fixed feedback matrices $B$, the training mechanism whose robustness is being tested.","marker":"[2]"},{"why":"Defines the Fast Gradient Sign Method, the one-step gradient-sign attack used in all experiments.","marker":"[3]"},{"why":"Defines the Momentum Iterative Fast Gradient Sign Method, the momentum-based attack used in the transfer experiments.","marker":"[4]"},{"why":"Reports that FA weight updates remain more than 80 degrees away from BP gradients, motivating the paper's robustness hypothesis.","marker":"[7]"},{"why":"Defines the Basic Iterative Method, the iterative attack used alongside FGSM.","marker":"[8]"},{"why":"Supplies the LeNet architecture used for all MNIST and CIFAR-10 experiments.","marker":"[9]"}],"fun_headline_variants":["Feedback-aligned nets resist gradient attacks that break backprop","FA nets: 98% accuracy under attacks, BP nets: 0.03%","Why training without weight transport beats backprop at robustness","Non-transferable adversarial examples: FA nets stay robust","Gradient attacks fail on feedback-aligned nets"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the attacker is limited to the network's own approximate feedback-alignment gradients; if the attacker uses the true input gradient instead, the paper's own transfer experiment shows the claimed protection disappears.","fun_headline_variants_meta":{"raw":{"variants":["Feedback-aligned nets resist gradient attacks that break backprop","FA nets: 98% accuracy under attacks, BP nets: 0.03%","Why training without weight transport beats backprop at robustness","Non-transferable adversarial examples: FA nets stay robust","Gradient attacks fail on feedback-aligned nets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000978,"raw_usage":{"total_tokens":4091,"prompt_tokens":822,"completion_tokens":3269,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":438,"completion_tokens_details":{"reasoning_tokens":3156}},"tokens_in":438,"tokens_out":3269,"duration_ms":23921,"temperature":1.0,"reasoning_tokens":3156,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T14:09:45.359996+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run FGSM and BIM against an FA-trained LeNet on MNIST using the exact input gradient, computed as in backpropagation, and measure accuracy across perturbation magnitudes from 0 to 1; if the accuracy falls toward zero, the claimed robustness is an artifact of which gradient the attacker is allowed to use.","supporting_citations":[{"cited_title":"Random synaptic feedback weights support error backpropagation for deep learning","cited_arxiv_id":null,"evidence_quote":"Introduces feedback alignment with random fixed feedback matrices $B$, the training mechanism whose robustness is being tested."},{"cited_title":"Boosting adversarial attacks with momentum","cited_arxiv_id":null,"evidence_quote":"Defines the Momentum Iterative Fast Gradient Sign Method, the momentum-based attack used in the transfer experiments."},{"cited_title":"Gradient-based learning applied to document recognition","cited_arxiv_id":null,"evidence_quote":"Supplies the LeNet architecture used for all MNIST and CIFAR-10 experiments."}],"review_version":1}