REVIEW 4 major objections 4 minor 50 references
Sign-Symmetry Learning Rules are Robust Fine-Tuners
T0 review · 4 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read Fine-tuning with Sign-Symmetry learning rules keeps backpropagation-level performance while making models markedly more resistant to gradient-based adversarial attacks.
desk verdict Robustness claim is likely a gradient-masking artifact; performance parity is unverified, but the fine-tuning recipe is a legitimate, testable idea. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the Sign-Symmetry feedback matrix $V$ that substitutes for the transposed weight matrix in the backward pass: for uSF, $V = \operatorname{sign}(W^\top)$; for frSF, $V = M \circ \operatorname{sign}(W^\top)$ with $M$ fixed random magnitudes; for brSF, $M$ is redrawn every update. This changes the error signal $\delta_l = (V_{l+1}\delta_{l+1}) \circ f'(a_l)$ that propagates through the network during fine-tuning. The sign alignment keeps updates correlated with true gradient descent—enough to fine-tune effectively—but the magnitudes are no longer exact, so the model's forward gradient no longer points an adversary straight to a loss-increasing perturbation. That mismatch is the mechanism the paper credits for the robustness.
What would settle it
Run a much stronger white-box attack on the Sign-Symmetry fine-tuned models—for example, PGD with fifty iterations and several random restarts, or an attack that approximates the true gradient by straight-through estimation of the sign function—and check whether the robust accuracy at $\epsilon=0.1$ collapses to the same near-zero level as backpropagation fine-tuning. If it does, the robustness is gradient masking, not a property of the decision boundary.
Extended reading notes
Core claim
The central discovery is that the weakness of bio-plausible learning rules—imprecise gradients—flips into an advantage once a model is already well initialized. Backpropagation pre-training supplies good representations; fine-tuning with Sign-Symmetry rules, where the backward matrix is the sign of the forward weight matrix (uSF), optionally with fixed or redrawn random magnitudes (frSF, brSF), continues to adapt the weights well enough to match backpropagation's task performance while resisting white-box adversarial perturbation. In the reported experiments, the accuracy gap between backpropagation and Sign-Symmetry fine-tuned models under FGSM and PGD can exceed 50 percentage points at large perturbation sizes, and the mean average precision gap under hashing attacks can reach tens of points, while black-box attacks show no consistent advantage for either family. The paper interprets this as a specificity: sign-symmetry fine-tuning hardens models against the very attacks that exploit gradient access.
Load-bearing premise
The load-bearing premise is that the apparent white-box robustness is genuine model robustness rather than obfuscated gradients; this enters at Section 6.2.1, where FGSM and PGD are the sole evidence, and it is undercut by the paper's own black-box tables (Tables 3-4) showing no advantage, the pattern the cited obfuscated-gradients literature warns about.
Editorial extensions
If this is right
- Sign-Symmetry fine-tuning can act as a drop-in replacement for backpropagation fine-tuning in pre-trained pipelines, preserving task performance while improving resistance to white-box gradient attacks.
- The robustness effect transfers across tasks, appearing in both image classification and hashing-based retrieval, which suggests it is a property of the learning rule rather than of one loss function or architecture.
- Among the variants, frSF is the most consistently performant and stable choice for fine-tuning.
- Because black-box attacks do not distinguish the methods, the advantage is specific to adversaries with gradient access, meaning the method hardens a particular attack surface without claiming global robustness.
Reading between the lines
- We read the absence of black-box advantage as the key unresolved point: if the robustness were due to a genuinely flatter loss landscape, some transfer-based or decision-based attacks would likely still show an edge; the authors attribute the parity to the attack being independent of the learning rule, but an alternative reading is that the white-box gains are in part an artifact of the gradient s
- A natural extension is a post-hoc hardening procedure: a short Sign-Symmetry fine-tuning pass applied after any backpropagation-trained model, possibly combined with adversarial training for defense in depth.
- The paper's own framing, using the word 'obfuscates' for what the approximate gradients do, invites a concrete test: compare the alignment between the true loss gradient and the gradient an adversary would compute on the Sign-Symmetry model; if they are nearly orthogonal, adaptive attacks should recover backpropagation-level vulnerability.
- If the robustness proves genuine, Sign-Symmetry fine-tuning becomes a cheap robustness knob for edge deployment, since it requires no changes to inference and no extra defense-specific training data.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes fine-tuning ImageNet-pretrained CNNs with Sign-Symmetry learning rules (uSF, frSF, brSF) instead of backpropagation, for image classification and hashing-based image retrieval. It claims that this approach maintains performance parity with BP while improving robustness against white-box gradient attacks (FGSM, PGD, HAG, SDHA), and that black-box attacks show no disadvantage. Experiments use AlexNet, VGG-16, and ResNet-18 on CIFAR-10, MS-COCO, NUS-WIDE, and ImageNet variants.
Significance. If the claims held, this would be a useful contribution: it would show that biologically motivated credit assignment can serve as a robustness-enhancing fine-tuning procedure without accuracy loss, and it would open a new empirical research direction. The paper also reports a broad benchmark matrix across architectures and datasets. However, the current evidence does not establish either performance parity or genuine robustness, and the proposed mechanism is not supported by the experiments.
major comments (4)
- [§5, Table 1] The classification accuracies in Table 1 are all multiples of 3.125%, which indicates that the evaluation set contains only 32 images (e.g., 90.62% vs. 93.75% is a one-sample difference). No seeds or confidence intervals are reported, so the claim that Sign-Symmetry fine-tuning 'maintains performance parity' with BP is not statistically established; the authors should evaluate on the full test sets or report repeated trials with error bars.
- [§6.2.1 and §7] The paper explains the robustness advantage by saying that approximate gradients 'obfuscate' exact gradients and make models 'inherently harder to fool.' However, a white-box attack computes gradients of the final trained network, which is fully differentiable and has the same functional form for BP- and Sign-Symmetry-fine-tuned models; the credit-assignment rule affects only the parameter trajectory during training, not the gradient computation at inference. The explanation is therefore a restatement of the observed effect rather than a mechanism, and it does not predict robustness.
- [§6.2.1, Fig. 2, and §5] The white-box evidence consists of FGSM and a 5-iteration PGD with no random restarts; this is much weaker than the attacks recommended in the literature the authors cite (Athalye et al., 2018; Carlini et al., 2019; Uesato et al., 2018). The large discrepancy between white-box and black-box results is the classic signature of gradient masking. The authors should validate with stronger attacks (more PGD steps, multiple restarts, or adaptive attacks) and test explicitly whether the apparent robustness survives; until then the central robustness claim is unsupported.
- [§6.2.2, Tables 3 and 4] The absence of any black-box advantage is not 'expected' but is directly problematic for the robustness claim. Decision-based attacks estimate the decision boundary, not the training gradients; if the model were genuinely more robust, black-box attacks should also require larger perturbations. The current tables (e.g., VGG16/MS-COCO under HSJA: BP 68.7% vs. brSF 3.1%) actually show Sign-Symmetry models being less robust under black-box attacks, which is what one would expect from gradient obfuscation. This needs to be addressed directly.
minor comments (4)
- [§4] In Section 4, 'RestNet-18' should be 'ResNet-18'; the same section uses 'Resnet-18' inconsistently.
- [§5] The heading 'Adverserial Robustness Evaluation Setup' contains a typo: 'Adverserial' should be 'Adversarial'.
- [§6.2.1] In Section 6.2.1, 'surpace' should be 'surpass'.
- [§6.2.2, Tables 3 and 4] Tables 3 and 4 report accuracy values like 6.25% and 9.37%, also consistent with very small evaluation sets; the authors should state the number of test images and the query budgets used by HSJA and Boundary Attack.
Circularity Check
No circular derivation; the robustness claim is empirical and not forced by the paper's own equations or self-citations.
full rationale
The paper does not attempt a mathematical derivation or a fitted-parameter prediction; it reports an empirical comparison of fine-tuning rules. The central robustness claim is supported by measured accuracy and mAP under FGSM, PGD, HAG, and SDHA, and no quantity entering those measurements is defined in terms of the conclusion it is used to support. The only self-citations (Berriche et al., 2024) appear as metric-practice references and are not load-bearing. The Section 7 explanation that approximate gradients "obfuscate" exact gradients is a causal hypothesis, and the lack of a black-box advantage (Tables 3-4) is a validity threat consistent with gradient masking, but neither is a circular reduction: the reported white-box robustness is not defined as "approximate gradients" nor fitted to reproduce itself. No circular step can therefore be exhibited from the paper's own equations or citation chain.
Assumptions & free parameters
free parameters (2)
- Fine-tuning hyperparameters (learning rates 1e-5 and 1e-4, 20 or 10 epochs, batch size 32, weight decay 5e-4)
- Adversarial attack settings (epsilon up to 0.5, PGD 5 iterations, step size epsilon/3)
assumptions (2)
- domain assumption Standard gradient-based white-box attacks (FGSM, PGD) are a fair and sufficient measure of adversarial robustness.
- domain assumption The evaluation set used for accuracy tables is large enough that differences of several percent are meaningful.
Cite this review
Pith. "Pith review of Sign-Symmetry Learning Rules are Robust Fine-Tuners." pith.science (2026). https://pith.science/paper/LXGDZ2E5
@misc{pith2026250205925,
author = {Pith},
title = {Pith review of: Sign-Symmetry Learning Rules are Robust Fine-Tuners},
year = {2026},
howpublished = {\url{https://pith.science/paper/LXGDZ2E5}},
note = {Machine review of arXiv:2502.05925}
}
read the original abstract
Backpropagation (BP) has long been the predominant method for training neural networks due to its effectiveness. However, numerous alternative approaches, broadly categorized under feedback alignment, have been proposed, many of which are motivated by the search for biologically plausible learning mechanisms. Despite their theoretical appeal, these methods have consistently underperformed compared to BP, leading to a decline in research interest. In this work, we revisit the role of such methods and explore how they can be integrated into standard neural network training pipelines. Specifically, we propose fine-tuning BP-pre-trained models using Sign-Symmetry learning rules and demonstrate that this approach not only maintains performance parity with BP but also enhances robustness. Through extensive experiments across multiple tasks and benchmarks, we establish the validity of our approach. Our findings introduce a novel perspective on neural network training and open new research directions for leveraging biologically inspired learning rules in deep learning.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...
-
[2]
Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples
Athalye, A., Carlini, N., and Wagner, D. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In International conference on machine learning, pp.\ 274--283. PMLR, 2018
2018
-
[3]
Bartunov, S., Santoro, A., Richards, B., Marris, L., Hinton, G. E., and Lillicrap, T. Assessing the scalability of biologically-motivated deep learning algorithms and architectures. Advances in neural information processing systems, 31, 2018
work page 2018
-
[4]
How auto-encoders could provide credit assignment in deep networks via target propagation
Bengio, Y. How auto-encoders could provide credit assignment in deep networks via target propagation. arXiv preprint arXiv:1407.7906, 2014
arXiv 2014
-
[5]
Leveraging High-Resolution Features for Improved Deep Hashing-based Image Retrieval
Berriche, A., Zakaria, M. A., and Baghdadi, R. Leveraging high-resolution features for improved deep hashing-based image retrieval. arXiv preprint arXiv:2403.13747, 2024
work page Pith review arXiv 2024
-
[6]
Bliss, T. V. and L mo, T. Long-lasting potentiation of synaptic transmission in the dentate area of the anaesthetized rabbit following stimulation of the perforant path. The Journal of physiology, 232 0 (2): 0 331--356, 1973
work page 1973
-
[7]
Decision-based adversarial attacks: Reliable attacks against black-box machine learning models, 2018
Brendel, W., Rauber, J., and Bethge, M. Decision-based adversarial attacks: Reliable attacks against black-box machine learning models, 2018. URL https://arxiv.org/abs/1712.04248
arXiv 2018
-
[8]
Cao, Z., Long, M., Wang, J., and Yu, P. S. Hashnet: Deep learning to hash by continuation. In Proceedings of the IEEE international conference on computer vision, pp.\ 5608--5617, 2017
work page 2017
Show all 50 references
-
[9]
On evaluating adversarial robustness
Carlini, N., Athalye, A., Papernot, N., Brendel, W., Rauber, J., Tsipras, D., Goodfellow, I., Madry, A., and Kurakin, A. On evaluating adversarial robustness. arXiv preprint arXiv:1902.06705, 2019
1902 arXiv
-
[10]
I., and Wainwright, M
Chen, J., Jordan, M. I., and Wainwright, M. J. Hopskipjumpattack: A query-efficient decision-based attack, 2020. URL https://arxiv.org/abs/1904.02144
2020 arXiv
-
[11]
Nus-wide: a real-world web image database from national university of singapore
Chua, T.-S., Tang, J., Hong, R., Li, H., Luo, Z., and Zheng, Y. Nus-wide: a real-world web image database from national university of singapore. In Proceedings of the ACM international conference on image and video retrieval, pp.\ 1--9, 2009
2009
-
[12]
The recent excitement about neural networks
Crick, F. The recent excitement about neural networks. Nature, 337 0 (6203): 0 129--132, 1989
1989
-
[13]
M., \'S wirszcz, G., Jaderberg, M., Osindero, S., Vinyals, O., and Kavukcuoglu, K
Czarnecki, W. M., \'S wirszcz, G., Jaderberg, M., Osindero, S., Vinyals, O., and Kavukcuoglu, K. Understanding synthetic gradients and decoupled neural interfaces. In International Conference on Machine Learning, pp.\ 904--912. PMLR, 2017
2017
-
[14]
Imagenet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp.\ 248--255. Ieee, 2009
2009
-
[15]
and Liu, L
Fang, Y. and Liu, L. Scalable supervised online hashing for image retrieval. Journal of Computational Design and Engineering, 8 0 (5): 0 1391--1406, 2021
2021
-
[16]
Goodfellow, I. J. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014
2014 arXiv
-
[17]
Competitive learning: From interactive activation to adaptive resonance
Grossberg, S. Competitive learning: From interactive activation to adaptive resonance. Cognitive science, 11 0 (1): 0 23--63, 1987
1987
-
[18]
Deep residual learning for image recognition
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 770--778, 2016
2016
-
[19]
Hebb, D. O. The organization of behavior: A neuropsychological theory. Psychology press, 2005
2005
-
[20]
Hinton, G. et al. How to do backpropagation in a brain. In Invited talk at the NIPS’2007 deep learning workshop, volume 656, pp.\ 1--16, 2007
2007
-
[21]
An efficient supervised deep hashing method for image retrieval
Hussain, A., Li, H.-C., Ali, M., Wali, S., Hussain, M., and Rehman, A. An efficient supervised deep hashing method for image retrieval. Entropy, 24 0 (10): 0 1425, 2022
2022
-
[22]
Learning multiple layers of features from tiny images
Krizhevsky, A., Hinton, G., et al. Learning multiple layers of features from tiny images. 2009
2009
-
[23]
Krizhevsky, A., Sutskever, I., and Hinton, G. E. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25, 2012
2012
-
[24]
J., Prakash, P
Lansdell, B. J., Prakash, P. R., and Kording, K. P. Learning to solve the credit assignment problem. arXiv preprint arXiv:1906.00889, 2019
1906 arXiv
-
[25]
Learning process in an asymmetric threshold network
Le Cun, Y. Learning process in an asymmetric threshold network. In Disordered systems and biological organization, pp.\ 233--240. Springer, 1986
1986
-
[26]
Difference target propagation
Lee, D.-H., Zhang, S., Fischer, A., and Bengio, Y. Difference target propagation. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2015, Porto, Portugal, September 7-11, 2015, Proceedings, Part I 15, pp.\ 498--515. Springer, 2015
2015
-
[27]
How important is weight symmetry in backpropagation? In Proceedings of the AAAI Conference on Artificial Intelligence, volume 30, 2016
Liao, Q., Leibo, J., and Poggio, T. How important is weight symmetry in backpropagation? In Proceedings of the AAAI Conference on Artificial Intelligence, volume 30, 2016
2016
-
[28]
P., Cownden, D., Tweed, D
Lillicrap, T. P., Cownden, D., Tweed, D. B., and Akerman, C. J. Random feedback weights support learning in deep neural networks. arXiv preprint arXiv:1411.0247, 2014
2014 arXiv
-
[29]
P., Cownden, D., Tweed, D
Lillicrap, T. P., Cownden, D., Tweed, D. B., and Akerman, C. J. Random synaptic feedback weights support error backpropagation for deep learning. Nature communications, 7 0 (1): 0 13276, 2016
2016
-
[30]
P., Santoro, A., Marris, L., Akerman, C
Lillicrap, T. P., Santoro, A., Marris, L., Akerman, C. J., and Hinton, G. Backpropagation and the brain. Nature Reviews Neuroscience, 21 0 (6): 0 335--346, 2020
2020
-
[31]
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll \'a r, P., and Zitnick, C. L. Microsoft coco: Common objects in context. In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp....
2014
-
[32]
A smart adversarial attack on deep hashing based image retrieval
Lu, J., Chen, M., Sun, Y., Wang, W., Wang, Y., and Yang, X. A smart adversarial attack on deep hashing based image retrieval. In Proceedings of the 2021 international conference on multimedia retrieval, pp.\ 227--235, 2021
2021
-
[33]
Image retrieval algorithm based on locality-sensitive hash using convolutional neural network and attention mechanism
Luo, Y., Li, W., Ma, X., and Zhang, K. Image retrieval algorithm based on locality-sensitive hash using convolutional neural network and attention mechanism. Information, 13 0 (10): 0 446, 2022
2022
-
[34]
Towards deep learning models resistant to adversarial attacks
Madry, A. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083, 2017
2017 arXiv
-
[35]
Regulation of synaptic efficacy by coincidence of postsynaptic aps and epsps
Markram, H., L \"u bke, J., Frotscher, M., and Sakmann, B. Regulation of synaptic efficacy by coincidence of postsynaptic aps and epsps. Science, 275 0 (5297): 0 213--215, 1997
1997
-
[36]
Foolbox: A python toolbox to benchmark the robustness of machine learning models
Rauber, J., Brendel, W., and Bethge, M. Foolbox: A python toolbox to benchmark the robustness of machine learning models. arXiv preprint arXiv:1707.04131, 2017
2017 arXiv
-
[37]
E., Hinton, G
Rumelhart, D. E., Hinton, G. E., and Williams, R. J. Learning representations by back-propagating errors. nature, 323 0 (6088): 0 533--536, 1986
1986
-
[38]
Schwartz, E. L. Computational neuroscience. Mit Press, 1993
1993
-
[39]
R., Resende, L., Orenstein, P., and Oliveira, R
Schwengber, L. R., Resende, L., Orenstein, P., and Oliveira, R. I. Deep hashing via householder quantization. arXiv preprint arXiv:2311.04207, 2023
2023 arXiv
-
[40]
and Zisserman, A
Simonyan, K. and Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014
2014 arXiv
-
[41]
and Gupta, S
Singh, A. and Gupta, S. Learning to hash: a comprehensive survey of deep learning-based hashing methods. Knowledge and Information Systems, 64 0 (10): 0 2565--2597, 2022
2022
-
[42]
Intriguing properties of neural networks
Szegedy, C. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013
2013 arXiv
-
[43]
Adversarial risk and the dangers of evaluating against weak attacks
Uesato, J., O’donoghue, B., Kohli, P., and Oord, A. Adversarial risk and the dangers of evaluating against weak attacks. In International conference on machine learning, pp.\ 5025--5034. PMLR, 2018
2018
-
[44]
R., Banburski, A., Pant, N., and Poggio, T
Vuyyuru, M. R., Banburski, A., Pant, N., and Poggio, T. Biologically inspired mechanisms for adversarial robustness. Advances in Neural Information Processing Systems, 33: 0 2135--2146, 2020
2020
-
[45]
Supervised hashing for image retrieval via image representation learning
Xia, R., Pan, Y., Lai, H., Liu, C., and Yan, S. Supervised hashing for image retrieval via image representation learning. In Proceedings of the AAAI conference on artificial intelligence, volume 28, 2014
2014
-
[46]
Hyp2 loss: Beyond hypersphere metric space for multi-label image retrieval
Xu, C., Chai, Z., Xu, Z., Yuan, C., Fan, Y., and Wang, J. Hyp2 loss: Beyond hypersphere metric space for multi-label image retrieval. In Proceedings of the 30th ACM international conference on multimedia, pp.\ 3173--3184, 2022
2022
-
[47]
Adversarial examples for hamming space search
Yang, E., Liu, T., Deng, C., and Tao, D. Adversarial examples for hamming space search. IEEE transactions on cybernetics, 50 0 (4): 0 1473--1484, 2018
2018
-
[48]
Deep parameter-free attention hashing for image retrieval
Yang, W., Wang, L., and Cheng, S. Deep parameter-free attention hashing for image retrieval. Scientific Reports, 12 0 (1): 0 7082, 2022
2022
-
[49]
Semantic-aware adversarial training for reliable deep hashing retrieval
Yuan, X., Zhang, Z., Wang, X., and Wu, L. Semantic-aware adversarial training for reliable deep hashing retrieval. IEEE Transactions on Information Forensics and Security, 2023
2023
-
[50]
S., and Hsieh, C.-J
Zhang, H., Chen, H., Song, Z., Boning, D., Dhillon, I. S., and Hsieh, C.-J. The limitations of adversarial training and the blind-spot attack. arXiv preprint arXiv:1901.04684, 2019
1901 arXiv
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.