{"id":"508a32c0-10cb-4c2f-8b67-6bc40ab15911","arxiv_id":"2502.05925","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"Fine-tuning with sign-symmetry rules preserves accuracy while resisting white-box adversarial attacks, but the robustness appears to be gradient masking rather than genuine defense.","lead":"The authors propose fine-tuning ImageNet-pretrained networks with sign-symmetry learning rules instead of backpropagation. They report accuracy close to backpropagation with better resistance to gradient-based attacks, but the evaluation uses very small test sets and likely reflects gradient masking.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Robustness claim rests on weak white-box attacks that can be explained by gradient masking; the proposed mechanism cannot produce the claimed effect, so the central claim is unestablished.","rationale":"I read the paper in good faith. The hybrid pipeline—BP pretraining followed by sign-symmetry fine-tuning—is a testable idea, and the paper correctly notes that prior bio-plausible learning rules underperform BP on large-scale training. For the central claim to hold, two conditions must both be true: (1) fine-tuned accuracy is statistically comparable to BP, and (2) the white-box robustness advantage reflects true boundary robustness. The first condition is unsupported because Table 1 reports values like 90.62, 93.75, and 100.0, which are exact multiples of 100/32; this implies test sets of only 32 images, making apparent differences of 3-6 points meaningless. The second condition is the load-bearing one and is undercut by both the mechanism and the black-box results. The paper attributes robustness to approximate gradients during training, but attack gradients are computed on the final network, which is a standard differentiable neural network; sign-symmetry does not alter the forward pass or the inference-time Jacobian. Consequently, the proposed mechanism cannot explain the observed white-box/black-box asymmetry. The black-box tables show no consistent advantage, exactly the signature of gradient masking described in Athalye et al. 2018, which the paper cites but never uses to test its own defense. A concrete adaptive-attack check—PGD-200 with restarts or AutoAttack—would settle whether the gap is real. If the gap collapses, the central claim fails; if it persists, the robustness finding would be worth further study. Because the reader's verdict of REJECT is already based on this weakness, my analysis does not change the verdict, though it sharpens the reason: the claimed mechanism is internally inconsistent with how white-box attacks operate.","tokens_in":13677,"tokens_out":3953,"duration_ms":42055,"concrete_test":"Run an adaptive white-box evaluation on a representative condition (e.g., ResNet-18 fine-tuned with frSF or uSF on CIFAR-10): replace 5-step PGD with PGD-200 using 20 random restarts or AutoAttack, and include a backward-pass differentiable approximation (BPDA) if the model contains any non-differentiable operation. If the sign-symmetry models' robust accuracy drops to BP levels under this attack, the claimed robustness is gradient masking rather than genuine robustness.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires the white-box robustness advantage to reflect genuine decision-boundary robustness, not gradient obfuscation. The paper's mechanism argument in Sections 5 and 6.2.1 is that sign-symmetry's approximate gradients make models 'inherently harder to fool' by gradient-based attacks. But adversarial attacks compute gradients of the loss with respect to the input on the final, fully differentiable network; sign-symmetry only changes the optimizer's parameter updates during training, not the forward/backward pass used at inference. Thus the proposed mechanism does not explain why FGSM/PGD should be less effective on these models. The paper's own black-box results (Tables 3-4) show no consistent advantage for sign-symmetry, which is the classic gradient-masking signature described in Athalye et al. 2018, a reference the paper cites but does not apply as a validation check. Additionally, the performance-parity premise is not established: Table 1 accuracies are multiples of 3.125%, implying 32-image evaluation sets, so reported differences of a few points are within one or two samples and are not statistically meaningful. The most load-bearing concern is therefore that the headline robustness enhancement is an artifact of weak white-box attacks rather than a property of the trained model's boundary.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes fine-tuning ImageNet-pretrained CNNs with Sign-Symmetry learning rules (uSF, frSF, brSF) instead of backpropagation, for image classification and hashing-based image retrieval. It claims that this approach maintains performance parity with BP while improving robustness against white-box gradient attacks (FGSM, PGD, HAG, SDHA), and that black-box attacks show no disadvantage. Experiments use AlexNet, VGG-16, and ResNet-18 on CIFAR-10, MS-COCO, NUS-WIDE, and ImageNet variants.","tokens_in":13987,"tokens_out":5177,"duration_ms":48378,"significance":"If the claims held, this would be a useful contribution: it would show that biologically motivated credit assignment can serve as a robustness-enhancing fine-tuning procedure without accuracy loss, and it would open a new empirical research direction. The paper also reports a broad benchmark matrix across architectures and datasets. However, the current evidence does not establish either performance parity or genuine robustness, and the proposed mechanism is not supported by the experiments.","major_comments":[{"comment":"The classification accuracies in Table 1 are all multiples of 3.125%, which indicates that the evaluation set contains only 32 images (e.g., 90.62% vs. 93.75% is a one-sample difference). No seeds or confidence intervals are reported, so the claim that Sign-Symmetry fine-tuning 'maintains performance parity' with BP is not statistically established; the authors should evaluate on the full test sets or report repeated trials with error bars.","section":"§5, Table 1"},{"comment":"The paper explains the robustness advantage by saying that approximate gradients 'obfuscate' exact gradients and make models 'inherently harder to fool.' However, a white-box attack computes gradients of the final trained network, which is fully differentiable and has the same functional form for BP- and Sign-Symmetry-fine-tuned models; the credit-assignment rule affects only the parameter trajectory during training, not the gradient computation at inference. The explanation is therefore a restatement of the observed effect rather than a mechanism, and it does not predict robustness.","section":"§6.2.1 and §7"},{"comment":"The white-box evidence consists of FGSM and a 5-iteration PGD with no random restarts; this is much weaker than the attacks recommended in the literature the authors cite (Athalye et al., 2018; Carlini et al., 2019; Uesato et al., 2018). The large discrepancy between white-box and black-box results is the classic signature of gradient masking. The authors should validate with stronger attacks (more PGD steps, multiple restarts, or adaptive attacks) and test explicitly whether the apparent robustness survives; until then the central robustness claim is unsupported.","section":"§6.2.1, Fig. 2, and §5"},{"comment":"The absence of any black-box advantage is not 'expected' but is directly problematic for the robustness claim. Decision-based attacks estimate the decision boundary, not the training gradients; if the model were genuinely more robust, black-box attacks should also require larger perturbations. The current tables (e.g., VGG16/MS-COCO under HSJA: BP 68.7% vs. brSF 3.1%) actually show Sign-Symmetry models being less robust under black-box attacks, which is what one would expect from gradient obfuscation. This needs to be addressed directly.","section":"§6.2.2, Tables 3 and 4"}],"minor_comments":[{"comment":"In Section 4, 'RestNet-18' should be 'ResNet-18'; the same section uses 'Resnet-18' inconsistently.","section":"§4"},{"comment":"The heading 'Adverserial Robustness Evaluation Setup' contains a typo: 'Adverserial' should be 'Adversarial'.","section":"§5"},{"comment":"In Section 6.2.1, 'surpace' should be 'surpass'.","section":"§6.2.1"},{"comment":"Tables 3 and 4 report accuracy values like 6.25% and 9.37%, also consistent with very small evaluation sets; the authors should state the number of test images and the query budgets used by HSJA and Boundary Attack.","section":"§6.2.2, Tables 3 and 4"}],"recommendation":"major_revision","confidential_remarks":"The paper is currently far from supporting its headline claims. If the authors redo the evaluation with proper test sets, seeds, and strong or adaptive attacks, the result may be salvageable; otherwise the robustness claim will likely disappear. The mechanism argument in Sections 6 and 7 should be substantially revised or removed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Interesting idea, but the central claim is not supported. The specific recipe—BP for pre-training, sign-symmetry for fine-tuning—is new and worth a test, but the robustness evidence is consistent with gradient masking and the performance numbers are statistically meaningless.\n\nThe paper does a few things right: it examines a plausible hypothesis, runs experiments on two tasks, three architectures, and several datasets, and it reports black-box results even though they undermine the main claim. The writing is clear, and they are honest about the possibility that the attack comparison is unfair.\n\nThe soft spots are serious. First, Table 1 accuracies are multiples of 3.125%, which implies a 32-image test set. A three-point difference is one image; with no error bars or seeds, the performance parity claim is not established. Second, the robustness experiments use weak attacks: PGD with only 5 iterations and step size epsilon/3, far too weak to separate genuine robustness from a model that simply has a less steep loss landscape. Third, the mechanism argument is flawed. Sign-symmetry changes the optimizer's parameter updates during training; at attack time the network is fully differentiable and exact gradients are available. So the claim that approximate gradients make the model 'inherently harder to fool' is not supported by the paper's own setup. Fourth, the black-box results show no consistent advantage, which is exactly the signature of obfuscated gradients described in Athalye et al. The authors cite that work but do not apply its recommended validation checks. Their statement that a fair comparison would require attacks designed for bio-plausible methods is a concession that the white-box numbers may be an artifact.\n\nIf these issues were fixed—proper evaluation sets, stronger PGD with multiple restarts, and an adaptive attack—the robustness claim could be confirmed or dismissed. As written, the paper does not establish its main result.\n\nThis is a paper for people working on robust fine-tuning and bio-plausible learning rules. It deserves a serious referee, not a desk reject, but I would expect the reviewers to ask for major revisions. I would not cite it in its current form.","headline":"Robustness claim is likely a gradient-masking artifact; performance parity is unverified, but the fine-tuning recipe is a legitimate, testable idea.","tokens_in":14486,"tokens_out":3793,"would_cite":false,"duration_ms":36972,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Fine-tuning with Sign-Symmetry learning rules keeps backpropagation-level performance while making models markedly more resistant to gradient-based adversarial attacks.","keywords":["backpropagation","sign-symmetry learning rules","feedback alignment","fine-tuning","adversarial robustness","white-box attacks","deep hashing","image retrieval"],"falsifier":"Run a much stronger white-box attack on the Sign-Symmetry fine-tuned models—for example, PGD with fifty iterations and several random restarts, or an attack that approximates the true gradient by straight-through estimation of the sign function—and check whether the robust accuracy at $\\epsilon=0.1$ collapses to the same near-zero level as backpropagation fine-tuning. If it does, the robustness is gradient masking, not a property of the decision boundary.","tokens_in":13504,"feed_emoji":"🛡️","tokens_out":6986,"duration_ms":59563,"temperature":0.7,"pith_summary":"This paper proposes a hybrid training recipe: start with a backbone pre-trained by backpropagation, then fine-tune all parameters with a Sign-Symmetry learning rule instead of backpropagation. The paper claims that, across image classification and hashing-based image retrieval, on three backbones and several datasets, this recipe keeps top-1 accuracy or mean average precision on par with backpropagation fine-tuning while making models substantially harder to fool with gradient-based white-box attacks such as FGSM, PGD, HAG, and SDHA. The authors attribute the effect to approximate, sign-based gradients that give a white-box adversary a less reliable landscape to optimize against. If the claim holds, biological learning rules become a practical robustness mechanism rather than a performance penalty, and standard pre-train/fine-tune pipelines gain a cheap hardening step.","feed_headline":"Sign-symmetry fine-tuning fends off gradient attacks","feed_subtitle":"Pre-trained models fine-tuned with sign-based rules match backpropagation accuracy and resist FGSM/PGD far better.","key_machinery":"The central object is the Sign-Symmetry feedback matrix $V$ that substitutes for the transposed weight matrix in the backward pass: for uSF, $V = \\operatorname{sign}(W^\\top)$; for frSF, $V = M \\circ \\operatorname{sign}(W^\\top)$ with $M$ fixed random magnitudes; for brSF, $M$ is redrawn every update. This changes the error signal $\\delta_l = (V_{l+1}\\delta_{l+1}) \\circ f'(a_l)$ that propagates through the network during fine-tuning. The sign alignment keeps updates correlated with true gradient descent—enough to fine-tune effectively—but the magnitudes are no longer exact, so the model's forward gradient no longer points an adversary straight to a loss-increasing perturbation. That mismatch is the mechanism the paper credits for the robustness.","core_discovery":"The central discovery is that the weakness of bio-plausible learning rules—imprecise gradients—flips into an advantage once a model is already well initialized. Backpropagation pre-training supplies good representations; fine-tuning with Sign-Symmetry rules, where the backward matrix is the sign of the forward weight matrix (uSF), optionally with fixed or redrawn random magnitudes (frSF, brSF), continues to adapt the weights well enough to match backpropagation's task performance while resisting white-box adversarial perturbation. In the reported experiments, the accuracy gap between backpropagation and Sign-Symmetry fine-tuned models under FGSM and PGD can exceed 50 percentage points at large perturbation sizes, and the mean average precision gap under hashing attacks can reach tens of points, while black-box attacks show no consistent advantage for either family. The paper interprets this as a specificity: sign-symmetry fine-tuning hardens models against the very attacks that exploit gradient access.","pith_inferences":["We read the absence of black-box advantage as the key unresolved point: if the robustness were due to a genuinely flatter loss landscape, some transfer-based or decision-based attacks would likely still show an edge; the authors attribute the parity to the attack being independent of the learning rule, but an alternative reading is that the white-box gains are in part an artifact of the gradient s","A natural extension is a post-hoc hardening procedure: a short Sign-Symmetry fine-tuning pass applied after any backpropagation-trained model, possibly combined with adversarial training for defense in depth.","The paper's own framing, using the word 'obfuscates' for what the approximate gradients do, invites a concrete test: compare the alignment between the true loss gradient and the gradient an adversary would compute on the Sign-Symmetry model; if they are nearly orthogonal, adaptive attacks should recover backpropagation-level vulnerability.","If the robustness proves genuine, Sign-Symmetry fine-tuning becomes a cheap robustness knob for edge deployment, since it requires no changes to inference and no extra defense-specific training data."],"forward_implications":["Sign-Symmetry fine-tuning can act as a drop-in replacement for backpropagation fine-tuning in pre-trained pipelines, preserving task performance while improving resistance to white-box gradient attacks.","The robustness effect transfers across tasks, appearing in both image classification and hashing-based retrieval, which suggests it is a property of the learning rule rather than of one loss function or architecture.","Among the variants, frSF is the most consistently performant and stable choice for fine-tuning.","Because black-box attacks do not distinguish the methods, the advantage is specific to adversaries with gradient access, meaning the method hardens a particular attack surface without claiming global robustness."],"supporting_citations":[{"why":"Defines the Sign-Symmetry feedback rules (uSF, frSF, brSF) that the fine-tuning method is built on.","marker":"Liao et al., 2016"},{"why":"Documents the performance gap between backpropagation and bio-plausible methods, the baseline this paper aims to bridge.","marker":"Bartunov et al., 2018"},{"why":"Supplies the biologically inspired robustness mechanisms and the adversarial testing setup the paper follows.","marker":"Vuyyuru et al., 2020"},{"why":"Warns that obfuscated gradients can fake robustness; the paper cites it in its evaluation setup, making it load-bearing for interpreting the white-box results.","marker":"Athalye et al., 2018"},{"why":"Defines the FGSM attack used as the primary white-box classification attack.","marker":"Goodfellow, 2014"},{"why":"Defines the PGD attack, the iterative white-box attack used alongside FGSM.","marker":"Madry, 2017"},{"why":"Provides Foolbox, the toolkit the paper uses to run the adversarial attacks.","marker":"Rauber et al., 2017"},{"why":"Provides the Boundary Attack used as the black-box comparison.","marker":"Brendel et al., 2018"},{"why":"Provides the HopSkipJump attack used as the second black-box comparison.","marker":"Chen et al., 2020"}],"fun_headline_variants":["Sign-symmetry fine-tuning improves adversarial robustness","Sign-symmetry fine-tuning matches BP and resists attacks","Fine-tuning with sign-symmetry: accuracy plus adversarial defense","Sign-based fine-tuning: equal accuracy, far better robustness"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the apparent white-box robustness is genuine model robustness rather than obfuscated gradients; this enters at Section 6.2.1, where FGSM and PGD are the sole evidence, and it is undercut by the paper's own black-box tables (Tables 3-4) showing no advantage, the pattern the cited obfuscated-gradients literature warns about.","fun_headline_variants_meta":{"raw":{"variants":["Sign-symmetry fine-tuning improves adversarial robustness","Sign-symmetry fine-tuning matches BP and resists attacks","Fine-tuning with sign-symmetry: accuracy plus adversarial defense","Sign-based fine-tuning: equal accuracy, far better robustness"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000576,"raw_usage":{"total_tokens":2689,"prompt_tokens":886,"completion_tokens":1803,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":502,"completion_tokens_details":{"reasoning_tokens":1736}},"tokens_in":502,"tokens_out":1803,"duration_ms":12798,"temperature":1.0,"reasoning_tokens":1736,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T17:21:13.039802+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a much stronger white-box attack on the Sign-Symmetry fine-tuned models—for example, PGD with fifty iterations and several random restarts, or an attack that approximates the true gradient by straight-through estimation of the sign function—and check whether the robust accuracy at $\\epsilon=0.1$ collapses to the same near-zero level as backpropagation fine-tuning. If it does, the robustness is gradient masking, not a property of the decision boundary.","supporting_citations":[{"cited_title":"How important is weight symmetry in backpropagation? In Proceedings of the AAAI Conference on Artificial Intelligence, volume 30, 2016","cited_arxiv_id":null,"evidence_quote":"Defines the Sign-Symmetry feedback rules (uSF, frSF, brSF) that the fine-tuning method is built on."},{"cited_title":"E., and Lillicrap, T","cited_arxiv_id":null,"evidence_quote":"Documents the performance gap between backpropagation and bio-plausible methods, the baseline this paper aims to bridge."},{"cited_title":"R., Banburski, A., Pant, N., and Poggio, T","cited_arxiv_id":null,"evidence_quote":"Supplies the biologically inspired robustness mechanisms and the adversarial testing setup the paper follows."}],"review_version":1}