{"id":"03cb8328-ca2f-4d00-a908-8cad4a40a277","arxiv_id":"2507.13485","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A NAS framework that searches a per-layer learning rule along with the architecture produces bio-inspired networks that beat single-rule bio-inspired baselines and some backprop-trained models.","lead":"This paper adds biologically plausible learning rules to the search space of neural architecture search, so a network can use different learning rules in different layers. The resulting mixed-rule networks set new accuracy records for bio-inspired models and keep strong robustness to adversarial attacks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim that mixed bio-inspired rules beat a uniform rule is not directly tested: the headline tables compare different architectures and training budgets, and the same-architecture uniform-rule control appears only in an unquantified appendix.","rationale":"The reader's weakest assumption correctly flags the 150- vs 600-epoch discrepancy and the architecture mismatch in Table 1. I agree this is a real confound, but the more general form of the problem is the absence of a same-architecture, same-budget uniform-rule control anywhere in the main text; the random-assignment ablation (Table 5) tests arbitrary mixes but not the single-rule endpoint, and the appendix only provides plots without numbers. The proposed control experiment would settle whether the mixed-rule advantage is genuine. Because the paper does include code, a plausible search formulation, and some unquantified same-architecture comparisons, the result is not demonstrably wrong, but the central claim is not established as stated. The reader's CONDITIONAL verdict is therefore appropriate; I do not move it.","tokens_in":16100,"tokens_out":7634,"duration_ms":79171,"concrete_test":"Train the exact final BioNAS-DARTS architecture from Table 1 on CIFAR-10 and CIFAR-100 with each single learning rule in the search space (FA, uSF, brSF, frSF) applied uniformly to all layers, using the identical 600-epoch protocol, and compare test accuracy to the reported mixed-rule results (95.16% and 76.48%). Report mean and standard deviation over at least 3 seeds; if any uniform-rule variant matches the mixed-rule accuracy within one standard deviation, the central claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract's central claim is that per-layer diversity of bio-inspired learning rules improves accuracy over a single rule applied to all layers. The main evidence (Table 1) compares BioNAS-DARTS, a searched DARTS cell trained for 600 epochs, with ResNet56 variants trained with a single rule; CIFAR-100 entries marked '*' were reproduced under a 150-epoch protocol. This comparison conflates rule diversity with architecture and training budget, so it cannot isolate the claimed effect. Table 5 shows that random rule assignments to the searched architecture retain high accuracy, but it omits the required uniform-rule control on that same architecture. Appendix 7.2 reportedly trains the same architecture with a single rule end-to-end, but reports only weight-distribution and variance plots (Figures 5-6) without numeric accuracies. Thus the paper never cleanly demonstrates that mixing rules, rather than the searched architecture or longer training, produces the observed gains. The ImageNet 'record' claim is also weakened by comparing against self-supervised SoftHebb/FastHebb, though the core issue is the missing controlled uniform-rule baseline.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces BioNAS, a neural architecture search framework that expands the DARTS/EG-NAS search space to include biologically plausible learning rules (FA, uSF, brSF, frSF, Hebbian, predictive coding) so that each layer can be assigned a different rule. The authors report that the searched architectures achieve 95.16% on CIFAR-10, 76.48% on CIFAR-100, 43.42% on ImageNet16-120, and 60.51% top-1 on ImageNet, and claim these are new records for bio-inspired models, with improved adversarial robustness. The central claim is that per-layer diversity of learning rules outperforms a single rule applied uniformly.","tokens_in":16254,"tokens_out":5491,"duration_ms":57330,"significance":"If the central claim were established by controlled comparisons, this paper would be a useful contribution: it is, to my knowledge, the first NAS framework that treats learning rules as part of the searchable space, and it provides code and builds on the open-source Biotorch package. The empirical claims are not the result of post hoc fitting; they are benchmark numbers generated by a stated search procedure. However, the current evidence does not cleanly separate the effect of rule diversity from architecture and training budget, and the theoretical justification is largely qualitative. With a same-architecture, matched-protocol uniform-rule control, the central claim could become convincing.","major_comments":[{"comment":"The headline comparison in Table 1 does not isolate the effect of mixing learning rules. BioNAS-DARTS is a 20-layer, 36-channel searched DARTS-style architecture trained for 600 epochs (Section 3.4.1), while the single-rule baselines are ResNet20/ResNet56 architectures, and the CIFAR-100 entries marked * were reproduced under a 150-epoch protocol. The large accuracy gaps (e.g., 23.52% versus 65.23% for ResNet56-uSF on CIFAR-100) can plausibly be attributed to architecture or training budget rather than rule diversity. This is load-bearing because the abstract's central claim is that mixed rules beat a single rule.","section":"Section 4, Table 1; Section 3.4.1"},{"comment":"The required same-architecture uniform-rule control is missing. Section 4.3 reports that training the searched architecture with a fixed rule yields higher gradient variance, but it gives no numeric test accuracies. Appendix 7.2 says that the BioNAS-EG architecture was trained end-to-end with each single rule, but it reports only weight-distribution and variance plots (Figures 5 and 6), again without accuracies. Table 5 tests random rule assignments but has no uniform-rule row. Consequently, the paper never directly demonstrates that mixing rules, rather than the searched architecture itself, produces the observed accuracy gains.","section":"Section 4.3, Table 5, Appendix 7.2"},{"comment":"The claim of new records for bio-inspired models is weakened by heterogeneous baselines. In Table 2, SoftHebb and FastHebb are self-supervised, as the authors acknowledge, so the ImageNet comparison (60.51% versus 27.00% and 21.34%) is not a like-for-like supervised record. In Table 1, the single-rule ResNet baselines are not only different architectures but also, for CIFAR-100, trained under a different (150-epoch) protocol. The record claims should be restricted to matched supervised settings or explicitly framed as indicative rather than controlled comparisons.","section":"Section 4, Tables 1 and 2"},{"comment":"The theoretical justification is post hoc and does not provide a testable mechanism. Equation (13) defines the mixed update as a weighted sum of rule-specific updates, but no argument is given for why this weighted sum should have lower variance, better landscape exploration, or improved adversarial robustness; the claims in Sections 7.4.1-7.4.3 are qualitative assertions. Since the main text cites this appendix as supporting the empirical results, it should either be made rigorous or explicitly labeled as speculation.","section":"Section 7.4"}],"minor_comments":[{"comment":"The search-space size paragraph says R = 4 learning rules (FA, uSF, brSF, Hebbian), but Section 3.2 and the rule definitions also include frSF and later mention predictive coding; please clarify the exact set of rules in the final search space.","section":"Section 3.3"},{"comment":"The table text refers to a 'ResNet-20 trained with standard backpropagation (RN56-BP)', which mixes ResNet20 and ResNet56 naming; use a consistent architecture name.","section":"Table 4"},{"comment":"The footnote '*CIFAR-100 values with * were reproduced under our 150-epoch protocol' is ambiguous: it should state precisely which rows and which hyperparameters differ, and whether any CIFAR-10 values were also reproduced under a non-standard protocol.","section":"Table 1 footnote"},{"comment":"Table 5 reports six random rule assignments with accuracies near 94.8% but gives no standard deviation or number of seeds; adding repeated runs would strengthen the claim that mixed rules are stable regardless of the specific assignment.","section":"Section 4.2, Table 5"}],"recommendation":"major_revision","confidential_remarks":"To the editor: the paper's main weakness is experimental design rather than execution. The central claim requires a uniform-rule baseline on the exact searched architecture with identical training setup; currently such controls appear only as qualitative weight/variance plots. If the authors can supply those numbers, the paper would likely be acceptable. The ImageNet 'record' comparison against self-supervised methods should also be reframed or supplemented with supervised bio-inspired baselines."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The real news here is the search dimension: adding bio-inspired learning rules to the NAS space and letting the search pick a rule per layer. I don't see that in the cited spiking-NAS work, and the mixed-rule result is genuinely new. The paper also does solid engineering: it builds cleanly on DARTS and EG-NAS, releases code, and benchmarks across four datasets with a reasonable compute budget. The adversarial robustness section is a nice extra, even if those comparisons inherit the same control problems.\n\nThe soft spot is exactly where the stress-test note lands. The central claim—mixed rules beat a single rule—is not actually isolated in the main tables. Table 1 compares BioNAS-DARTS (a searched cell, 600 epochs) against ResNet56 variants trained with a single rule, and the CIFAR-100 values are reproduced under a 150-epoch protocol. That conflates rule diversity with architecture and training budget. Table 5 shows random rule assignments retain accuracy, but it omits the uniform-rule control on the same architecture. The appendix (7.2) does train the resulting architecture with a single rule end-to-end, but only reports distribution and variance plots, not accuracies. So the paper never cleanly demonstrates that mixing, rather than the searched architecture or longer training, drives the gains. The theoretical justification in 7.4 is post hoc and qualitative; equation 13 just asserts the benefit of diversity.\n\nThat said, the flaw is fixable. The authors appear to have the data already; they just need to report numeric accuracy for the same-architecture uniform-rule baselines, ideally with multiple seeds and error bars. If the gap holds under that control, this becomes a solid subfield-level advance. If it shrinks, the paper is still a useful NAS extension, but the headline claim needs heavy qualification.\n\nWho is this for? Researchers working on bio-inspired learning, feedback alignment, or NAS who want a concrete demonstration that learning rules can be searched over. It deserves a serious referee; I'd send it to peer review with a request for the controlled baseline before acceptance.","headline":"Useful first step: NAS with per-layer learning rules is new and plausible, but the paper's core 'mixed beats single' claim still lacks a controlled same-architecture baseline.","tokens_in":16821,"tokens_out":1809,"would_cite":false,"duration_ms":20810,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that mixing different bio-inspired learning rules across layers, with the per-layer rule chosen automatically by neural architecture search, yields higher accuracy than using one rule throughout, and that the resulting…","keywords":["neural architecture search","bio-inspired learning rules","feedback alignment","sign-concordant feedback","adversarial robustness","mixed learning rules","CIFAR-10","ImageNet"],"falsifier":"Retrain the best mixed-rule architecture with each single learning rule applied uniformly to every layer, using the same 600-epoch protocol on CIFAR-10 and CIFAR-100; if any uniform-rule version equals or beats the mixed-rule accuracy (95.16% and 76.48%, respectively), the paper's claim that rule diversity itself drives the gain would be refuted.","tokens_in":15855,"feed_emoji":"🧠","tokens_out":10520,"duration_ms":113966,"temperature":0.7,"pith_summary":"This paper claims that allowing different bio-inspired learning rules in different layers, with the per-layer rule discovered automatically by neural architecture search (NAS), produces more accurate networks than using a single rule everywhere. The authors build on differentiable and evolutionary NAS procedures, expanding each candidate operation into a pair consisting of a computational block and a learning rule, then search over both jointly. They report new bio-inspired accuracy records of 95.16% on CIFAR-10, 76.48% on CIFAR-100, 43.42% on ImageNet16-120, and 60.51% top-1 on ImageNet, and find that in some regimes these networks match or exceed comparable back-propagation-trained networks. If these results hold, layer-wise diversity in learning rules is a scalable design choice that narrows the accuracy gap with back-propagation while preserving adversarial robustness.","feed_headline":"Per-layer learning rules lift bio-inspired nets to 95.16% on CIFAR-10","feed_subtitle":"Per-layer NAS choice of feedback rule sets bio-inspired records on CIFAR and ImageNet.","key_machinery":"The central machinery is an enlarged NAS search space in which every edge of the supernetwork carries a weighted mixture over operation-rule pairs, so the network jointly selects both what computation to apply and which bio-inspired rule should train it. The learning rules are feedback-alignment variants that replace the transposed-weight feedback of back-propagation with a feedback matrix: one uses a fixed random matrix, one uses the sign of the forward weights, one uses a redrawn random magnitude times that sign, and one uses a fixed random magnitude times that sign. The search relaxes the discrete choice of pairs into continuous architecture parameters through a softmax, optimizes weights and architecture parameters alternately, and at the end keeps the highest-probability pair on each edge. This object is what lets the paper test whether per-layer rule diversity, rather than architecture alone, drives accuracy and robustness.","core_discovery":"The paper's central claim is that the choice of learning rule is a searchable architectural dimension, not a fixed training detail. By pairing each candidate operation in the search space with a bio-inspired rule, the NAS procedure selects a rule for every layer. The resulting model reaches test errors of 4.84% on CIFAR-10 and 23.52% on CIFAR-100, and the same architecture transfers to 43.42% accuracy on ImageNet16-120 and 60.51% top-1 on ImageNet. Randomly assigning rules to layers keeps accuracy high, which the authors take as evidence that the improvement comes from mixing rules rather than from any particular discovered combination. The paper also reports lower gradient variance under mixed rules and links the diversity to stronger performance under one-pixel, square, transfer, and projected-gradient attacks.","pith_inferences":["This suggests that mixing learning rules may act as an implicit regularizer: the paper's own weight-distribution and gradient-variance observations point that way, but the connection to L1 or L2 regularization is not proven.","One testable extension is applying the same per-layer rule search to transformer or spiking architectures, where current single-rule biological training methods struggle; if the diversity benefit transfers, it would support a general principle rather than a CNN-specific effect.","The robustness finding may mean that mixed feedback matrices give gradient-based attackers an inconsistent target across layers, an idea that could be used to harden conventional networks by injecting rule diversity during training.","The paper's own caveat is that the theoretical account, lower gradient variance and better loss-landscape exploration, is left for future work, so the causal story behind the accuracy gain remains open."],"forward_implications":["Bio-inspired networks can be trained to accuracy competitive with back-propagation on standard benchmarks, narrowing the gap that previously limited them to small tasks.","Learning-rule assignment becomes a searchable design axis: future NAS pipelines can treat the training algorithm per layer as part of the architecture.","The benefit of mixing does not depend on finding one special pattern, since random assignments also perform well, which suggests the effect is robust across searches.","Mixed-rule networks keep useful accuracy under gradient-based attacks where single-rule and back-propagation models collapse, so robustness is preserved at higher accuracy.","The approach extends to larger datasets such as ImageNet, and a CIFAR-searched architecture transfers, indicating the mixed-rule benefit is not limited to small-scale tasks."],"supporting_citations":[{"why":"This is the differentiable NAS base that the mixed-rule search extends by enlarging the search space to operation-rule pairs.","marker":"[32]"},{"why":"This is the evolutionary NAS base that the mixed-rule search also extends, providing the evolutionary update and the training settings used for larger datasets.","marker":"[5]"},{"why":"This supplies the feedback-alignment learning-rule implementations used as candidates in the search space.","marker":"[46]"},{"why":"This work defines the sign-concordant feedback rules that the mixed-rule search space relies on.","marker":"[30]"},{"why":"This provides the ImageNet16-120 benchmark used to test the searched architectures.","marker":"[17]"},{"why":"This supplies the ImageNet dataset used for the large-scale accuracy evaluation.","marker":"[14]"},{"why":"This benchmarking study provides the adversarial robustness context and the single-rule baselines for robustness comparisons.","marker":"[47]"}],"fun_headline_variants":["NAS picks per-layer bio-inspired rules, setting CIFAR-10 record","Mixed learning rules per layer close gap with backprop in NAS","Layer-wise rule mixing in NAS beats single-rule bio-inspired nets","NAS discovers per-layer bio-inspired rules, rivals BP accuracy","Bio-inspired nets get a NAS boost from mixed per-layer rules"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The accuracy comparison assumes that the single-rule baselines were trained under the same protocol as the mixed-rule model; the paper reports some CIFAR-100 baselines under a 150-epoch protocol while the mixed-rule model trained for 600 epochs on a different architecture, so the reported gap could shrink under equal conditions.","fun_headline_variants_meta":{"raw":{"variants":["NAS picks per-layer bio-inspired rules, setting CIFAR-10 record","Mixed learning rules per layer close gap with backprop in NAS","Layer-wise rule mixing in NAS beats single-rule bio-inspired nets","NAS discovers per-layer bio-inspired rules, rivals BP accuracy","Bio-inspired nets get a NAS boost from mixed per-layer rules"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000301,"raw_usage":{"total_tokens":1737,"prompt_tokens":946,"completion_tokens":791,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":562,"completion_tokens_details":{"reasoning_tokens":702}},"tokens_in":562,"tokens_out":791,"duration_ms":9423,"temperature":1.0,"reasoning_tokens":702,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T16:24:09.215428+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the best mixed-rule architecture with each single learning rule applied uniformly to every layer, using the same 600-epoch protocol on CIFAR-10 and CIFAR-100; if any uniform-rule version equals or beats the mixed-rule accuracy (95.16% and 76.48%, respectively), the paper's claim that rule diversity itself drives the gain would be refuted.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"This is the evolutionary NAS base that the mixed-rule search also extends, providing the evolutionary update and the training settings used for larger datasets."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"This supplies the feedback-alignment learning-rule implementations used as candidates in the search space."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"This work defines the sign-concordant feedback rules that the mixed-rule search space relies on."},{"cited_title":"Dong and Y","cited_arxiv_id":null,"evidence_quote":"This provides the ImageNet16-120 benchmark used to test the searched architectures."},{"cited_title":"Benchmarking the Accuracy and Robustness of Feedback Alignment Algorithms","cited_arxiv_id":"2108.13446","evidence_quote":"This benchmarking study provides the adversarial robustness context and the single-rule baselines for robustness comparisons."}],"review_version":1}