{"id":"b94bd099-4739-4037-98f4-7e2c580c40d9","arxiv_id":"2506.17169","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A columnar spiking neural network with local learning rules reaches 92% average accuracy and about 4% forgetting on ten permuted-MNIST tasks, but only when tasks share no features.","lead":"This paper tests a column-based spiking neural network with local learning rules on sequential image-classification tasks. The network forgets little when tasks are made unrelated by random pixel shuffling, but it forgets heavily when tasks share visual features.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The Permuted MNIST result is a selected, not predicted, outcome: alpha is tuned on the same ten test tasks, so the reported 92%/4% figures may be optimistic.","rationale":"I agree with the reader that the method's success is tied to task-disjointness, and the paper's own MNIST/EMNIST results make that limitation explicit. However, the more load-bearing issue for the specific headline claim is the lack of a held-out task split during hyperparameter selection. The paper describes choosing α by inspecting AA and FM on the same ten Permuted MNIST tasks, which means the reported numbers are the best selected outcome, not an unbiased estimate for a new sequence of permuted tasks. This does not require suspecting the authors of anything improper; it is a standard model-selection gap that the manuscript does not address. The open-source code and the explicit Section 6 limitation are real strengths, and the conditional verdict is appropriate. I would keep the verdict as CONDITIONAL, contingent on a nested validation demonstration and on reporting variance over fresh task permutations.","tokens_in":12634,"tokens_out":12741,"duration_ms":121538,"concrete_test":"Perform a nested model-selection check: generate ten random permutations as a development set, run the same α sweep (and virtual-synapse sweep if used) on them, pick the best configuration by the paper's criterion, then freeze it and evaluate on ten fresh test permutations. If the final average accuracy or first-task forgetting on the fresh permutations differs from 92.31%/1.19 by more than the across-seed standard error in Table 7, the headline result is partly an artifact of tuning on the test tasks. This requires only the open-source ArNI-X code and roughly the reported per-run cost for 45 microcolumns.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central Permuted MNIST result is a selected, not predicted, outcome. In Section 5.1, stages two and three sweep α (0 to 0.023817, then selected values) and choose the configuration by inspecting the resulting average-accuracy and forgetting curves on the same ten tasks used for the reported tables (Figure 4; 'We found the optimal α value ... to be 0.01'). No validation split or separate permutation set is described, so Table 5 and the Table 3 45-microcolumn row (AA-like 92.31, FM 1.19) are the best point of that sweep rather than an unbiased estimate on a new task sequence. The standard deviations in Table 7 are at a fixed, already-selected α and do not account for selection across hyperparameters. Task-disjointness is honestly acknowledged in Section 6, but even within the disjoint-task regime the headline 'maintaining 92% accuracy' and 'only 4% performance degradation' statements may overstate performance because the test tasks themselves were used for model selection.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes CoLaNET, a columnar spiking neural network with local learning rules, and evaluates it on continual learning benchmarks. The experiments cover an online streaming task-based domain-incremental setting with ten Permuted MNIST tasks and a two-task MNIST/EMNIST protocol, comparing against a one-layer ANN baseline and several existing continual learning methods. The paper reports that CoLaNET, with tuned hyperparameters, achieves about 92% average accuracy and low forgetting on Permuted MNIST, while acknowledging that this success depends on the tasks lacking shared feature structure; on MNIST/EMNIST, forgetting is much higher (43% and 32%). The source code is made available.","tokens_in":12820,"tokens_out":11013,"duration_ms":88215,"significance":"If the quantitative claims were reproducible, the paper would make a useful contribution to biologically plausible continual learning: it demonstrates a specific architectural mechanism (columnar separation plus adaptive threshold and synaptic renormalization) for trading off stability and plasticity without backpropagation or replay. The paper is honest about the restricted regime in which the method works, explicitly stating in Section 6 that Permuted MNIST results do not generalize to tasks with shared features. The open-source code and the detailed degradation profiles are strengths. However, the central quantitative claim is currently undercut by numerical inconsistencies between the aggregation table and the detailed profiles, and by in-sample hyperparameter selection, so the significance cannot be fully assessed until these issues are resolved.","major_comments":[{"comment":"The metrics in Table 3 are not consistent with the corresponding degradation profiles. For the 45-microcolumn configuration, the final row of Table 5 yields an average accuracy of 91.76% and a forgetting measure of 1.27, whereas Table 3 lists AA=92.31 and FM=1.19, and Table 7 reports AA=92.39±0.13 and FM=0.94±0.17. Moreover, the AIA values in Table 3 (e.g., 30.02 for the 45-microcolumn row) are implausibly low for the definition in Eq. (2), which should produce values in the 90s. Please clarify whether Table 3 aggregates multiple independent runs and, if so, provide the run-level data; otherwise correct the tables and the abstract, which cite these numbers.","section":"§5.1, Tables 3–5"},{"comment":"The hyperparameter α is selected by sweeping values and inspecting the resulting AA and FM on the same ten Permuted MNIST tasks that are later reported (Figure 4; \"We found the optimal α value ... to be 0.01\"). No validation split or held-out task sequence is described. Consequently, the headline \"92% accuracy\" and \"4% degradation\" are the best point of an in-sample model-selection procedure, not an unbiased estimate for a new task sequence. The standard deviations in Table 7 are conditional on the selected α and do not capture the selection. Please add a validation procedure or report the full sensitivity of the results across a reasonable α range, and temper the abstract accordingly.","section":"§5.1, hyperparameter selection"},{"comment":"The statement \"maintaining 92% accuracy on each\" overstates the results. In Table 5, after training on all ten tasks, task 1 accuracy is 88.95% and task 10 is 91.22%, so not every task reaches 92%. The average accuracy across tasks is 91.76% in that run (or 92.31% as stated in Table 3). Please rephrase to \"an average accuracy of about 92%\" or give the exact minimum per-task accuracy, and clarify the \"only 4% performance degradation\" figure, which is 4.35 percentage points on task 1 in Table 5.","section":"Abstract and §5.1"}],"minor_comments":[{"comment":"The row label \"45 microcolumns, α = 0.1\" contradicts the text in §5.1, which states that the optimal α for the 45-microcolumn configuration is 0.01; please correct this typo.","section":"Table 3"},{"comment":"Reference [12] lists the arXiv identifier as \"arXiv preprint arXiv:12403.05175\"; this appears to be a typo, likely \"2403.05175\".","section":"References"},{"comment":"The caption does not define the abbreviation WTA; please add a short explanation of the winner-take-all mechanism for readers unfamiliar with the CoLaNET architecture.","section":"Figure 1"},{"comment":"The column heading \"Iterations\" is used to denote training steps on successive tasks; consider renaming it to \"Tasks\" or \"Training stage\" for clarity.","section":"Tables 1–5"},{"comment":"The reported AIA values in Table 3 are not computed according to Eq. (2), which defines AIA as the average of AA over task indices; please verify that the metric values in Table 3 are correct and consistent with the definitions.","section":"Equation (2)"}],"recommendation":"major_revision","confidential_remarks":"The paper presents an interesting architecture and an honest discussion of its limitations, but the internal numerical inconsistencies in Table 3, the absence of a validation split for hyperparameter selection, and the discrepancy between Table 5 and the abstract's claims are serious enough that I cannot recommend acceptance without major revisions. I suggest requesting the raw run-level data or per-seed results to verify the reported metrics."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This paper is a legitimate extension of the authors' CoLaNET architecture to continual learning, and it is the first evaluation of CoLaNET in that setting. The honest takeaway: columnar SNNs with local learning can sidestep catastrophic forgetting on Permuted MNIST, but mostly when tasks are disjoint. On MNIST→EMNIST, where features overlap, it forgets 43% and 32% respectively, which the authors acknowledge in Section 6. That candid limitation is the strongest part of the paper.\n\nWhat's genuinely new: the ten-task Permuted MNIST degradation profile, the two-task MNIST/EMNIST transfer, and a hyperparameter analysis of α and ns. The code is open-source, the architecture descriptions are clear, and the heatmaps are a useful diagnostic. The comparison with replay methods in Table 7 gives context, even if the comparison isn't perfectly apples-to-apples.\n\nNow the soft spots. The stress-test note is right: α is selected by inspecting AA/FM curves on the same ten tasks used for the reported tables. No validation split or separate permutation set is described. So the 92%/4% figures are the best point of a manual sweep, not a prediction on unseen tasks.\n\nMore concerning, the tables don't agree with each other. Table 3's metrics don't match Tables 4 and 5. For the 15-microcolumn α=0.005 row, Table 3 lists FM 2.88, but Table 4 shows first-task forgetting of 12.6 points. For the 45-microcolumn row, Table 3 says α=0.1 while the text and Table 5 use α=0.01, and the corresponding AA and FM values differ. Table 7's averaged CoLaNET numbers differ again. This is fixable, but it undermines confidence in the reported numbers.\n\nAlso, the abstract says \"maintaining 92% accuracy on each,\" but after all ten tasks, task 1 sits at 88.95 and task 8 at 91.43. That's an overstatement.\n\nWho this is for: people working on neuromorphic continual learning, especially those interested in biologically plausible local learning rules. They'll find the architecture and the disjoint-task insight useful, but they should treat the precise numbers with caution.\n\nRecommendation: I would send this to peer review. It's a serious empirical study with an important limitation openly discussed, and the core claim—columnar separation helps when tasks lack shared structure—is plausible. A referee should ask for a clean separation between tuned and predicted results, error bars on the headline tables, and a reconciliation of the inconsistent tables. With those fixes it would be a solid contribution to a workshop or a specialized journal.","headline":"A serious but uneven empirical study: CoLaNET avoids catastrophic forgetting on disjoint-task Permuted MNIST, but the headline numbers are tuned on the test tasks and the tables contradict each other.","tokens_in":13347,"tokens_out":9045,"would_cite":true,"duration_ms":74148,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Columnar spiking network holds 92% accuracy across ten sequential tasks.","keywords":["continual learning","catastrophic forgetting","spiking neural network","columnar organization","local learning","anti-Hebbian plasticity","dopamine-modulated plasticity","Permuted MNIST"],"falsifier":"Run CoLaNET on a sustained task sequence with measurable feature overlap between successive tasks, for example MNIST followed by EMNIST as the paper itself does, and compare first-task forgetting: the paper reports 4.35% on permuted tasks but 43% and 32% on the shared-feature pair, so a result near 4% on a shared-feature sequence would contradict the paper's stated scope.","tokens_in":12432,"feed_emoji":"🧠","tokens_out":5475,"duration_ms":51459,"temperature":0.7,"pith_summary":"The paper argues that a spiking neural network organized into cortical-like columns can learn a sequence of tasks without catastrophic forgetting, provided the tasks do not share features. On ten sequentially learned pixel-permuted MNIST tasks, the authors report 92.31% average accuracy and only 4.35% degradation on the first task after learning the other nine, against a 49% drop for a standard one-layer ANN baseline. The same network fails to control forgetting when tasks share features: on MNIST followed by EMNIST letters, forgetting reaches 43%, and 32% in the reverse direction. The contribution is a demonstration that columnar separation of microcolumns, rather than the biological learning rule alone, is what buys continual learning, and that this works precisely when task boundaries coincide with feature boundaries.","feed_headline":"Columnar spiking net holds 92% accuracy across 10 tasks","feed_subtitle":"Its microcolumns isolate each task and cut forgetting to 4%—but only when tasks share no features.","key_machinery":"The central object is a microcolumn: a small group of leaky integrate-and-fire neurons inside a class-specific column that competes via winner-take-all for the right to encode a subclass. Learning is carried by two opposed local rules operating on the same plastic synapses — dopamine-modulated potentiation strengthens synapses that fired shortly before a reward spike, while anti-Hebbian depression weakens synapses that fired without reward — together with an adaptive firing threshold $u_{tr} = u_{const} + \\alpha \\sum_i w_i^+$ that raises a neuron's threshold as its positive weights grow, and optional synaptic renormalization controlled by a virtual-synapse count $n_s$. This machinery lets each microcolumn become a dedicated detector; whether it stays dedicated to one task or remains available for new tasks is governed by $\\alpha$ and $n_s$.","core_discovery":"CoLaNET's microcolumns dedicate themselves to subclasses of a single task: within each class column, winner-take-all competition selects one learning neuron, dopamine-modulated plasticity potentiates the synapses that produced the correct spike, and anti-Hebbian plasticity suppresses synapses that fired without reward. The adaptive firing threshold and synaptic renormalization tune how specialized these microcolumns become. On Permuted MNIST, specialization is high enough that columns trained on earlier tasks stop responding to later permuted inputs, so old accuracies stay nearly frozen; with 45 microcolumns and threshold coefficient $\\alpha = 0.01$, the network keeps learning each new task at roughly 92% accuracy while first-task accuracy falls only from 93.30% to 88.95%. On MNIST/EMNIST, microcolumns trained on digits also respond to similar-looking letters, so the same mechanism produces 43% and 32% forgetting. The paper's claim is that columnar SNNs offer a workable, biologically plausible route to continual learning in the domain-incremental setting where tasks occupy disjoint feature spaces, and that hyperparameters $\\alpha$ and the virtual-synapse count $n_s$ control the stability-plasticity balance.","pith_inferences":["Because the low-forgetting result depends on random pixel permutations destroying shared structure, a natural testable extension is to build task sequences with controlled degrees of feature overlap and measure where forgetting begins to climb; the paper's MNIST/EMNIST numbers suggest the transition is sharp, not gradual.","The same columnar logic might be applied to other local-learning spiking networks: if dedicated microcolumns are what isolate tasks, then any architecture that forms hard winner-take-all cell assemblies could inherit similar stability without the exact dopamine/anti-Hebbian combination.","The authors' proposed hierarchical fix is essentially a prediction that shared low-level features should be learned once in early layers, with columns reserved for the differing deep representations; this could be tested on a single network with a frozen shallow feature extractor and columnar top layers."],"forward_implications":["If the scope condition holds — tasks share no features — CoLaNET reduces first-task forgetting to about 4% over ten tasks while holding average accuracy near 92%, with no replay buffer, no gradient, and no task identities during inference.","The stability-plasticity trade-off is directly visible in two knobs: raising the adaptive-threshold coefficient $\\alpha$ freezes old tasks but starves new ones; raising the virtual-synapse count $n_s$ diverts renormalization and trades accuracy for plasticity.","On the Permuted MNIST benchmark, CoLaNET's 92.39% average accuracy with 0.94% forgetting at 45 microcolumns beats the replay-based methods compared in the paper, including their joint-training reference of 89.05%.","For tasks with shared features, the mechanism does not solve continual learning by itself, and the paper points to a hierarchical architecture as the next step, applying columns only at deep layers where representations are task-specific."],"supporting_citations":[{"why":"Supplies the CoLaNET configuration (15 microcolumns, alpha=0.023817) that the Permuted MNIST and E/MNIST experiments start from.","marker":"[8]"},{"why":"Defines the Permuted MNIST protocol that generates the ten sequential tasks.","marker":"[10]"},{"why":"Previous local-learning SNN study on MNIST->EMNIST with 42% forgetting, the baseline CoLaNET must match, and the motivation for columnar organization.","marker":"[6]"},{"why":"Defines the adaptive threshold mechanism that controls microcolumn specialization and the stability-plasticity balance.","marker":"[7]"},{"why":"Provides the EMNIST Balanced subset with ten letter classes used in the shared-feature two-task experiment.","marker":"[11]"},{"why":"Provides the replay-method comparison (NCC) that CoLaNET outperforms on Permuted MNIST with one epoch per task.","marker":"[24]"},{"why":"Supplies the forgetting measure and backward-transfer metrics used to evaluate memory stability.","marker":"[18]"}],"fun_headline_variants":["CoLaNET spiking net learns 10 tasks with only 4% forgetting","Columnar SNN cuts catastrophic forgetting to 4% over 10 tasks","Biologically plausible spiking net keeps 92% accuracy on 10 tasks","Microcolumn spiking net adapts to each task, forgetting only 4%","CoLaNET: columnar spiking networks for continual learning without forgetting"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline result stands on Permuted MNIST's random permutations making every task feature-disjoint; the moment tasks share visual features, as in the paper's own MNIST/EMNIST run, forgetting jumps to 43% and 32%.","fun_headline_variants_meta":{"raw":{"variants":["CoLaNET spiking net learns 10 tasks with only 4% forgetting","Columnar SNN cuts catastrophic forgetting to 4% over 10 tasks","Biologically plausible spiking net keeps 92% accuracy on 10 tasks","Microcolumn spiking net adapts to each task, forgetting only 4%","CoLaNET: columnar spiking networks for continual learning without forgetting"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000633,"raw_usage":{"total_tokens":2926,"prompt_tokens":955,"completion_tokens":1971,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":571,"completion_tokens_details":{"reasoning_tokens":1879}},"tokens_in":571,"tokens_out":1971,"duration_ms":13759,"temperature":1.0,"reasoning_tokens":1879,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:10:05.565575+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run CoLaNET on a sustained task sequence with measurable feature overlap between successive tasks, for example MNIST followed by EMNIST as the paper itself does, and compare first-task forgetting: the paper reports 4.35% on permuted tasks but 43% and 32% on the shared-feature pair, so a result near 4% on a shared-feature sequence would contradict the paper's stated scope.","supporting_citations":[{"cited_title":"Antonov, K.V","cited_arxiv_id":null,"evidence_quote":"Previous local-learning SNN study on MNIST->EMNIST with 42% forgetting, the baseline CoLaNET must match, and the motivation for columnar organization."},{"cited_title":"Mitigating forgetting in online continual learning with neuron calibration","cited_arxiv_id":null,"evidence_quote":"Provides the replay-method comparison (NCC) that CoLaNET outperforms on Permuted MNIST with one epoch per task."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the forgetting measure and backward-transfer metrics used to evaluate memory stability."}],"review_version":2}