{"id":"db190f80-fd8b-483a-9354-3e49378868b4","arxiv_id":"2412.04847","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"MTSpark combines active dendrites and dueling in a deep spiking Q-network, reporting strong multi-task RL and classification scores, but the experimental setup may not fairly test the claimed continual-learning benefit.","lead":"A spiking neural network that uses active dendrites and a dueling structure is applied to multi-task reinforcement learning, with tests on three Atari games and three image datasets. The paper reports strong performance, but the evaluation protocol does not clearly test catastrophic forgetting, which is the central problem the method claims to solve.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 1 updates on every task's replay buffer at each timestep, so the experiments jointly train on all tasks and never actually test catastrophic forgetting; the central CF claim is therefore untested, and the baseline comparison is confounded by replay.","rationale":"The reader's weakest-assumption analysis identifies exactly the issue that is most load-bearing: Algorithm 1's inner loop over all environments means the network is trained jointly on all tasks, so the experimental setup does not test catastrophic forgetting. This is not a minor methodological quibble; it directly undermines the paper's framing as a solution to catastrophic forgetting and its contribution of being the first SNN-RL multi-task learner. The abstract further overstates results, claiming human-level performance while reporting Breakout 0.6 versus human 31, but the training-protocol issue is more fundamental because no amount of number-fixing would make the current experiment a valid sequential-learning study. The same protocol also makes the baseline comparison unfair if baselines were trained sequentially, as the text implies. For these reasons, the paper's central claim is currently unsupported, and the REJECT verdict stands. If the authors were to rerun the proposed concrete test and show a real forgetting advantage under sequential updates, the verdict could be reconsidered, but as written the evidence cannot be trusted to distinguish MTSpark's mechanism from vanilla experience replay.","tokens_in":12282,"tokens_out":3655,"duration_ms":39928,"concrete_test":"Retrain DQN, DSQN, MTSpark_AD, and MTSpark_ADD with a strictly sequential protocol: delete the per-environment loop in Algorithm 1 (lines 15-17) and instead sample and update only from the currently active environment's replay buffer, switching environments every 25 episodes as described. If MTSpark retains high performance on earlier tasks while DQN and DSQN forget, the catastrophic-forgetting claim is supported; if all models now perform comparably (or all retain performance), the reported advantage is an artifact of multi-task replay. Additionally, run MTSpark under this sequential protocol with the context signal set to all zeros (active dendrites disabled) to check whether dendritic modulation is the actual cause of any retention.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that MTSpark overcomes catastrophic forgetting when learning multiple tasks subsequently, enabled by active dendrites and context signals. But Algorithm 1, lines 14-18, performs a gradient step on every environment's replay buffer at every timestep, regardless of which environment the agent is currently interacting with. This is joint multi-task training with experience replay, not sequential learning: the network continually sees all tasks, so forgetting is structurally prevented. The paper's own description in Section 3.3 says the network is trained on each environment for P episodes before switching, yet the inner loop makes the actual updates multi-task. Thus the experiments cannot demonstrate any catastrophic-forgetting advantage for MTSpark. Moreover, the comparison with DQN and DSQN is confounded: the paper says these baselines were evaluated when learning tasks subsequently, but their training protocol is not reported. If the baselines used a sequential update while MTSpark uses all-task replay, the performance gap is explained by replay, not by active dendrites. The image-classification ablations in Section 5.3.3 also lack a no-replay/no-dendrite control, so they do not isolate the mechanism. The load-bearing assumption — that the results reflect forgetting prevention — is not supported by the stated algorithm.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes MTSpark, a spiking Q-network architecture that combines active dendrites with a dueling structure for multi-task reinforcement learning. The authors claim that task-specific context signals modulate dendritic segments, forming specialized sub-networks that can learn multiple Atari games without catastrophic forgetting, and also report accuracy improvements over DNN baselines on MNIST, Fashion-MNIST, and CIFAR-10. The evaluation compares MTSpark variants against DQN, DSQN, and their dueling versions over 250 episodes on Pong, Breakout, and Enduro, and against a DNN on classification tasks, along with ablations on two- and three-task classification scenarios.","tokens_in":12556,"tokens_out":5896,"duration_ms":55828,"significance":"If the results were reliable, showing that a single spiking network with active dendrites and dueling can learn multiple Atari tasks at human level would be a valuable step for energy-efficient generalist agents, particularly because the parameter overhead is negligible (Table 5). The paper also provides a clear algorithmic description and detailed architecture configurations. However, the central claim about catastrophic forgetting is not tested by the proposed training protocol, and several reported results are internally inconsistent or overstated. The significance of the contribution is therefore not established by the current evidence.","major_comments":[{"comment":"Algorithm 1 (lines 14-18) samples and updates from every environment's replay buffer at each timestep, so the network is trained jointly on all tasks; the text in Section 3.3 saying the network is trained on each environment for P episodes before switching is inconsistent with the algorithm. Consequently, the experiments never test catastrophic forgetting, which is the paper's central motivation. The comparison with DQN and DSQN is also confounded because their training protocol is not reported; if they were trained sequentially, the performance gap could be explained by replay alone rather than by active dendrites.","section":"Section 3.3, Algorithm 1"},{"comment":"The abstract states that MTSpark reaches human-level performance on Breakout with a score of 0.6 against the human score of 31, but Table 4 and Section 5.1 show that no model, including MTSpark ADD, approaches the human score in Breakout (the best reported score is DQN at 3.2). This is a direct misrepresentation of the reported results and overstates the central experimental claim.","section":"Abstract and Table 4"},{"comment":"The Key Results in the Introduction report the state-of-the-art MNIST accuracy as 33.9%, while Section 5.2 and Figure 8 report the DNN baseline reaching approximately 57% accuracy on MNIST. No explanation reconciles these numbers, so the claimed superior accuracy of MTSpark over the baseline is ambiguous and the abstract's accuracy comparisons are not reproducible from the text.","section":"Section 1.3 vs Section 5.2"},{"comment":"All RL and classification results are reported as single runs without error bars, confidence intervals, or multiple seeds. Given the known high variance of Atari RL and the small gaps between some models (e.g., Enduro 371.2 vs 362.2, Pong -5.4 vs -9.4), the statistical significance of the claimed improvements is not established, and the results cannot be considered reliable evidence for the superiority of MTSpark.","section":"Section 5 (all experiments)"},{"comment":"The ablation study compares MTSpark, which uses spiking neurons with active dendrites, against a DNN with ReLU activations, without a spiking no-dendrite control and without specifying whether training was sequential or joint. This confounds the effect of active dendrites with architecture differences and replay strategy, so the mechanism claimed for the improvements is not isolated.","section":"Section 5.3.3"}],"minor_comments":[{"comment":"The replay buffer size is listed as '220' in Table 3 and in the text; this should be clarified as 2^20 or the correct numeric value.","section":"Table 3"},{"comment":"In the advantage branch, the row 'Non-Spiking Neuron 512 ×1 18 ×1' appears to be a typo; the input dimension should be 18×1, not 512×1.","section":"Table 2"},{"comment":"There is a typo in the sentence preceding Equation 1: 'whise behavior' should be 'whose behavior'.","section":"Section 3.1"},{"comment":"There are typos in this subsection: 'convetional' should be 'conventional' and 'Fahion' should be 'Fashion'.","section":"Section 5.3.3"},{"comment":"The membrane threshold V_th and the number of simulation timesteps are not specified anywhere, which are essential hyperparameters for reproducing spiking network results.","section":"Section 3.1 and Table 3"}],"recommendation":"reject","confidential_remarks":"The paper overclaims in the abstract and key results relative to its own tables; the conflicting MNIST numbers and the Breakout claim should be checked by the editors. The central evaluation protocol issue (Algorithm 1 joint replay) is the main reason for rejection, as it invalidates the catastrophic-forgetting comparison."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a decent engineering idea wrapped in a broken evaluation. The new part is applying context-modulated active dendrites to a dueling spiking Q-network for multi-task RL, and the parameter overhead is tiny. The authors also show the resulting network gets better balanced scores on three Atari games than the DQN/DSQN baselines they implemented, and it does fine on image classification. That part is real and worth a look.\n\nBut the headline claim — that MTSpark overcomes catastrophic forgetting — is untested. Algorithm 1 does a gradient step on every task's replay buffer at every timestep. That's joint training with replay, not sequential learning. Forgetting is structurally prevented, so the experiments can't show any advantage in that regard. The text in Section 3.3 says the network trains on each environment for P episodes before switching, which contradicts the algorithm. If the baselines were trained sequentially while MTSpark saw all tasks every step, the comparison is confounded by replay, not active dendrites. That's a load-bearing flaw.\n\nThere are also smaller consistency problems. The abstract claims human-level performance on Breakout, but the reported MTSpark score is 0.6 vs the human 31 — that's not human-level. The MNIST numbers don't line up either: the ablation says the DNN gets around 57% on MNIST, while the key results say the state-of-the-art gets 33.9%. And there are no error bars or multiple seeds anywhere.\n\nOn the positive side, the paper engages with the active-dendrite literature and the parameter counts are honestly reported. The architecture itself might be useful for someone who wants a cheap multi-task spiking net. But as written, the central scientific claim is not supported.\n\nMy recommendation: this deserves a serious referee, not a desk reject, because the combination is novel and the flaws are fixable with a redesigned evaluation. But the reviewers should send it back for a major revision — sequential training protocol, proper baseline setup, and corrected reporting — before it's publishable.","headline":"A useful combination of active dendrites and dueling in a spiking Q-network, but the catastrophic-forgetting claim is untested because the algorithm jointly trains on all tasks at every timestep.","tokens_in":13054,"tokens_out":5332,"would_cite":false,"duration_ms":48587,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MTSpark enables multi-task learning in spiking Q-networks by gating neurons with task-specific context signals.","keywords":["spiking neural networks","multi-task learning","reinforcement learning","catastrophic forgetting","active dendrites","deep Q-network","dueling architecture","continual learning"],"falsifier":"Train MTSpark with a strictly sequential protocol in which the inner loop over environments (lines 14-18 of Algorithm 1) samples only the current task's replay buffer, and compare Pong, Breakout, and Enduro scores with those reported; if the scores fall to baseline levels, the reported multi-task performance is explained by joint replay, not by active-dendrite gating.","tokens_in":12111,"feed_emoji":"🕹️","tokens_out":7546,"duration_ms":71448,"temperature":0.7,"pith_summary":"This paper proposes MTSpark, a method for training a single spiking neural network on multiple reinforcement-learning tasks without forgetting earlier ones. The key idea is to give each spiking neuron active dendrites that read a one-hot task-identity signal, so the network routes each task through its own sub-network while sharing most parameters. On three Atari games (Pong, Breakout, Enduro) the method reports scores close to or above human levels and clearly higher than DQN and DSQN baselines, and it also beats a conventional DNN on MNIST, Fashion-MNIST, and CIFAR-10 classification. The authors claim this is the first demonstration of multi-task learning in spiking neural networks under a reinforcement-learning setting, a step toward energy-efficient generalist agents.","feed_headline":"One spiking network takes on three Atari games at once","feed_subtitle":"Task-identity signals route each game through its own sub-network, beating DQN and DSQN baselines.","key_machinery":"The central mechanism is the integrate-and-fire neuron with active dendrites: each dendritic segment $j$ has weights $d_j$ and reads the context signal $c$ (one-hot task identity), and the membrane potential updates as $V(t) = V(t-\\Delta t) + f(\\sum_i s_i(t), \\max_j(d_j^T c))$, with $f(\\sum_i s_i, \\max_j(d_j^T c)) = \\sum_i s_i \\cdot \\sigma(\\max_j(d_j^T c))$. The max over dendrites makes each neuron selectively sensitive to the task identity, gating incoming spikes and forming per-task sub-networks. On top of this, the dueling structure in the MTSpark ADD variant computes a state value and per-action advantages separately before combining them into Q-values, which the paper argues improves action evaluation across tasks.","core_discovery":"The paper's central discovery, stated on its own terms, is that context-signal-driven active dendrites let a deep spiking Q-network hold several tasks in one parameter set: when the one-hot task vector $c$ is fed into dendritic segments, the modulating function $f(\\sum_i s_i(t), \\max_j(d_j^T c))$ gates presynaptic spikes before they reach the membrane potential, so the network develops task-specific spiking patterns. Adding a dueling structure (separate state-value and advantage streams) on top of this gives the MTSpark ADD variant, which the paper reports achieves $-5.4$ in Pong, $0.6$ in Breakout, and $371.2$ in Enduro, outperforming DQN, DSQN, and their dueling variants. The same architecture also reaches 97.5% (MNIST), 86.4% (Fashion-MNIST), and 56% (CIFAR-10) accuracy, above the DNN comparison. The authors take these results as evidence that spiking networks can be made into generalist agents without growing the network.","pith_inferences":["Ablating the context signal (feeding the same vector for all tasks) would test how much of the reported performance depends on task identity gating rather than on the network's shared feature extractor.","Because the reported training loop samples from every task's replay buffer at each timestep, a strictly sequential version of the same algorithm would separate the contribution of active dendrites from the contribution of joint rehearsal.","The one-hot context signal presumes the task identity is known in advance; replacing it with a learned embedding or an inferred task descriptor would be a natural step toward task-agnostic multi-task RL.","The results cover three Atari games and three image sets; extending the evaluation to continuous-control or sparse-reward environments would map the limits of the gating approach."],"forward_implications":["A single spiking network can serve multiple RL tasks without adding task-specific parameters, since the one-hot context signal alone routes behavior.","The parameter counts of MTSpark AD (1,693,691) and MTSpark ADD (3,300,357) are nearly identical to their non-dendritic counterparts, so the multi-task capability does not come from added memory.","If the reported scores hold, spiking generalist agents become plausible for edge and robotic settings where energy-efficient multi-task adaptation is needed.","The dueling structure can be added on top of other spiking RL architectures to improve multi-task performance, as the comparison between MTSpark AD and MTSpark ADD suggests.","The same context-gating principle transfers to supervised image classification, indicating the mechanism is not specific to reinforcement learning."],"supporting_citations":[{"why":"The deep spiking Q-network baseline that MTSpark extends to the multi-task setting.","marker":"[3]"},{"why":"The active-dendrites mechanism for multi-task learning that MTSpark ports into spiking neurons.","marker":"[6]"},{"why":"The DQN baseline and source of the human-level Atari scores used for comparison.","marker":"[16]"},{"why":"Prior work showing active dendrites in time-to-first-spike SNNs for continual learning, motivating the spiking implementation.","marker":"[18]"},{"why":"The dueling network architecture that MTSpark ADD uses to separate state value and action advantages.","marker":"[34]"}],"fun_headline_variants":["Spiking network learns three Atari games with shared parameters","MTSpark: multi-task spiking RL with specialized sub-networks","Context-gated spiking neurons power multi-task game playing","One spiking net beats DQN on Pong, Breakout, Enduro","Energy-efficient spiking Q-network achieves generalist agent"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that MTSpark prevents catastrophic forgetting assumes the benchmark actually presents tasks sequentially, but the training loop samples from all tasks' replay buffers at every timestep (Algorithm 1, lines 14-18), so the network trains on all tasks jointly and forgetting is never directly measured.","fun_headline_variants_meta":{"raw":{"variants":["Spiking network learns three Atari games with shared parameters","MTSpark: multi-task spiking RL with specialized sub-networks","Context-gated spiking neurons power multi-task game playing","One spiking net beats DQN on Pong, Breakout, Enduro","Energy-efficient spiking Q-network achieves generalist agent"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000786,"raw_usage":{"total_tokens":3552,"prompt_tokens":1114,"completion_tokens":2438,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":730,"completion_tokens_details":{"reasoning_tokens":2349}},"tokens_in":730,"tokens_out":2438,"duration_ms":18224,"temperature":1.0,"reasoning_tokens":2349,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T21:12:46.833312+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train MTSpark with a strictly sequential protocol in which the inner loop over environments (lines 14-18 of Algorithm 1) samples only the current task's replay buffer, and compare Pong, Breakout, and Enduro scores with those reported; if the scores fall to baseline levels, the reported multi-task performance is explained by joint replay, not by active-dendrite gating.","supporting_citations":[{"cited_title":"Avoiding catastrophe: Active dendrites enable multi-task learning in dynamic environments","cited_arxiv_id":null,"evidence_quote":"The active-dendrites mechanism for multi-task learning that MTSpark ports into spiking neurons."},{"cited_title":"Active dendrites enable efficient continual learning in time-to-first-spike neural networks, 2024","cited_arxiv_id":null,"evidence_quote":"Prior work showing active dendrites in time-to-first-spike SNNs for continual learning, motivating the spiking implementation."},{"cited_title":"Dueling network architectures for deep reinforcement learning","cited_arxiv_id":null,"evidence_quote":"The dueling network architecture that MTSpark ADD uses to separate state value and action advantages."}],"review_version":1}