{"id":"5beb1f7d-109e-4fff-85ce-bdd43f2e68cd","arxiv_id":"2412.00278","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"AOT-SNN is extended to regression with heteroscedastic Gaussian and classification-based uncertainty estimates, with mixed benchmark results.","lead":"This paper adapts an existing spiking neural network framework (AOT-SNN) to regression, using two known uncertainty estimation approaches. The proposed methods give event-driven models predictive means and variances, but reported performance gains are inconsistent and some experimental details are missing.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The NLL metric for the RAC model is never defined, so Table II's uncertainty comparisons to Gaussian DNN baselines are unverifiable; a toy-figure inconsistency heightens this concern.","rationale":"The reader's weakest assumption focused on the independence of time-step outputs in the AOT-SNN variance formula. That is a legitimate theoretical nuance, but it is not the most load-bearing issue for the paper's central empirical claim. The mixture-variance formula (Eq. 5 analog) is valid for any mixture distribution regardless of component independence, and the experimental NLL results could still support the claim even if the time steps are correlated. The truly load-bearing vulnerability is that the paper's primary uncertainty metric—NLL—is never defined for the RAC model. Without knowing whether NLL is computed as a continuous density or as a discrete bin probability, the comparisons in Table II cannot be interpreted or reproduced. This is compounded by the unspecified q hyperparameter in the distance loss and the direct contradiction between the text and Fig. 3 on the toy dataset. Together these issues mean the strongest claim is not currently verifiable. However, all are addressable by the authors: they can provide the exact NLL formula, disclose q, and correct the toy-dataset description. Therefore the appropriate verdict remains CONDITIONAL (unchanged from the reader's verdict), contingent on these clarifications and corrections.","tokens_in":9112,"tokens_out":7578,"duration_ms":65945,"concrete_test":"Recompute the RAC NLL values in Table II using the explicit continuous-density formula -log f_k = -log p_k + log(b_{k+1}-b_k), with the actual bin boundaries used for each dataset. If the recomputed NLLs differ from the reported values beyond rounding, the central claim is not supported. Also re-run the toy experiment described in Section IV-A and check whether the SNN Gaussian NLL is truly 2.59 versus DNN 2.48; if so, amend the text in Section IV-A to reflect that only RMSE improves, not NLL.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that both AOT-SNN methods achieve uncertainty performance on par with state-of-the-art DNN approaches—rests primarily on the NLL comparisons in Table II. Yet the paper never specifies how NLL is computed for the RAC method. Section III-C defines the predictive density as piecewise uniform: f_k = p_k/(b_{k+1}-b_k) (Eq. 8). The correct continuous NLL for a point y in bin k is therefore -log f_k = -log p_k + log(b_{k+1}-b_k). If the authors instead used the discrete cross-entropy -log p_k, the resulting values are not continuous densities and are not comparable to the Gaussian NLLs reported for MC-dropout and Deep Ensembles. The omission makes Table II impossible to verify or reproduce. Additionally, the distance loss in Eq. 14 includes a hyperparameter q>0, but the experiments never state its value; the text says only that the toy model used grid search and the benchmark models used τ=1, leaving q unspecified. Finally, Section IV-A claims the Gaussian-based AOT-SNN 'significantly outperforms its DNN counterpart in both RMSE and NLL', but Fig. 3 shows SNN Gaussian NLL (2.59) is worse than DNN Gaussian NLL (2.48). This internal inconsistency suggests the reported comparisons may contain errors, undermining confidence in the headline result.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper adapts the Average-Over-Time SNN (AOT-SNN) framework to regression tasks for uncertainty estimation. Two methods are proposed: a heteroscedastic Gaussian approach, where each time step outputs a mean and variance that are combined across time steps, and a Regression-as-Classification (RAC) approach, where the target is discretized into bins and a piecewise-uniform predictive density is formed from softmax probabilities. The methods are evaluated on a toy dataset and eight UCI benchmark datasets, with claims that the proposed SNN models achieve uncertainty performance comparable to or better than MC-dropout and deep ensemble DNN baselines. The paper also describes a distance loss with entropy regularization for training the RAC models.","tokens_in":9435,"tokens_out":4215,"duration_ms":38642,"significance":"If the reported results are reliable, the paper would provide a principled and efficient way to obtain predictive uncertainty from event-driven spiking networks in regression settings, an area with comparatively few existing solutions. The conceptual extension of AOT-SNN to regression and the use of RAC with a continuous piecewise-uniform density are reasonable and align with recent work on regression-as-classification. The experimental evaluation covers standard benchmarks and includes both accuracy (RMSE) and uncertainty quality (NLL). However, the central claim is currently hampered by a missing definition of the NLL metric for RAC, an internal inconsistency in the toy experiment, and unspecified hyperparameters, which together prevent verification of the headline result. The derivations of the predictive mean and variance are straightforward applications of the mixture variance formula and are not circular.","major_comments":[{"comment":"The manuscript never defines how NLL is computed for the RAC model. Equation (8) defines the predictive density as piecewise uniform, f_k = p_k/(b_{k+1}-b_k); the continuous NLL for a point y in bin k is therefore -log f_k = -log p_k + log(b_{k+1}-b_k). If instead the discrete cross-entropy -log p_k was used, those values are not continuous densities and are not comparable to the Gaussian NLL values reported for MC-dropout and Deep Ensembles. Please state the exact formula used to compute every NLL value in Table II, and correct the table if necessary.","section":"Section III-C and Table II"},{"comment":"The text states that the Gaussian-based AOT-SNN 'significantly outperforms its DNN counterpart in both RMSE and NLL', but Figure 3 reports SNN Gaussian NLL = 2.59 versus DNN Gaussian NLL = 2.48, so the SNN is worse on NLL. The figure caption also claims the Gaussian-based AOT-SNN 'shows improved RMSE and NLL', which is contradicted by the displayed numbers. This internal inconsistency in a core illustrative experiment undermines confidence in the reported comparisons; please correct the text or the figure and recheck the benchmark tables for analogous errors.","section":"Section IV-A and Figure 3"},{"comment":"The distance loss in Eq. (14) depends on the hyperparameter q>0, but its value is never stated for any experiment. The toy-experiment paragraph mentions grid search and the benchmark paragraph specifies tau=1, yet q is omitted entirely. Without q, the RAC training loss is not reproducible. Please report the chosen q values or the selection procedure for both the toy and benchmark experiments.","section":"Section III-C.2 and Section IV"},{"comment":"The abstract and conclusion claim that both AOT-SNN methods achieve uncertainty performance 'on par with' or 'often surpassing' state-of-the-art DNN approaches, but Table II shows the Gaussian AOT-SNN NLL is substantially worse than MC-dropout on Boston housing (2.74 vs 2.46), Kin8nm (-1.13 vs -0.95), and Power plant (3.19 vs 2.80), and the RAC NLL is worse than MC-dropout on Kin8nm (-0.75 vs -0.95). The paper does not provide a quantitative criterion for 'on par' nor statistical significance tests. Please qualify the claim with respect to the datasets that actually support it, or report error bars and tests that substantiate the stated level of performance.","section":"Abstract, Section V, and Tables I-II"}],"minor_comments":[{"comment":"The variance combination formula is presented as analogous to MC-dropout and deep ensembles; please clarify that it is a mixture variance across time steps, not a variance over independent samples, since membrane potentials in an SNN are temporally correlated and the independence assumption is not stated or tested.","section":"Section III-B"},{"comment":"The sentence 'Both SNN models set their dropout rate as 0.05' and 'optimized via the grid search' is ambiguous: please specify which hyperparameters (q, K, tau, dropout rate) were grid-searched and which were fixed for the toy experiment.","section":"Section IV-A"},{"comment":"There are several typos and grammatical errors, e.g., 'the the mean' in the Figure 1d caption, 'can often outperforms' in Section II-C, and 'solutions' should be 'solution' in Section V. Please proofread the manuscript.","section":"Throughout"},{"comment":"The Gaussian models are trained for 600 epochs while the RAC models are trained for 200 epochs; this difference could affect the comparison and should be justified or the training lengths should be matched.","section":"Section IV-B"},{"comment":"The caption of Table II does not state the units or the exact definition of NLL; given that Gaussian NLL can be negative and RAC NLL depends on bin width, please add a note specifying how each NLL value was computed.","section":"Table II"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses an interesting and timely problem, and the proposed methods are a natural extension of the authors' prior AOT-SNN framework. However, the missing NLL definition for RAC and the toy-figure inconsistency are serious reproducibility concerns that must be resolved before the central claim can be accepted. The omission of the q hyperparameter is easily fixable. I recommend major revision rather than rejection because the core approach is defensible and the issues appear addressable within the scope of the manuscript."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a legitimate extension of the AOT-SNN framework to regression, with two sensible methods, but the paper's headline claims are undercut by a missing definition of the NLL metric for the RAC model and a misreported toy result. If those are fixed, it's a solid subfield contribution.\n\nWhat's actually new: applying AOT-SNNs to regression with heteroscedastic Gaussian and RAC outputs. The variance combination across time steps is a correct use of the law of total variance, and the RAC predictive-mean derivation (Eqs. 9-13) is correct. The empirical study covers a toy problem plus eight UCI benchmarks. The RAC-based AOT-SNN shows real gains in RMSE on Concrete, Energy, and Protein, and several NLL numbers beat the published MC-dropout baselines. That is real evidence the approach works.\n\nWhere it gets soft. First and most important: the NLL for RAC is never defined. Eq. 8 gives a piecewise-uniform density, so the proper continuous NLL for a point in bin k is -log p_k + log(bin width). If the authors instead used the discrete cross-entropy -log p_k, the Table II entries are not densities and cannot be compared to the Gaussian NLLs. There is no code, and the exact metric is absent, so Table II is not reproducible as reported. This is a real problem for the central claim. Second, the distance loss in Eq. 14 has a hyperparameter q>0 that never appears in the experimental write-up; the text only says tau=1. That is minor but sloppy. Third, the text says the Gaussian AOT-SNN \"significantly outperforms its DNN counterpart in both RMSE and NLL\" on the toy set, but Figure 3 shows SNN NLL 2.59 versus DNN NLL 2.48, i.e. worse. A one-line misdescription, but it makes you wonder about other numbers.\n\nOne conceptual caveat: treating the T time-step outputs as independent MC-dropout samples is an approximation given the recurrent dynamics. It's a workable assumption and the empirical results are decent, but it is untested. I'd call that minor, not a flaw.\n\nWho is this for? People working on neuromorphic regression and SNN uncertainty. It will not reshape broader ML, but it fills a gap. The derivations are sound and the benchmarks are standard.\n\nRecommendation: send it to peer review, not desk reject. A referee can and should demand the RAC NLL definition, the q value, and a corrected toy description. If the authors supply those and dial back the \"surpassing\" language, this is a publishable workshop or specialized-journal paper.","headline":"A legitimate AOT-SNN regression extension whose headline comparison is undercut by an undefined RAC NLL metric and a misreported toy result.","tokens_in":9920,"tokens_out":5570,"would_cite":false,"duration_ms":49346,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Spiking neural networks can estimate regression uncertainty on par with deep nets.","keywords":["spiking neural networks","uncertainty estimation","regression","average-over-time","heteroscedastic Gaussian","regression-as-classification","MC-dropout","energy efficiency"],"falsifier":"Take a trained AOT-SNN, feed the same input many times, and compute the autocorrelation of the per-time-step outputs; then compare the predictive variance from the mixture formula with the variance obtained from truly independent forward passes that reset membrane potentials each time. If the independent-pass version gives systematically different NLL or reliability-diagram coverage, the time-average-as-MC-dropout equivalence is not valid.","tokens_in":8927,"feed_emoji":"⚡","tokens_out":8610,"duration_ms":68556,"temperature":0.7,"pith_summary":"The paper sets out to give spiking neural networks (SNNs) the same ability to estimate prediction uncertainty in regression that deep neural networks (DNNs) have, without paying the cost of repeated forward passes. It adapts the Average-Over-Time SNN (AOT-SNN) framework to two regression setups: one where each time step predicts a mean and variance under a heteroscedastic Gaussian assumption, and one based on Regression-as-Classification (RAC), where the target range is binned and probabilities over bins are averaged over time. On a toy dataset and eight UCI benchmarks, the reported experiments show both methods reaching error and negative-log-likelihood values comparable to state-of-the-art DNN baselines (MC-dropout and deep ensembles), with the RAC variant improving RMSE notably on Concrete Strength, Energy Efficiency, and Protein Structure datasets. The efficiency gain comes from replacing many stochastic forward passes with a single forward pass whose time steps supply the samples to average.","feed_headline":"Spiking nets match deep nets in regression uncertainty","feed_subtitle":"Two AOT-SNN methods give event-driven regressors calibrated error bars with a single forward pass.","key_machinery":"The central mechanism is the Average-Over-Time trick: an SNN whose output layer is a set of non-spiking readout integrator neurons, so the membrane potential at each time step $t$ is treated as one stochastic prediction, and the predictive distribution is the average over the $T$ time steps of a single forward pass. Two output parameterizations plug into this machinery: (i) a two-neuron readout emitting $\\mu_t(x)$ and $\\sigma_t^2(x)$ at every time step, combined through the deep-ensemble formula of Eq. 5, and (ii) a $K$-neuron readout for binned targets, whose time-averaged softmax probabilities become a continuous density through the uniform-within-bin assumption. The training objectives are the negative log-likelihood for the Gaussian variant and, for the RAC variant, a distance loss $L_{dis} = \\sum_k |k-j|^q p_k$ with entropy regularization $\\tau H$ that keeps the predictive distribution neither overconfident nor misplaced.","core_discovery":"The core claim is that uncertainty for regression can be estimated efficiently in event-driven spiking networks by reusing their inherent time-step structure: instead of running many forward passes, one forward pass with $T$ time steps produces $T$ predictions whose average approximates what MC-dropout obtains with multiple passes. In the Gaussian variant, each time step $t$ emits a mean $\\mu_t(x)$ and a variance $\\sigma_t^2(x)$, and the predictive distribution is assembled as $\\mu_*(x) = \\frac{1}{T}\\sum_t \\mu_t(x)$ and $\\sigma_*^2(x) = \\frac{1}{T}\\sum_t (\\sigma_t^2(x)+\\mu_t^2(x)) - \\mu_*^2(x)$, the standard ensemble combination rule (Eq. 5) applied across time steps. In the RAC variant, the target range is discretized into $K$ bins, each time step produces logits over bins, the averaged softmax probabilities $p_k$ are converted into a continuous conditional density by assuming uniformity within each bin ($f_k = p_k/(b_{k+1}-b_k)$), and the predictive value is $\\sum_k p_k m_k$; training uses a distance loss $\\sum_k |k-j|^q p_k$ plus entropy regularization $\\tau H$. The paper reports that on eight UCI regression datasets, both AOT-SNN variants achieve RMSE and NLL in the same range as MC-dropout and deep ensembles, and that the RAC variant is better in RMSE on three of the datasets.","pith_inferences":["The independence assumption can be probed directly by measuring the autocorrelation of per-time-step outputs for a fixed input; strong correlation would mean the reported variances understate true uncertainty.","If time-step outputs are close to independent, the same trick could extend to mixture density networks by treating each time step as one mixture component, yielding richer densities at negligible extra cost.","The uniform-within-bin density could be swapped for a smooth kernel (triangular or Gaussian) per bin, giving continuous predictive densities and possibly better NLL without changing the training objective.","In streaming settings, time-averaged mean and variance can be updated incrementally as time steps arrive, yielding anytime uncertainty that deep ensembles cannot supply without retraining."],"forward_implications":["AOT-SNN regression needs only one forward pass for uncertainty estimates, so event-driven networks become practical for regression with confidence bounds.","The RAC variant with distance loss beats MC-dropout and deep ensembles in RMSE on Concrete Strength, Energy Efficiency, and Protein Structure, suggesting classification-style losses can improve accuracy in spiking regression.","NLL values reported on most UCI benchmarks are in the DNN baseline range, indicating calibrated uncertainty does not require abandoning spiking hardware.","Because time averaging emulates ensemble averaging, improvements to MC-dropout or deep ensembles can likely be ported to SNNs through the same temporal averaging mechanism."],"supporting_citations":[{"why":"Provides the MC-dropout baseline and the dropout-rate selection procedure used in the UCI experiments.","marker":"[1]"},{"why":"Defines deep ensembles and the mean/variance combination rule (Eq. 5) that the Gaussian AOT-SNN applies across time steps.","marker":"[3]"},{"why":"Introduces the AOT-SNN framework, the average-over-time loss, and the classification-mode uncertainty technique being extended to regression.","marker":"[6]"},{"why":"Gives the dropout-as-Bayesian-approximation foundation that motivates treating time steps as MC samples.","marker":"[9]"},{"why":"Establishes the heteroscedastic Gaussian regression objective (mean and variance outputs with NLL training).","marker":"[11]"},{"why":"Introduces regression-as-classification and reports its feature-learning and accuracy advantages.","marker":"[13]"},{"why":"Supplies the distance loss with entropy regularization and the bin-count choice (K=50) used for the RAC variant.","marker":"[14]"},{"why":"Describes the readout integrator layer with non-spiking output neurons used to read mean, variance, and logits.","marker":"[21]"},{"why":"Supplies the PLIF neuron model whose learnable membrane time constant is used in the SNN experiments.","marker":"[22]"}],"fun_headline_variants":["Spiking nets match deep nets in regression uncertainty","One-pass uncertainty estimation with spiking neural nets","AOT-SNN framework brings uncertainty to regression spikes","Event-driven regressors now output error bars","Time-step averaging yields calibrated uncertainty in SNNs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the outputs at different time steps can be treated as independent MC-dropout samples when combining mean and variance, even though the membrane-potential dynamics make those time steps correlated within one forward pass.","fun_headline_variants_meta":{"raw":{"variants":["Spiking nets match deep nets in regression uncertainty","One-pass uncertainty estimation with spiking neural nets","AOT-SNN framework brings uncertainty to regression spikes","Event-driven regressors now output error bars","Time-step averaging yields calibrated uncertainty in SNNs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000373,"raw_usage":{"total_tokens":2362,"prompt_tokens":1059,"completion_tokens":1303,"prompt_tokens_details":{"cached_tokens":1024},"prompt_cache_hit_tokens":1024,"prompt_cache_miss_tokens":35,"completion_tokens_details":{"reasoning_tokens":1232}},"tokens_in":35,"tokens_out":1303,"duration_ms":270380,"temperature":1.0,"reasoning_tokens":1232,"cache_read_input_tokens":1024,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T05:31:47.828988+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a trained AOT-SNN, feed the same input many times, and compute the autocorrelation of the per-time-step outputs; then compare the predictive variance from the mixture formula with the variance obtained from truly independent forward passes that reset membrane potentials each time. If the independent-pass version gives systematically different NLL or reliability-diagram coverage, the time-average-as-MC-dropout equivalence is not valid.","supporting_citations":[{"cited_title":"Uncertainty in deep learning,","cited_arxiv_id":null,"evidence_quote":"Provides the MC-dropout baseline and the dropout-rate selection procedure used in the UCI experiments."},{"cited_title":"Simple and scalable predictive uncertainty estimation using deep ensembles,","cited_arxiv_id":null,"evidence_quote":"Defines deep ensembles and the mean/variance combination rule (Eq. 5) that the Gaussian AOT-SNN applies across time steps."},{"cited_title":"Efficient uncertainty estimation in spiking neural networks via mc-dropout,","cited_arxiv_id":null,"evidence_quote":"Introduces the AOT-SNN framework, the average-over-time loss, and the classification-mode uncertainty technique being extended to regression."},{"cited_title":"Dropout as a bayesian approximation: Representing model uncertainty in deep learning,","cited_arxiv_id":null,"evidence_quote":"Gives the dropout-as-Bayesian-approximation foundation that motivates treating time steps as MC samples."},{"cited_title":"Estimating the mean and variance of the target probability distribution,","cited_arxiv_id":null,"evidence_quote":"Establishes the heteroscedastic Gaussian regression objective (mean and variance outputs with NLL training)."},{"cited_title":"Regression as clas- sification: Influence of task formulation on neural network features,","cited_arxiv_id":null,"evidence_quote":"Introduces regression-as-classification and reports its feature-learning and accuracy advantages."}],"review_version":1}