{"id":"a5cfa515-3254-4266-b78a-f0b3ef459d85","arxiv_id":"2506.23757","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"An Expectation-Propagation framework trains spiking neural networks with mini-batches, supporting discrete or continuous weights and deterministic or stochastic spiking, and returns approximate posterior uncertainties.","lead":"This paper proposes a training method for spiking neural networks based on Expectation-Propagation, a Bayesian message-passing technique that avoids gradient computation and supports discrete or continuous weights. The method reports results comparable to gradient-based baselines on small classification and regression tasks while also producing uncertainty estimates.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The load-bearing step is Eq. (17), where the Bernoulli presynaptic spike is replaced by its cavity mean; the induced bias in weight updates is unquantified and likely severe for sparse activations.","rationale":"The reader's weakest-assumption analysis identifies the same load-bearing approximation I would stress-test: Eq. (17) fixes a Bernoulli presynaptic spike to its mean when deriving the weight update. This is not a minor technicality; it is the step that makes the backward pass tractable without Monte Carlo, and it feeds directly into the moment matching that defines q1(w) and hence the claimed posterior over weights. My concern is therefore best framed as an unquantified bias in the core message-passing update, not as a disagreement with the overall design or with the positive empirical results. The paper's small experiments show that the pipeline can run and give sensible predictions, but they do not isolate the effect of Eq. (17) or compare it against a reference calculation. I have not found an internal contradiction that would make the method invalid; rather, the missing evidence is a controlled test of the approximation's accuracy. Because the reader's conditional verdict already asks for stronger derivations and empirical support, my concern reinforces that request without moving the verdict. I recommend UNCHANGED: the paper should remain conditional, with the concrete test above used to decide whether the Eq. (17) approximation is acceptable in the sparse-activation regimes the paper claims to support.","tokens_in":15010,"tokens_out":3827,"duration_ms":50949,"concrete_test":"On a synthetic 2-layer network (e.g., 20→5→1) with known Gaussian weights and Bernoulli hidden activations, run one backward pass of Algorithm 2. For each δ factor, compute the exact tilted distribution of w_{i,j} by enumerating all 2^5 hidden spike patterns and integrating over the Gaussian cavity exactly, then compare with the Eq. (17)–(18) update. Sweep the hidden firing probability p∈{0.1,0.3,0.5}. If the KL divergence between the exact and EPSNN per-factor weight posteriors grows as p decreases (e.g., exceeds 0.1 nats at p=0.1), Eq. (17) is the bottleneck; if the divergence remains small at all p, the approximation is acceptable and the central claim survives this test.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In the mixing-block update of Section IV-B, the delta factor δ(w_{i,j}^{(ℓ)} − η_w) is valid only when v_{n,j}^{(ℓ−1)} ≠ 0, and Eq. (17) then replaces that Bernoulli presynaptic spike by its q_c mean. For a Bernoulli variable, E[v]=p and Var(v)=p(1−p), so replacing the random denominator by its mean is uncontrolled except when p is near 0 or 1 and the numerator is sufficiently independent; at intermediate or small p the approximation is not variance inflation alone but can move the posterior mode in the wrong direction. Conditional on v_j=1 the factor constrains w_j directly, while the mean-substituted update rescales the numerator by 1/p, which can overstate or understate the evidence. If p→0 the denominator tends to 0 and the update becomes numerically unstable, yet Section V-B does not specify a constraint or fallback for this case. Because this local factor is moment-matched into q1,n(w_{i,j}) and combined across samples by Eq. (19), the bias propagates through every hidden-layer weight update and affects all four claimed variants of the method (discrete/continuous weights, deterministic/stochastic activations). The paper provides no ablation or diagnostic quantifying this bias, so the central claim of learning marginal weight distributions rests on an unverified approximation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes EPSNN, a message-passing framework based on Expectation-Propagation (EP) for training spiking neural networks (SNNs). The method combines average EP and stochastic EP to handle mini-batches, and it is designed to estimate marginal posterior distributions of network weights, including discrete or continuous weights, for deterministic or stochastic spiking activations. The authors present the Bayesian model, derive the local EP updates for mixing and activation blocks, discuss algorithmic considerations such as damping and convergence stabilization, and report experiments on MNIST classification and a 1D regression task. The paper claims that this is the first framework enabling training of discrete and continuous weights for deterministic and stochastic SNNs using batches of training samples, while also providing uncertainty estimates without Monte Carlo sampling.","tokens_in":15268,"tokens_out":4141,"duration_ms":48363,"significance":"If the method is correct, it constitutes a useful contribution to Bayesian training of SNNs: it is gradient-free, provides marginal weight uncertainties, handles discrete weights, and supports mini-batch training through SEP. The paper gives a clear modular derivation and includes pseudo-code for the main algorithms, which aids reproducibility. The experimental results, while limited to small architectures and datasets, are plausible and show competitive performance with an MLE baseline. However, the central algorithmic approximation is not quantitatively validated, and several details needed to reproduce the algorithm are omitted, which tempers the strength of the claims.","major_comments":[{"comment":"The approximation that replaces the Bernoulli presynaptic spike v_{n,j}^{(ell-1)} by its mean under the cavity distribution q_c is a load-bearing step in the derivation of the weight updates. For a Bernoulli variable, conditioning on v=1 versus v=0 yields qualitatively different constraints on the weight: when v=0, the delta factor does not constrain w_{i,j} at all. Substituting the mean collapses this mixture and discards the zero-spike branch, which can systematically bias both the mean and the variance of the estimated weight posterior, particularly for sparse or highly variable activations. This bias propagates through every hidden-layer weight update and affects all four claimed variants of the method (discrete/continuous weights, deterministic/stochastic activations). The paper provides no ablation, diagnostic, or theoretical bound quantifying this bias, and no fallback for the case where the mean of v is near zero, which would make the denominator in Eq. (17) unstable. Please provide an analysis of the bias or an alternative update that marginalizes over the Bernoulli variable, and demonstrate its effect in a controlled experiment with varied spike rates.","section":"Section V-B, Convergence paragraph"},{"comment":"The manuscript states that stability is achieved by bounding the natural parameters of the Gaussian approximating factors and by using constrained KL minimization for categorical distributions, but the derivation of these constrained updates is omitted, with the text saying it is 'similar to that in [37]'. Since these constraints are necessary to prevent negative variances and to keep the algorithm stable in the reported experiments, their omission prevents full reproduction of the method. Please include the derivations in a supplementary document or provide a precise specification of the constraints and the optimization procedure used, as this is essential for the paper's claims to be independently verified.","section":"Section V-B, Convergence paragraph"},{"comment":"The abstract's 'first time' claim is supported by only a narrow experimental evaluation: MNIST with up to 10^4 training samples, a one-layer and a two-layer architecture, and a single 1D regression problem. Moreover, the text in Section VI says the regression experiment allows visual comparison with the BiSNN method, but no BiSNN results are shown; the comparison is only against a basic MLE baseline. There is also no wall-clock time comparison, despite the claim of faster convergence than gradient-based methods. Please either broaden the experimental evaluation (e.g., compare with existing Bayesian SNN methods such as BiSNN, report runtimes, include a deeper architecture or a more realistic dataset) or moderate the 'first time' and 'faster than gradient-based methods' claims to match the evidence presented.","section":"Section VI, Experiments"}],"minor_comments":[{"comment":"The phrase 'The first (resp. second) values are the metrics computed on the test (resp. test) data' appears to contain a typo; presumably the first value is on the training data and the second on the test data. Please correct the captions.","section":"Tables I, II, III captions"},{"comment":"In Eq. (16), the numerator uses u_{i,j}, but the membrane potential notation elsewhere is u_{n,i} (or u_{i,n} is defined for sample n and neuron i). Please standardize the subscripts to avoid confusion.","section":"Section IV-B, Eq. (16)"},{"comment":"There is a grammatical error: 'The training sets consists of up to 10^4 frames' should be 'The training set consists of up to 10^4 frames'.","section":"Section VI-B"},{"comment":"The paper uses q_c(v_n^{(l)}) and q_c(v_n^{(l-1)}) interchangeably in the cavity definitions; please ensure notation is consistent, especially in Eqs. (13) and (20).","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper's core idea is promising and falls within the scope of the journal, but the load-bearing approximation in Eq. (17) needs rigorous analysis or an alternative derivation, and the omission of the constrained update derivations is a reproducibility gap. The 'first time' claim would be more credible with a broader experimental comparison. I recommend major revision rather than rejection because the central derivation is standard EP machinery and the approximation issue, while serious, might be addressable with additional analysis and experiments."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, I read arXiv:2506.23757. The real news: this is the first EP-based training scheme for SNNs that genuinely works with mini-batches, and it covers the four combinations of discrete/continuous weights and deterministic/stochastic activations in one framework. The message-passing derivation is competent and the authors are upfront that convergence is not guaranteed. The regression experiment with interpolation gaps is a good illustration of what the uncertainty estimates buy you.\n\nNow the soft spots. The Eq. (17) approximation is load-bearing: fixing the Bernoulli presynaptic spike to its cavity mean in the denominator of the weight update is uncontrolled. For a Bernoulli with probability p, the variance is p(1-p), and the mean-substituted update rescales by 1/p, which can move the posterior mode in the wrong direction or blow up as p->0. The paper does not quantify this bias or offer an ablation, and it affects every hidden-layer weight update in all four variants. That is a real gap, not a stylistic one.\n\nThe experiments are thin: two shallow architectures, MNIST at ~85% accuracy, no error bars, no code or data. The MLE baseline uses test-based early stopping, which flatters the comparison. The constrained natural-parameter updates are mentioned but the derivation is omitted, despite the paper relying on them for stability. These are all addressable, but together they mean the paper's central claim is broader than the demonstrated evidence.\n\nOn the other hand, the paper is honest about its limitations, the derivations are standard EP, and the mini-batch combination of SEP and AEP is a legitimate building block. I think it deserves a serious referee, but the referee should push for a quantified analysis of the Eq. (17) approximation, code release, and stronger experiments. I would not cite it in my own work yet; it's too early-stage and the empirical bar is low. But I'd bring it to a reading group if we were working on Bayesian SNNs.","headline":"Solid EP-extension paper whose mini-batch innovation is real, but the load-bearing mean-substitution approximation in Eq. (17) needs quantification and the experiments need strengthening.","tokens_in":15774,"tokens_out":1748,"would_cite":false,"duration_ms":19087,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Expectation-propagation can train spiking neural networks without gradients, learning distributions over discrete or continuous weights in mini-batches.","keywords":["spiking neural networks","expectation-propagation","Bayesian inference","gradient-free training","discrete weights","stochastic spiking neurons","uncertainty quantification","mini-batch learning"],"falsifier":"For a small two-layer SNN with Bernoulli-sigmoid activations and low firing rates, compare the weight-update moments produced by replacing each presynaptic spike with its current average (the rule in Section IV-B) against exact moments computed by numerical integration or Monte Carlo from the same approximate distribution; a systematic divergence as firing rate drops would show where that approximation breaks.","tokens_in":14774,"feed_emoji":"⚡","tokens_out":17041,"duration_ms":164318,"temperature":0.7,"pith_summary":"The paper claims that Expectation-Propagation can train spiking neural networks by message passing instead of gradient descent, and that this is the first framework to cover all four settings previously handled separately: continuous or discrete weights and deterministic (Heaviside) or stochastic (Bernoulli-sigmoid) spiking neurons. The method, called EPSNN, returns approximate posterior distributions over the weights while integrating out the hidden layers' spike outputs, so uncertainty estimates come as a by-product. Because it combines stochastic and average EP updates, it can learn from mini-batches and make multiple passes through the data. The experiments on MNIST classification and 1D regression suggest the algorithm converges faster than gradient-based training, needs fewer passes through the data, and gives predictive intervals that widen in regions where training data are missing.","feed_headline":"Spiking networks train gradient-free, learning weight distributions","feed_subtitle":"One training framework handles discrete or continuous weights and deterministic or stochastic neurons in batches.","key_machinery":"The central machinery is a factor graph in which each network layer is split into a mixing block, enforcing the linear constraint $u = W v$, and an activation block, enforcing the spiking likelihood $f(v|u)$. Every variable group carries a two-factor approximation $q(W,\\Theta)=q_0 q_1$, and updates are done by KL-based moment matching between a cavity distribution (the current approximation with the factor being updated removed) and the tilted distribution (the cavity times that factor). The mixing-block update handles each delta constraint $\\delta(u - w^T v)$ with closed-form Gaussian moment matching; the backward pass becomes tractable by fixing the presynaptic spike $v$ in the denominator of the weight update to its cavity mean. Mini-batch learning is handled by stochastic EP, which treats the weight factor as an average over batches and creates a cavity $q(W)/q_1(W)$ for each batch.","core_discovery":"The paper's central claim is that the intractable Bayesian posterior over SNN weights and latent activities can be approximated by a factorized message-passing distribution, updated by Expectation-Propagation, in a way that preserves the familiar forward/backward pass structure of neural network training. A forward pass propagates approximate moments of membrane potentials and spike outputs through mixing and activation blocks; a backward pass refines the weight and activity factors by matching tilted moments. The result is a training algorithm that estimates marginal weight distributions, supports discrete and continuous weight families, handles deterministic and stochastic spiking activations in one framework, and produces predictive distributions at inference time from a single forward pass without Monte Carlo sampling.","pith_inferences":["Beyond the paper, the mean-replacement approximation in the backward pass should be stress-tested in the low-firing-rate regime; if its bias grows as spike variance increases, the uncertainty estimates would be least reliable precisely in sparse networks where they matter most.","The factor-graph block structure suggests a path to convolutional and recurrent spiking layers, but shared weights and temporal dependencies would require new cavity derivations rather than direct reuse of the present equations.","Because predictive entropy is cheap to compute, a natural extension is out-of-distribution detection for neuromorphic sensors, where input spike patterns that fall outside the training manifold should inflate the predictive variance; the paper does not test this."],"forward_implications":["SNNs with binary weights can be trained directly, without surrogate gradients or relaxation tricks, and the trained model carries a posterior probability for every weight.","One implementation covers both Heaviside and Bernoulli-sigmoid neurons, so switching between deterministic and stochastic spiking models does not require a different training loop.","Predictive means and variances at test time come from one forward pass through the factor graph, which makes Bayesian uncertainty practical on hardware where repeated sampling is too expensive.","Sparse priors such as Bernoulli-Gaussian produce per-weight activation probabilities, giving feature-relevance maps that are not available from standard point-estimate training."],"supporting_citations":[{"why":"It defines expectation-propagation and its moment-matching local update, which is the foundation of the training algorithm.","marker":"[27]"},{"why":"It supplies expectation backpropagation, whose closed-form delta-constraint updates are reused for the mixing block with continuous or discrete weights.","marker":"[32]"},{"why":"It supplies the probabilistic backpropagation forward/backward structure that the method extends to spiking layers and multi-epoch mini-batch training.","marker":"[33]"},{"why":"It introduces stochastic expectation propagation, the mini-batch cavity construction and per-batch combination rule used for large datasets.","marker":"[34]"},{"why":"It applies expectation-propagation to probabilistic spiking networks and establishes the spiking likelihood family this paper broadens.","marker":"[31]"},{"why":"It provides the binary-weight Bayesian SNN baseline and the regression task the experiments reproduce for visual uncertainty comparison.","marker":"[26]"},{"why":"It supplies spike-and-slab priors, constrained KL minimization, and damping/natural-parameter bounding used to stabilize the EP updates.","marker":"[37]"},{"why":"It gives the semi-analytic moments of sigmoid activations used in the activation blocks of stochastic spiking neurons.","marker":"[40]"},{"why":"It supplies the Heaviside-activation moment formulas for deterministic spiking neurons and the EP derivation framework used throughout.","marker":"[41]"}],"fun_headline_variants":["Gradient-free spiking training via Expectation-Propagation","Message-passing unifies SNN training: discrete, stochastic, batch","Expectation-Propagation trains spiking nets without gradients","No backprop needed: EP trains spiking networks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The backward pass assumes each presynaptic spike value can be replaced by its current average when computing weight updates, and if spike activity is sparse or highly variable, that substitution may bias the learned weight distributions.","fun_headline_variants_meta":{"raw":{"variants":["Gradient-free spiking training via Expectation-Propagation","Message-passing unifies SNN training: discrete, stochastic, batch","Expectation-Propagation trains spiking nets without gradients","No backprop needed: EP trains spiking networks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000777,"raw_usage":{"total_tokens":3351,"prompt_tokens":777,"completion_tokens":2574,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":393,"completion_tokens_details":{"reasoning_tokens":2504}},"tokens_in":393,"tokens_out":2574,"duration_ms":21392,"temperature":1.0,"reasoning_tokens":2504,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T21:32:06.083540+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"For a small two-layer SNN with Bernoulli-sigmoid activations and low firing rates, compare the weight-update moments produced by replacing each presynaptic spike with its current average (the rule in Section IV-B) against exact moments computed by numerical integration or Monte Carlo from the same approximate distribution; a systematic divergence as firing rate drops would show where that approximation breaks.","supporting_citations":[{"cited_title":"Expectation propagation for approximate Bayesian inference,","cited_arxiv_id":null,"evidence_quote":"It defines expectation-propagation and its moment-matching local update, which is the foundation of the training algorithm."},{"cited_title":"Expectation backpropagation: Parameter-free training of multilayer neural networks with continuous or discrete weights,","cited_arxiv_id":null,"evidence_quote":"It supplies expectation backpropagation, whose closed-form delta-constraint updates are reused for the mixing block with continuous or discrete weights."},{"cited_title":"Probabilistic backpropagation for scalable learning of bayesian neural networks,","cited_arxiv_id":null,"evidence_quote":"It supplies the probabilistic backpropagation forward/backward structure that the method extends to spiking layers and multi-epoch mini-batch training."},{"cited_title":"Stochastic expectation propagation,","cited_arxiv_id":null,"evidence_quote":"It introduces stochastic expectation propagation, the mini-batch cavity construction and per-batch combination rule used for large datasets."},{"cited_title":"Probabilistic spiking neural networks training with expectation-propagation,","cited_arxiv_id":null,"evidence_quote":"It applies expectation-propagation to probabilistic spiking networks and establishes the spiking likelihood family this paper broadens."},{"cited_title":"Bisnn: training spiking neural networks with binary weights via bayesian learning,","cited_arxiv_id":null,"evidence_quote":"It provides the binary-weight Bayesian SNN baseline and the regression task the experiments reproduce for visual uncertainty comparison."},{"cited_title":"Expectation propagation in linear regression models with spike-and- slab priors,","cited_arxiv_id":null,"evidence_quote":"It supplies spike-and-slab priors, constrained KL minimization, and damping/natural-parameter bounding used to stabilize the EP updates."},{"cited_title":"A family of algorithms for approximate bayesian inference,","cited_arxiv_id":null,"evidence_quote":"It supplies the Heaviside-activation moment formulas for deterministic spiking neurons and the EP derivation framework used throughout."}],"review_version":1}