{"id":"7846983c-8ee3-4b0c-8e68-9084a5c63aaf","arxiv_id":"2411.18250","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"IKUN sets initial SNN weights with a surrogate-gradient variance correction and reaches accuracy thresholds in fewer epochs on FashionMNIST, but final accuracy is close to standard initializations.","lead":"Researchers propose a new way to set the initial weights of spiking neural networks, an energy-efficient class of AI models, so that signals and gradients stay stable during training. On FashionMNIST, their initialization reaches accuracy targets in up to roughly half the training time of standard methods, though final accuracy gains are small.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central variance condition (Eq. 8) is not derived from the stated LIF dynamics: it omits the τV[t−1] recurrence, spike reset, and threshold, and uses the surrogate derivative in a forward-variance formula.","rationale":"The reader's REJECT verdict is well-founded. The paper's central claim depends entirely on Theorem 1; the experimental section is a single two-layer CSNN on a subset of Fashion-MNIST, with no seeds or error bars, and the authors explicitly acknowledge these limits in Sec. 5. My focus is the theorem itself. Eq. 8 has the shape of an ANN variance-preservation condition for a smooth activation f, but the theorem is stated for an LIF neuron whose forward dynamics are a recurrence with leak and spike reset, and whose backward dynamics include a temporal path. Each of the three omissions identified in the attack is independently sufficient to invalidate the claimed equivalence: the recurrent term τV[t−1] changes the variance recurrence; the spike output's variance is a function of firing rate, not of the surrogate derivative; and BPTT gradient variance includes τ. The unresolved definition of Vthreshold = μH and of the expectation in Eq. 8 make the condition underdetermined. A proof could not simply be 'provided in the appendix' because the appendix is absent from the manuscript. The open-source code link is positive but does not supply the missing mathematical derivation. Thus the central claim is unsupported, and the reader's REJECT verdict should stand unchanged. If a complete proof under spike-reset dynamics and multi-seed experiments on more datasets were added, the verdict could be revisited.","tokens_in":10732,"tokens_out":8241,"duration_ms":79342,"concrete_test":"Independently re-derive Eq. 8 from Eq. 7 under a subtractive spike-reset rule, keeping the τV[t−1] term and the backpropagation-through-time gradient recursion; check whether variance stationarity forces an additional condition involving τ, the reset, and the firing rate p. If the derivation requires dropping τV[t−1], replacing the spike output S[t] by f′(H[t]), or assuming the backward temporal path contributes zero variance, then Theorem 1 is false as stated. A companion Monte Carlo run (10^5 neurons, 100 steps, i.i.d. standard-normal inputs, weights drawn from Eq. 8) would confirm whether Var(H[t]) drifts with t.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step is Theorem 1 (Sec. 3.2), which asserts that σ_W^2 = α / (fanin · σ_X^2 · E[f′(H[t])^2]) stabilizes forward signal variance and prevents gradient vanishing/explosion for the LIF recursion H[t] = τ V[t−1] + Σ_i w_i X_i[t]. The proof is not in the manuscript (the appendix is referenced but absent), and the condition as written cannot be obtained from that recursion. Three concrete omissions. (1) The variance of H[t] includes τ^2 Var(V[t−1]) plus covariance with the input sum; Eq. 8 balances only the instantaneous sum, so it is a memoryless-layer condition, not a condition for Eq. 7. (2) The forward nonlinearity is spike-and-reset, whose output variance is p(1−p) for firing probability p; E[f′(H)^2] is a surrogate-gradient quantity used in backpropagation and does not determine p. (3) Backpropagation through time contributes a temporal Jacobian of magnitude τ, and the gradient-variance balance must include it; Eq. 8 does not. The threshold Vthreshold = μH is also undefined because μ is never specified, and the expectation in Eq. 8 is over an unspecified distribution despite σ_W appearing implicitly through H. The conclusion (Sec. 5) concedes that experiments used a two-layer CSNN on a fixed subset of Fashion-MNIST with no error bars; such evidence cannot substitute for the missing guarantee.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes IKUN, a weight-initialization scheme for spiking neural networks trained with surrogate gradients. The central theoretical claim (Theorem 1, Section 3.2) is that setting the weight variance according to Eq. (8) stabilizes forward signal variance and prevents gradient vanishing or explosion for the LIF-type dynamics in Eq. (7). The authors report experiments on a two-layer convolutional SNN on Fashion-MNIST, claiming faster convergence and better generalization than Xavier, Kaiming, LeCun, and default initialization, and support this with Hessian eigenvalue and trace analyses. The proof of Theorem 1 is deferred to an appendix that is absent from the preprint.","tokens_in":11134,"tokens_out":7236,"duration_ms":66375,"significance":"If the theoretical result were correct, IKUN would be a useful plug-and-play initialization for surrogate-gradient SNN training, and the paper deserves credit for open-sourcing the code and for examining multiple surrogate-gradient compatibility questions. However, the mathematical core is unverified, the variance condition as stated omits the recurrent and spike-reset structure of the neuron model, and the empirical evidence is limited to a single small dataset with no error bars. The paper also contains an inconsistency between the reported epoch-reduction percentages and the numbers in Table 1. These issues are load-bearing for the paper's central claims.","major_comments":[{"comment":"The variance condition is asserted for the recurrent dynamics H[t] = tau V[t-1] + sum_i w_i X_i[t], but Eq. (8) balances only the instantaneous input term. From Eq. (7), Var(H[t]) includes tau^2 Var(V[t-1]) and a covariance term with the input sum; neither appears in Eq. (8). The theorem also uses E[f'(H)^2], a surrogate-gradient quantity from the backward pass, to control forward variance, although the forward nonlinearity is a spike-and-reset operation whose output variance depends on the firing probability, not on f'. The proof is deferred to an appendix that is not present in the preprint, so these gaps cannot be checked.","section":"Section 3.2, Eqs. (7)-(8)"},{"comment":"Equation (8) is not an explicit initialization rule. The expectation E[f'(H)^2] is over an unspecified distribution of H, and H depends on sigma_W through Eq. (7), making the equation a fixed-point condition. The threshold Vthreshold = mu H introduces an additional unspecified parameter mu, and alpha is described only as a given hyperparameter with no value reported in Section 4. Without specifying these quantities, Eq. (8) cannot be instantiated or reproduced.","section":"Section 3.2, Eq. (8)"},{"comment":"The gradient-vanishing/explosion claim is not supported because backpropagation through time is ignored. The temporal recursion contributes products of the leakage factor tau, together with the reset mechanism, to the gradient Jacobian; any variance balance for gradients must account for these terms and the time horizon. Equation (8) contains no tau and no time dependence, so the theorem's conclusion about backpropagation does not follow from the forward-variance balance it states.","section":"Section 3.2, Theorem 1"},{"comment":"The paper reports a 59.38% training-epoch reduction under SGD, but the listed SGD epochs are IKUN v2 = 26, Normal = 29, and Xavier = 27; the largest reduction visible in the table is 10.3%. The Adam reduction of 42.31% is consistent with the table (26 to 15 epochs), but the SGD headline number is not derivable from the printed results and needs a stated baseline.","section":"Section 4.3, Table 1"},{"comment":"The experimental evidence is too narrow for the general claims in the abstract. The model is a two-layer convolutional SNN, the data is a fixed subset of Fashion-MNIST, and no seeds, error bars, or repeated runs are reported; Section 5 explicitly concedes these limitations. This does not support broad statements about training efficiency and generalization, nor the Hessian-based generalization conclusions, without additional statistical validation.","section":"Section 5 and Section 4.1"}],"minor_comments":[{"comment":"The symbol f in Eq. (8) is never defined; specify the surrogate activation function whose derivative is used.","section":"Section 3.2"},{"comment":"\"LeCun Initialization (Bi & Poo, 1998)\" cites the wrong reference; the correct source is LeCun et al. (2002).","section":"Section 4.1"},{"comment":"The \"Normal\" initialization is not described; specify the distribution and scale used.","section":"Section 4.1"},{"comment":"The text says lower absolute Hessian Trace values indicate flatter minima, but the table reports large negative traces for several methods; define the metric and its sign convention.","section":"Table 2 and Section 4.2"},{"comment":"The abstract states \"up to 50%\" improvement in training efficiency, while Table 1 reports 59.38%; unify these numbers.","section":"Abstract and Table 1"},{"comment":"Notation switches from V(t) to H[t] without defining the relationship; clarify whether H[t] is the pre-spike membrane potential and V[t] is the post-reset potential.","section":"Eqs. (6)-(7)"}],"recommendation":"reject","confidential_remarks":"The central theorem is unreviewable as submitted because the proof is in a missing appendix, and the stated variance condition appears inconsistent with the LIF dynamics it claims to analyze. Any resubmission should include the full proof or a substantially weakened theoretical claim, correct the experimental-reporting inconsistencies, and provide repeated runs with a complete dataset description."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nIKUN is a variance-preserving initialization for SNNs that folds in a surrogate-gradient term, giving formulas like sigma_W^2 = alpha / (fanin * sigma_X^2 * E[f'(H)^2]) and a fan-in+fan-out variant. That specific form does not appear in the paper's cited ANN initialization literature, and it is a natural extension of Xavier/Kaiming logic to the surrogate-gradient setting. So the core idea is new in a modest, useful way.\n\nWhat the paper does well: it correctly identifies that SNNs break the forward/backward symmetry that classic init methods assume, and it tries to account for the surrogate gradient explicitly. The Hessian analysis, though limited, is a reasonable extra probe. The authors also honestly note in Section 5 that they only used a two-layer CSNN on a fixed FashionMNIST subset with constrained compute. That honesty is to their credit.\n\nThe soft spots are load-bearing. Theorem 1's proof is deferred to an appendix that is not in the preprint, so the central guarantee is unverifiable. More concerning, the stress-test argument lands: Eq. 8 balances only the instantaneous input sum wi Xi[t] and ignores the tau V[t-1] recurrence, the reset, and the threshold. LIF variance propagation is temporal; a single per-layer condition of this form cannot follow from Eq. 7 without extra assumptions about stationarity or independence that are never stated. The use of E[f'(H)^2] in a forward-variance formula is also conceptually odd, since that term comes from backpropagation, and the expectation is never defined. Alpha is a free knob that could in principle absorb the gap, which makes the formula underdetermined as a theoretical statement.\n\nEmpirically, the evidence is thin: one dataset, no seeds or error bars, and the reported speedups are threshold-dependent and not consistent across optimizers (IKUN underperforms Kaiming under Adam in some plots). The Hessian trace results are suggestive but not conclusive, and the claim about flat minima is not tied to any statistical test.\n\nBottom line: the idea deserves a proper treatment, but this version does not support its own claims. A revised paper with the proof made explicit, alpha and expectation specified, multi-seed experiments on at least one more dataset, and code that runs would be worth refereeing. As it stands, I would not send it to peer review; I would encourage the authors to resubmit with those gaps closed.\n\nFor us: I would only bring this to reading group as a case study in how variance-preservation arguments can silently skip temporal dynamics.","headline":"A genuinely new initialization formula, but the proof is missing and the experiments are too thin to carry the claim.","tokens_in":11581,"tokens_out":2858,"would_cite":false,"duration_ms":30109,"reading_group":"maybe","serious_thinker":"unclear","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A surrogate-aware weight initialization, IKUN, claims to stabilize signal and gradient variance in spiking neural networks, reaching accuracy thresholds in up to 59% fewer epochs.","keywords":["spiking neural networks","weight initialization","surrogate gradient","variance stabilization","gradient vanishing and explosion","Hessian spectrum","flat minima","FashionMNIST"],"falsifier":"Train the reported architecture at greater depth (say ten layers) or with random spike-train inputs, and measure the variance of the membrane potential at each layer and time step. IKUN predicts flat variance and bounded gradient norms; if variance grows or gradients vanish as depth or time steps increase, the balancing condition fails.","tokens_in":10546,"feed_emoji":"⚡","tokens_out":9386,"duration_ms":77226,"temperature":0.7,"pith_summary":"Spiking neural networks trained with surrogate gradients usually inherit ANN-style initializations that ignore how spiking changes signal statistics. This paper claims a new initialization, IKUN, can fix that by choosing each layer's weight variance so that forward signal variance stays constant and backward gradients neither vanish nor explode. The rule uses the surrogate gradient's expected squared derivative as a correction factor, replacing the plain fan-in/fan-out scaling of Xavier and Kaiming. On FashionMNIST with a two-layer convolutional SNN, IKUN reaches high-accuracy thresholds in up to roughly 59% fewer epochs under SGD and 42% fewer under Adam, reaching 95% training and 91% test accuracy, and its Hessian spectrum looks flat and nearly all positive.","feed_headline":"Spiking-network initialization cuts training epochs by up to 59%","feed_subtitle":"IKUN balances signal and gradient variance, reaching 95% train and 91% test accuracy in fewer epochs.","key_machinery":"The load-bearing object is the surrogate-stable variance condition of Equation (8), an SNN analogue of Xavier and Kaiming initialization. The surrogate gradient $f'$ is the smooth derivative of the spiking activation used during backpropagation in place of the non-differentiable spike function. IKUN sets $\\sigma_W^2 = \\alpha / (\\mathrm{fanin} \\cdot \\sigma_X^2 \\cdot \\mathbb{E}[f'(H)^2])$, so the scale of the random weights is inversely proportional to the expected squared surrogate gradient; IKUN v2 replaces $\\mathrm{fanin}$ with $\\mathrm{fanin}+\\mathrm{fanout}$ in the denominator. This factor carries the argument: it converts the surrogate's effect on backward gradients into a per-layer variance-balance condition, exactly as ReLU's half-wave rectification enters Kaiming initialization.","core_discovery":"The paper's central claim is Theorem 1: in an SNN layer with membrane potential $H[t] = \\tau V[t-1] + \\sum_i w_i X_i[t]$ and weights $w_i \\sim \\mathcal{N}(0, \\sigma_W^2)$, the condition $\\sigma_W^2 = \\alpha / (\\mathrm{fanin} \\cdot \\sigma_X^2 \\cdot \\mathbb{E}[f'(H)^2])$ keeps signal variance stable during forward propagation and prevents gradient vanishing or explosion during backpropagation. A second version, IKUN v2, uses $\\mathrm{fanin}+\\mathrm{fanout}$ in the denominator to balance input and output dimensions. The authors support the theorem with experiments on a two-layer convolutional SNN trained on FashionMNIST, where IKUN reaches 95% training and 91% test accuracy in fewer epochs than Xavier, Kaiming, LeCun, and normal initialization, and where trained models show Hessian eigenvalues concentrated near zero on the positive side, which they interpret as convergence to flat minima with better generalization.","pith_inferences":["Editorial inference: the variance-balance argument could be extended to deep or recurrent SNNs by replacing the static $\\mathbb{E}[f'(H)^2]$ with a time-averaged or layer-dependent value, but the paper does not prove that extension.","Editorial inference: if the condition holds at scale, SNN libraries could adopt an IKUN-style default instead of Kaiming, removing a common source of instability in deep spike-based models.","Editorial inference: the flat-minima result hints that IKUN may combine well with sharpness-aware optimizers or pruning, though the paper does not test such combinations.","Editorial inference: because the condition depends on $\\sigma_X^2$, input encoding (rate vs. temporal) should change the optimal initialization; testing IKUN under temporal coding would be a natural next step."],"forward_implications":["IKUN reaches accuracy thresholds in up to 59.38% fewer epochs under SGD and 42.31% fewer under Adam compared with standard initializations on the reported FashionMNIST setup.","Trained models show a nearly all-positive Hessian spectrum close to zero, which the paper links to flat minima and improved generalization.","The method is designed to work with any surrogate gradient (sigmoid, tanh, or linear), so the same initialization rule transfers across surrogate choices.","IKUN v2's fanin+fanout variant explicitly accounts for both input and output dimensions, making it suitable for convolutional and other non-square layers.","The gains appear under both SGD and Adam, whereas the baselines the paper tests do not dominate on both optimizers at once."],"supporting_citations":[{"why":"Defines the Xavier condition that balances forward and backward variance; IKUN v2 modifies its fanin+fanout denominator.","marker":"(Glorot & Bengio, 2010)"},{"why":"Introduces Kaiming initialization for ReLU networks, the direct ancestor of IKUN v1 and a baseline in the experiments.","marker":"(He et al., 2015)"},{"why":"Establishes the early variance-stabilizing initialization and serves as a comparison method.","marker":"(LeCun et al., 2002)"},{"why":"Supplies the surrogate-gradient training framework whose derivative term $\\mathbb{E}[f'(H)^2]$ enters the IKUN condition.","marker":"(Neftci et al., 2019)"},{"why":"Provides the FashionMNIST dataset used for all training and generalization comparisons.","marker":"(Xiao et al., 2017)"},{"why":"Provides the Hessian eigenvalue and trace estimation used to support the flat-minima generalization claim.","marker":"(Yao et al., 2020)"},{"why":"Gives the spatio-temporal backpropagation dynamics that motivate the membrane-potential update and SNN training setting.","marker":"(Wu et al., 2018)"}],"fun_headline_variants":["IKUN stabilizes SNN variance for 50% faster training","SNN init cuts training time by half with stable variance","Variance-stable SNN initialization hits 91% test accuracy","IKUN converges to flatter minima for better SNN generalization","IKUN: 50% faster SNN training, 91% test accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The proof assumes each layer's weighted input is a zero-mean Gaussian whose variance factorizes as fanin times weight variance times input variance, so the only effect of spiking is the scalar $\\mathbb{E}[f'(H)^2]$; if temporal accumulation or non-Gaussian inputs break that, the variance condition stops holding.","fun_headline_variants_meta":{"raw":{"variants":["IKUN stabilizes SNN variance for 50% faster training","SNN init cuts training time by half with stable variance","Variance-stable SNN initialization hits 91% test accuracy","IKUN converges to flatter minima for better SNN generalization","IKUN: 50% faster SNN training, 91% test accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000213,"raw_usage":{"total_tokens":1433,"prompt_tokens":967,"completion_tokens":466,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":583,"completion_tokens_details":{"reasoning_tokens":375}},"tokens_in":583,"tokens_out":466,"duration_ms":4354,"temperature":1.0,"reasoning_tokens":375,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T11:22:51.049382+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the reported architecture at greater depth (say ten layers) or with random spike-train inputs, and measure the variance of the membrane potential at each layer and time step. IKUN predicts flat variance and bounded gradient norms; if variance grows or gradients vanish as depth or time steps increase, the balancing condition fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Hessian eigenvalue and trace estimation used to support the flat-minima generalization claim."}],"review_version":1}