Pith. sign in

REVIEW 3 major objections 4 minor 24 references

ReverB-SNN: Reversing Bit of the Weight and Activation for Spiking Neural Networks

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A spiking network that swaps binary spikes for real-valued spikes and binarizes its weights claims 70.91% top-1 on ImageNet at four timesteps.

desk verdict Solid low-latency SNN result with a specific, addressable flaw in the learnable variant's inference-time re-parameterization for residual networks. read the letter →

arxiv 2506.07720 v1 pith:U6WERKXM submitted 2025-06-09 cs.CV

Yufei Guo , Yuhan Zhang , Zhou Jie , Xiaode Liu , Xin Tong , Yuanpei Chen , Weihang Peng , Zhe Ma This is my paper
classification cs.CV
keywords spikingneuralnetworksreal-valuedspikeactivationsbinaryweightsre-parameterizationLIFneuronmultiplication-freeinferenceevent-drivencomputationImageNetclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Spiking neural networks transmit binary spikes, which saves energy but caps accuracy because a spike carries only one bit. This paper proposes swapping the usual precision assignment: keep real-valued spike activations, so each spike carries a continuous number, and binarize the weights to +1 or −1. The central claim is that this reversal raises accuracy while keeping inference event-driven and addition-only, because a real-valued spike times a binary weight is still just an addition or subtraction. A learnable variant scales each channel by an amplitude factor during training and folds that factor into the previous layer's output at inference, so deployment again uses only ±1 weights. If the claim holds, directly trained SNNs can close much of the accuracy gap to ANNs while keeping low-latency, multiplication-free inference; the paper reports 70.91% top-1 on ImageNet with ResNet34 at four timesteps.

What carries the argument

The load-bearing mechanism is the modified LIF neuron with real-valued output spikes, paired with sign-binarized weights. Its update is $U^t_l = \tau U^{t-1}_l + \operatorname{sign}(W_l) O^t_{l-1}$ with $O^t_l = U^t_l$ if $U^t_l \ge V_{\mathrm{th}}$ and $0$ otherwise, so the multiply is always by $+1$ or $-1$. The learnable variant replaces $\operatorname{sign}(W_l)$ with $\alpha_l \operatorname{sign}(W_l)$, trains $\alpha_l$ by gradient descent, and then re-parameterizes at inference by folding $\alpha_l$ into the previous layer's output: $O^{t}_{\mathrm{new},l-1} = \alpha_l U^t_{l-1}$ when the neuron fires. Straight-through estimation handles the non-differentiability of the sign function during training.

What would settle it

Take a trained learnable-$\alpha$ ResNet, run the published re-parameterization (fold each layer's $\alpha_l$ into the previous layer's output), and feed the same test batch through both the original and the folded network. If the top-1 predictions or output logits differ by more than floating-point rounding, the inference-time network is not the network that was trained, and the reported accuracy belongs to a different model than the one actually deployed.

Watch

Extended reading notes

Core claim

The discovery the paper argues for is that activation precision, not weight precision, is the binding constraint on directly trained SNN accuracy, so the standard allocation—binary spikes with full-precision weights—can be reversed with a net gain. Formally, the membrane update becomes $U^t_l = \tau U^{t-1}_l + \operatorname{sign}(W_l) O^t_{l-1}$, with outputs $O^t_l = U^t_l$ when above threshold and $0$ otherwise, and with a channel-wise learnable amplitude $\alpha$ in the trainable variant. Because $\operatorname{sign}(W_l) \in \{-1, 1\}$, every multiply reduces to adding or subtracting the incoming real-valued spike. The paper claims this design consistently beats prior SNN methods on CIFAR-10, CIFAR-100, ImageNet, and CIFAR10-DVS, with the learnable variant reaching 70.91% top-1 accuracy on ImageNet at 4 timesteps and 80.50% on CIFAR10-DVS.

Load-bearing premise

The claim that the trainable variant can be folded back into a standard ±1-weight network at no accuracy cost assumes that each activation tensor is consumed by exactly one downstream operation; in ResNet the same activation also enters the residual shortcut, so the folded network is not automatically the trained network.

Editorial extensions

If this is right

  • If the central claim is right, state-of-the-art directly trained SNNs can gain several accuracy points without changing the deployment operation: the ImageNet ResNet34 result rises to 70.91% top-1, 3.22% over prior methods, at four timesteps.
  • The same recipe transfers across static and neuromorphic datasets: improvements appear on CIFAR-10, CIFAR-100, and CIFAR10-DVS, where ResNet19 reaches 80.50%.
  • The energy overhead of the reversal is small: the paper estimates a 0.52% energy increase over vanilla SNN on CIFAR-10 with ResNet20 at two timesteps.
  • Ablations attribute the gain to the reversal itself: reverting from binary spikes and real weights to real spikes and binary weights raises CIFAR-10/100 accuracy by about 1.3–2.5 points over the vanilla baseline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The re-parameterization step in the paper is only exact when a layer's output feeds a single consumer; on residual architectures, folding $\alpha$ into the main path changes shortcut contributions, so the trained and deployed networks may differ. A direct output-comparison experiment would settle this.
  • The paper's rule of thumb—activation precision binds more than weight precision—suggests other precision swaps worth testing: ternary or integer spikes with binary weights, or real-valued spikes with 2-bit weights, may occupy useful accuracy-efficiency trade-offs.
  • On neuromorphic hardware, real-valued spike events require hardware that can encode analog magnitudes; the addition-only energy advantage is therefore conditional on the hardware's ability to process non-binary event amplitudes, not just binary spikes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes ReverB-SNN, a spiking neural network training scheme that replaces binary spike activations with real-valued spike activations while binarizing weights to {−1, +1}, so that inference remains addition-only and event-driven. A learnable variant scales the binary weights by a trainable factor α and then proposes a re-parameterization (Eqs. 16–18 and Algorithm 1) that folds α into the previous layer's output spike, restoring purely binary weights at inference. The method is evaluated on CIFAR-10, CIFAR-100, ImageNet, and CIFAR10-DVS with ResNet backbones; the headline result is 70.91% top-1 accuracy on ImageNet with ResNet34 at 4 timesteps for the learnable variant, plus accuracy improvements on the other datasets.

Significance. If the central claim is fully established, the paper gives a practical way to combine the information capacity of real-valued spike activations with the addition-only, event-driven advantages of conventional SNNs, and the reported gains over strong baselines on ImageNet and CIFAR10-DVS are substantial. The accuracy tables are a genuine strength: they report means and standard deviations from 3 trials, the results are direct measurements against external baselines, and the paper does not fit constants and then relabel them as predictions. The main significance risk is that the deployment-time equivalence for the learnable variant is not verified for residual architectures, so the efficiency claim for the best-performing model is not yet supported.

major comments (3)
  1. [Section 3.3, Eqs. (16)-(18), Algorithm 1] The re-parameterization is exact only if the activation O_{l-1} is consumed by exactly one downstream operation, namely layer l. In the ResNet18/34 architectures used for the ImageNet results (Table 3), a residual block's input is also consumed by the shortcut path. After setting O_new,l-1 = α_l O_{l-1} and using Eq. (17), the shortcut contribution is changed from O_{l-1} to α_l O_{l-1}, so the re-parameterized network is not functionally identical to the trained α-scaled network unless some residual-specific handling is supplied. The paper neither states this single-consumer assumption nor describes such handling, and it reports no experiment comparing the trained network with the re-parameterized network. The 'addition-only inference' claim for the learnable variant is therefore unverified for exactly the architectures that produce the headline ImageNet accuracy.
  2. [Section 3.3, Eqs. (16) and (18)] The channel-wise implementation of α is dimensionally ambiguous. If α_l is a per-output-channel vector, then α_l Wb_l in Eq. (16) is well defined, but folding α_l into the previous layer's output via Eq. (18) is not, because α_l's dimension does not match the input-channel dimension of O_{l-1}. If α_l is intended to be a scalar per layer, or a vector along the input-channel dimension, this must be stated explicitly because Algorithm 1's folding step depends entirely on that choice.
  3. [Section 4.2, Table 3] The paper does not state whether the reported learnable-variant accuracies (e.g., 70.91% on ImageNet with ResNet34) are measured on the trained α-scaled network or on the re-parameterized inference network. No ablation checks that the trained and re-parameterized networks give the same outputs, and no energy estimate is provided for the learnable variant's deployed form. As a result, the reader cannot determine whether the headline accuracy is compatible with the claimed addition-only inference or whether it reflects a different, multiply-based network.
minor comments (4)
  1. [Table 1 caption] The caption reads 'Ablation study for the ternary spike on CIFAR,' but the table compares the vanilla SNN, ReverB, and the learnable variant; the caption should refer to the proposed ReverB method rather than ternary spikes.
  2. [References] The reference list contains two separate entries for Fang et al. 2021a and 2021b with identical titles ('Deep residual learning in spiking neural networks'); the in-text citations should be disambiguated, or the entries should be merged.
  3. [Section 3.1, Eqs. (1)-(2)] The reset operation is described only in prose ('resets to its resting state') but not included in the equations; since the membrane potential update in Eq. (1) is recurrent, the absence of an explicit reset rule makes the dynamics ambiguous when a spike has just been emitted.
  4. [Algorithm 1] The re-parameterization loop says 'Fold the parameters of α_i into i-1 firing function by Eq. 18,' but Eq. (18) defines a new output for layer l-1; the pseudocode would be clearer if it specified that α_i belongs to layer i and that O_new replaces O_{l-1} in all downstream uses, including any residual paths.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation found: the accuracy numbers are external measurements, the alpha re-parameterization is an explicit algebraic identity, and the paper's heavy self-citations are not load-bearing; the residual-shortcut caveat is a correctness risk, not circularity.

full rationale

Walking the derivation chain, I found no equation that reduces to a fitted quantity and no fitted parameter renamed as a prediction. The central empirical claims (e.g. 70.91% top-1 on ImageNet, Table 3) are measured accuracies compared directly with external baselines such as STBP-tdBN, TET, GLIF, and Real Spike; they do not follow by construction from the model equations. The vanilla ReverB-SNN's addition-only property (Eqs. 8-9) is an algebraic consequence of multiplying a real-valued activation by ±1, stated transparently rather than derived from hidden inputs. For the learnable-alpha variant, Eqs. 16-18 are an explicit re-parameterization: defining O_new = alpha_l O_{l-1} makes U = tau U + alpha_l sign(W)O equal U = tau U + sign(W)O_new by associativity; this is openly called re-parameterization, not a circular reduction. The real weakness is a correctness gap, not circularity: this folding is exact only when an activation tensor has a single downstream consumer, which fails at residual shortcuts in the ResNet backbones; the paper neither states that assumption nor reports a train/inference equivalence check. The related work is heavily self-cited (e.g. Guo et al. 2022d; 2025), but those citations provide context and baselines rather than the load-bearing justification for the reported gains. Accordingly, no circular step is established; the score reflects minor, non-load-bearing self-citation density and the unverified re-parameterization caveat, not a reduction by construction.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No invented physical entities are introduced; real-valued spike activations already appear in prior work such as Real Spike. The central claim rests on four assumptions: the LIF model, the STE surrogate gradient, the energy model, and the re-parameterization equivalence. The first two are standard in the field; the last two are under-validated and are the main sources of correctness risk.

free parameters (3)
  • firing threshold V_th = 0 for static datasets, 0.25 for CIFAR10-DVS
    Set by hand rather than learned; controls sparsity and the real-valued output regime, and affects accuracy and energy estimates.
  • membrane decay constant tau = 0.25
    Chosen by hand for all experiments; controls temporal integration of membrane potential and influences accuracy.
  • learnable weight amplitude alpha_l = learned per channel, final values not reported
    Introduced to increase the capacity of binary weights; learned from data during training and central to the learnable variant.
assumptions (4)
  • domain assumption The LIF neuron dynamics in Eq. 1-2 accurately model the SNN computation.
    The forward pass and the training gradients are built on this model; it is standard in the SNN literature but not derived in the paper.
  • domain assumption The Straight-Through Estimator gradient for sign(W) in Eq. 10 provides a usable training signal.
    Training relies on this surrogate gradient for weight binarization; it is standard in binary neural network research but is an approximation without convergence guarantees.
  • domain assumption Real-valued spikes can be processed in an event-driven way at the same marginal energy as binary spikes.
    The energy claim in Table 5 assumes that per-SOP cost is independent of activation bit-width; no hardware measurement supports this.
  • ad hoc to paper Folding alpha into the previous layer output preserves the trained network function, Eq. 18.
    This is central to the learnable variant. It is invalid when an activation tensor is consumed by multiple downstream paths, as in residual shortcuts, and the paper provides no equivalence check.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ReverB-SNN: Reversing Bit of the Weight and Activation for Spiking Neural Networks." pith.science (2026). https://pith.science/paper/U6WERKXM

@misc{pith2026250607720,
  author       = {Pith},
  title        = {Pith review of: ReverB-SNN: Reversing Bit of the Weight and Activation for Spiking Neural Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/U6WERKXM}},
  note         = {Machine review of arXiv:2506.07720}
}
read the original abstract

The Spiking Neural Network (SNN), a biologically inspired neural network infrastructure, has garnered significant attention recently. SNNs utilize binary spike activations for efficient information transmission, replacing multiplications with additions, thereby enhancing energy efficiency. However, binary spike activation maps often fail to capture sufficient data information, resulting in reduced accuracy. To address this challenge, we advocate reversing the bit of the weight and activation for SNNs, called \textbf{ReverB-SNN}, inspired by recent findings that highlight greater accuracy degradation from quantizing activations compared to weights. Specifically, our method employs real-valued spike activations alongside binary weights in SNNs. This preserves the event-driven and multiplication-free advantages of standard SNNs while enhancing the information capacity of activations. Additionally, we introduce a trainable factor within binary weights to adaptively learn suitable weight amplitudes during training, thereby increasing network capacity. To maintain efficiency akin to vanilla \textbf{ReverB-SNN}, our trainable binary weight SNNs are converted back to standard form using a re-parameterization technique during inference. Extensive experiments across various network architectures and datasets, both static and dynamic, demonstrate that our approach consistently outperforms state-of-the-art methods.

Figures

Figures reproduced from arXiv: 2506.07720 by the authors.

Figure 1
Figure 1. The difference between our SNN and the vanilla SNN. Our SNN differs significantly from the vanilla SNN. The vanilla SNN employs binary spikes, leading to significant information loss of the activations. In contrast, our SNN utilizes real-valued spikes alongside binary weights, thereby enhancing the neuron’s information capacity. This approach retains the benefits of event-driven processing and multiplication-additio… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

24 extracted references · 12 canonical work pages

  1. [6]

    Deep residual learning in spiking neural net- works

    Fang, W., Yu, Z., Chen, Y ., Huang, T., Masquelier, T., and Tian, Y . Deep residual learning in spiking neural net- works. Advances in Neural Information Processing Sys- tems, 34:21056–21069, 2021a. Fang, W., Yu, Z., Chen, Y ., Huang, T., Masquelier, T., and Tian, Y . Deep residual learning in spiking neural net- works. Advances in Neural Information Proc...

    work page 2022

  2. [7]

    URL https://doi

    doi: 10.24963/ijcai.2022/343. URL https://doi. org/10.24963/ijcai.2022/343. Gong, R., Liu, X., Jiang, S., Li, T., Hu, P., Lin, J., Yu, F., and Yan, J. Differentiable soft quantization: Bridging full- precision and low-bit neural networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4852–4861,

    work page doi:10.24963/ijcai.2022/343 2022

  3. [9]

    Hu, Y ., Tang, H., and Pan, G

    URL https: //arxiv.org/abs/1503.02531. Hu, Y ., Tang, H., and Pan, G. Spiking deep residual net- works. IEEE Transactions on Neural Networks and Learning Systems,

    arXiv

  4. [12]

    Deep dive into gradients: Better optimization for 3d object detection with gradient- corrected iou supervision

    Ming, Q., Miao, L., Ma, Z., Zhao, L., Zhou, Z., Huang, X., Chen, Y ., and Guo, Y . Deep dive into gradients: Better optimization for 3d object detection with gradient- corrected iou supervision. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pp. 5136–5145,

    work page 2023

  5. [13]

    doi: 10.1109/CVPR52729.2023. 00497. Neftci, E. O., Mostafa, H., and Zenke, F. Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks. IEEE Signal Processing Magazine, 36(6):51–63,

    arXiv 2023

  6. [14]

    T2fsnn: Deep spiking neural networks with time-to-first-spike coding

    Park, S., Kim, S., Na, B., and Yoon, S. T2fsnn: Deep spiking neural networks with time-to-first-spike coding. In 2020 57th ACM/IEEE Design Automation Conference (DAC), pp. 1–6. IEEE,

    work page 2020

  7. [15]

    Model compres- sion via distillation and quantization

    Polino, A., Pascanu, R., and Alistarh, D. Model compres- sion via distillation and quantization. arXiv preprint arXiv:1802.05668,

    arXiv

  8. [18]

    Spiking PointNet: Spiking Neural Networks for Point Clouds

    Ren, D., Ma, Z., Chen, Y ., Peng, W., Liu, X., Zhang, Y ., and Guo, Y . Spiking pointnet: Spiking neural networks for point clouds. arXiv preprint arXiv:2310.06232,

    work page Pith review arXiv

Show all 24 references
  1. [19]

    net/forum?id=lJdOlWg8td

    URL https://openreview. net/forum?id=lJdOlWg8td. Sun, C., Chen, Q., Fu, Y ., and Li, L. Deep spiking neural network with ternary spikes. In 2022 IEEE Biomedical Circuits and Systems Conference (BioCAS), pp. 251–254. IEEE,

    2022

  2. [20]

    Xu, Q., Li, Y ., Shen, J., Liu, J

    URL https://openreview.net/ forum?id=Siv3nHYHheI. Xu, Q., Li, Y ., Shen, J., Liu, J. K., Tang, H., and Pan, G. Constructing deep spiking neural networks from artifi- cial neural networks with knowledge distillation. arXiv preprint arXiv:2304.05627,

    arXiv

  3. [21]

    Training spiking neural networks with local tandem learning

    Yang, Q., Wu, J., Zhang, M., Chua, Y ., Wang, X., and Li, H. Training spiking neural networks with local tandem learning. arXiv preprint arXiv:2210.04532,

    arXiv

  4. [22]

    Glif: A unified gated leaky integrate-and-fire neuron for spiking neural networks

    Yao, X., Li, F., Mo, Z., and Cheng, J. Glif: A unified gated leaky integrate-and-fire neuron for spiking neural networks. arXiv preprint arXiv:2210.13768,

    arXiv

  5. [23]

    Temporal separation with entropy regularization for knowledge distillation in spiking neural networks

    Yu, K., Yu, C., Zhang, T., Zhao, X., Yang, S., Wang, H., Zhang, Q., and Xu, Q. Temporal separation with entropy regularization for knowledge distillation in spiking neural networks. arXiv preprint arXiv:2503.03144,

    arXiv

  6. [24]

    Zhang, W

    doi: 10.1109/TPAMI.2021.3067100. Zhang, W. and Li, P. Temporal spike sequence learning via backpropagation for deep spiking neural networks. Advances in Neural Information Processing Systems, 33: 12022–12033,

    2021

  7. [1997]

    doi: https://doi.org/10.1016/S0893-6080(97)00011-7

    ISSN 0893-6080. doi: https://doi.org/10.1016/S0893-6080(97)00011-7. URL https://www.sciencedirect.com/ science/article/pii/S0893608097000117. Meng, Q., Xiao, M., Yan, S., Wang, Y ., Lin, Z., and Luo, Z.- Q. Training high-performance low-latency spiking neural networks by diffe...

    doi

  8. [2009]

    Temporal effi- cient training of spiking neural network via gradient re- weighting

    Deng, S., Li, Y ., Zhang, S., and Gu, S. Temporal effi- cient training of spiking neural network via gradient re- weighting. arXiv preprint arXiv:2202.11946,

    arXiv

  9. [2015]

    Simple online and realtime tracking

    Bewley, A., Ge, Z., Ott, L., Ramos, F., and Upcroft, B. Simple online and realtime tracking. In 2016 IEEE in- ternational conference on image processing (ICIP), pp. 3464–3468. IEEE,

    2016

  10. [2017]

    A free lunch from ann: Towards efficient, accurate spiking neu- ral networks calibration

    Li, Y ., Deng, S., Dong, X., Gong, R., and Gu, S. A free lunch from ann: Towards efficient, accurate spiking neu- ral networks calibration. In International Conference on Machine Learning, pp. 6316–6325. PMLR, 2021a. Li, Y ., Guo, Y ., Zhang, S., Deng, S., Hai, Y ., and Gu, S....

    2022

  11. [2018]

    Imagenet: A large-scale hierarchical image database

    Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248–255. Ieee,

    2009

  12. [2019]

    IM-loss: Information maximization loss for spiking neural networks

    Guo, Y ., Chen, Y ., Zhang, L., Liu, X., Wang, Y ., Huang, X., and Ma, Z. IM-loss: Information maximization loss for spiking neural networks. In Oh, A. H., Agarwal, A., Belgrave, D., and Cho, K. (eds.), Advances in Neural Information Processing Systems, 2022a. URL https: //ope...

    2022 arXiv

  13. [2020]

    Enabling deep spiking neural networks with hybrid conversion and spike timing dependent backpropagation

    Rathi, N., Srinivasan, G., Panda, P., and Roy, K. Enabling deep spiking neural networks with hybrid conversion and spike timing dependent backpropagation. arXiv preprint arXiv:2005.01807,

    2005 arXiv

  14. [2022]

    doi: 10.1109/TNNLS.2022. 3153985. Davies, M., Srinivasa, N., Lin, T.-H., Chinya, G., Cao, Y ., Choday, S. H., Dimou, G., Joshi, P., Imam, N., Jain, S., et al. Loihi: A neuromorphic manycore processor with on-chip learning. Ieee Micro, 38(1):82–99,

    2022 doi

  15. [2023]

    Bert: Pre-training of deep bidirectional transformers for lan- guage understanding

    Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. Bert: Pre-training of deep bidirectional transformers for lan- guage understanding. arXiv preprint arXiv:1810.04805,

    arXiv

  16. [2024]

    and Roy, K

    Rathi, N. and Roy, K. Diet-snn: Direct input encoding with leakage and threshold optimization in deep spiking neural networks. arXiv preprint arXiv:2008.03658,

    2008 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.