Pith. sign in

REVIEW 3 major objections 5 minor 17 references

By reassigning the all-zero spike train to the most common activation instead of the rarest, M-TTFS makes a spiking transformer match a quantized transformer’s function and beat prior spiking models on GLUE with a large energy cut.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 06:22 UTC pith:5VBZTWRY

load-bearing objection The M-TTFS encoding idea is genuinely new and the GLUE numbers are strong if they reproduce, but Proposition 1's equivalence theorem is false as stated: Appendix D proves it with non-causal integration, and a simple two-input example with positive weights breaks it. the 3 major comments →

arxiv 2601.22876 v2 pith:5VBZTWRY submitted 2026-01-30 cs.LG

Matterhorn: Masked Time-to-First-Spike Encoding by Reassigning the Silent State for Sparse and Energy-Efficient Spiking Transformers

Zhanglu Yan , Kaiwen Tang , Zixuan Zhu , Zhenyu Bai , Qianhui Liu , Yongxin Zhu , Weng-Fai Wong This is my paper
classification cs.LG
keywords spiking neural networkstime-to-first-spike encodingmasked TTFSdead zonespiking transformersenergy-efficient LLM inferencecompute-in-memoryquantization-to-spiking conversion
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that standard time-to-first-spike encoding uses its only cost-free codeword—the all-zero spike train, or silence—on the wrong value: silence is reserved for rare extreme activations while common activations still fire. M-TTFS fixes this by masking the most frequent firing time so the most common activation becomes silent, and a dead-zone variant silences a whole band of frequent values. The paper shows this is not ad hoc: an M-TTFS layer with a dead zone is functionally equivalent to a trained n-bit quantized network under explicit parameter choices, so spike firing times exactly encode quantized activations. On GLUE, the resulting transformer, Matterhorn, averages 84.64, 1.42 points above the best previous spiking transformer, with a 1.64% spike rate and large measured energy savings under hardware-faithful accounting that includes data movement. A memristive compute-in-memory unit removes weight-access costs, so the induced sparsity translates directly into physical power savings.

Core claim

The central claim is Proposition 1: an M-TTFS spiking layer with dead-zone radius k is functionally equivalent to a trained n-bit QNN layer, provided the time window T=2^n, the most frequent quantized activation µ is aligned with the masked firing time I_max through µ=T/2−1−I_max, the encoding function f(t) is flattened to µ inside the dead zone, and the threshold is the step-wise decreasing function θ(t)=α(T/2−1−⌊t⌋). Under these conditions the integrated membrane potential equals the QNN pre-activation and the firing time decodes to the masked quantized output, so silence represents the most common value rather than the smallest. Empirically, Matterhorn with k=1 reaches an average GLUE sco

What carries the argument

The carrying mechanism is the M-TTFS encoding: a temporal mask M(t) that suppresses the candidate spike at the most frequent firing time I_max (and, with the dead zone, the band I_max−k to I_max+k), reassigning the all-zero silent state to the most common activation while every non-silent value still fires exactly once. The equivalence proof rests on four linked choices: T=2^n, the alignment µ=T/2−1−I_max, a flattened amplitude function f(t) inside the dead zone, and a step-wise decreasing threshold θ(t); these make the firing time a quantized readout of the pre-activation. On the hardware side, the Memristive Synapse Unit (MSU) is an nT1R resistive-memory crossbar that performs bit-serial i

Load-bearing premise

The functional-equivalence proof assumes that, at the moment a neuron fires, its membrane potential already contains every presynaptic spike contribution—but in a causal time-stepped computation, later spikes arrive after an early firing decision has been made, so a large late-arriving signed contribution could change the correct output after the neuron is silenced.

What would settle it

Simulate a single M-TTFS layer with signed binary weights, T=16 and I_max=7, and choose inputs so that a large-magnitude presynaptic spike arrives at a late time step, after the neuron would have fired early under Eq. 1. Compare the decoded firing-time output with the full QNN pre-activation a_j: if they differ for any such input, Proposition 1's integration-phase equivalence does not hold for online causal inference.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Under hardware-faithful accounting, spike transfer and weight reads dominate spiking-transformer energy (42–55% and 27–32% respectively), so encoding choices that cut spike rate matter more than cheaper arithmetic; M-TTFS cuts spike-movement energy 2.46× on SST-2 versus standard TTFS.
  • Reassigning the silent state costs no information as long as the mapping is deterministic; the dead-zone radius k is the tunable knob that trades a controlled amount of information for sparsity, with k=1 the best observed balance.
  • Matterhorn with k=1 achieves 84.64 average GLUE accuracy, 1.42 points above the prior best spiking transformer, and its 1.64% overall spike rate makes most inter-core transfers vanish.
  • The QNN-to-SNN equivalence means a QNN trained with dead-zone distillation can be converted layer-by-layer into a spiking network whose firing times carry exactly the masked quantized activations, so accuracy is preserved by construction.
  • Pairing M-TTFS with the MSU lowers a transformer block's energy from 16.80 mJ to 6.14 mJ, a 2.7× reduction, and the same encoding principle extends consistently to spiking LLaMA models from 7B to 70B parameters.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Our reading: the same coding principle—assign the free all-zero codeword to the most frequent symbol—is a general energy-aware coding rule for any event-driven system, not just spiking transformers; other sparse-communication schemes could be reordered by measured activation statistics rather than by magnitude.
  • A testable extension is to make I_max or the dead-zone radius k data-dependent per layer or per head rather than globally fixed at 7, which could push the accuracy-sparsity frontier further.
  • The paper's energy accounting method implies that earlier spiking-LLM efficiency ratios based only on operation counts may overstate gains; a fair comparison would re-evaluate prior methods on the same data-movement-aware model.
  • The asymmetric variant in Appendix E suggests M-TTFS can handle non-negative activations such as ReLU outputs and softmax scores with even higher silence rates on SST-2, pointing toward direct application in attention-score paths.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes M-TTFS, a time-to-first-spike encoding that reassigns the all-zero 'silent' spike train to the most frequent activation value via a temporal mask at I_max, and extends it with a dead zone of radius k. It also introduces a Memristive Synapse Unit (MSU) for compute-in-memory inference, and reports a spiking transformer called Matterhorn that reaches 84.64 average GLUE score with k=1, exceeding the best prior spiking transformer by 1.42 points while consuming substantially less energy under a hardware-aware accounting. The central theoretical claim is Proposition 1: under T=2^n, μ=T/2−1−I_max, a flattened decay f(t), and threshold θ(t)=α(T/2−1−⌊t⌋), an M-TTFS layer with dead zone is functionally equivalent to a trained n-bit QNN layer. The proof in Appendix D attempts to establish this via an integration-phase and a firing-phase equivalence.

Significance. The paper addresses a real and important issue: operation-count-based energy evaluation of SNNs overlooks data movement, and the idea of assigning the zero-energy codeword to the most frequent value is intuitively appealing. The hardware-faithful energy modeling, the attempt to tie sparsity to physical power gating, and the MSU design are also relevant contributions if the underlying conversion is valid. However, the central functional-equivalence theorem is false under the stated dynamics, and the reported SNN accuracy and energy results are supported only by that theorem. Without a corrected proof or direct event-driven validation of the converted SNN, the paper cannot support its main claims.

major comments (3)
  1. [Appendix D, Eq. (30) (with Eq. (1))] The integration-phase proof assumes V_j^l = a_j^l, i.e., that the membrane potential equals the full QNN pre-activation. But Eq. (1) defines V_j^l(t) as the causal sum of PSPs from spikes with t_i ≤ t, and firing is the first t with V_j^l(t) ≥ θ^l(t). A neuron that fires early has not yet received later spikes. This is not a signed-weight edge case: it fails with positive weights and positive activations. Example with T=16, α=1, b=0, I_max=7, k=1: inputs q=(3,2) with weights (1,1) produce spikes at t=4 and t=5. At t=4, V=3 and θ(4)=7−4=3, so the neuron fires and decodes q̂=3, while the QNN pre-activation is 5 and quantizes to 5. Thus the earliest threshold crossing does not equal the full pre-activation, and the derivation in D.2 (Eqs. 32–41) is unsupported.
  2. [Section 3.2, Eqs. (11)–(13), and Table 1] Proposition 1 is the sole justification for the QNN-to-SNN conversion and for the dead-zone training objective in Eqs. (11)–(13). Since Proposition 1 is false as stated, the conversion pipeline is unsupported. The paper reports Matterhorn's GLUE accuracy in Table 1 without presenting any event-driven simulation or measurement of the converted spiking network; the accuracy numbers appear to be inherited from the QNN. If the mapping is not functionally equivalent, these results are not evidence about the SNN. The authors need either to repair the theorem with explicit conditions under which the earliest crossing partial sum equals the full pre-activation (e.g., a schedule where integration completes before threshold comparison) or to provide direct SNN inference results.
  3. [Section 4.4, Tables 2 and 3, Figs. 5–7] The energy-efficiency claims are also tied to the same equivalence: Section 4.4 compares Matterhorn (k=1) with MSU against 'its equivalent QNN baseline' using Proposition 1. If the equivalence fails, the energy numbers do not describe a network that computes the reported QNN outputs. In addition, the reported savings are internally inconsistent: the abstract states 67% less energy, while Section 4.4 reports 57% (14.21→6.14 mJ); the improvement factor appears as 2.31×, 2.35×, and 2.7× in different places. These discrepancies need reconciliation before the energy claims can be assessed.
minor comments (5)
  1. [Abstract, Section 1, Section 4.2, Fig. 5] The 'overall spike rate' of 1.64% or 1.65% is not consistent with Fig. 5(c), where k=1 on SST-2 gives 61.2% silent neurons, i.e., 38.8% active; with T=16 this corresponds to a per-timestep spike rate of about 2.4%, not 1.65%. The definition of spike rate should be stated explicitly.
  2. [Eq. (13)] The masked STE gradient is written as ∂L/∂x_q ≈ ∂L/∂a · indicator · M_qnn, but the standard chain rule gives ∂L/∂a = ∂L/∂x_q · indicator · M_qnn. As written, the equation is dimensionally inconsistent and likely a typo.
  3. [Fig. 6] The y-axis is labeled 'Energy (mJ)' and the text says 'energy consumption', but the reported values (6.98, 4.75, 2.84, ...) appear to be spike-movement energy only, not total energy as in Table 2. Please label the quantity precisely.
  4. [Section 4.2] I_max=7 is fixed globally across all layers and heads, but Proposition 1 requires μ to be the most frequent QNN activation per layer. The paper does not justify this global choice with per-layer statistics.
  5. [Appendix E] Proposition 2 (asymmetric quantization) is stated without proof; the figures do not substitute for a derivation. The asymmetric case should receive the same detailed treatment as Proposition 1.

Circularity Check

1 steps flagged

Proposition 1's equivalence proof assumes the full QNN pre-activation is available at the firing instant, contradicting the causal membrane update in Eq. (1); the central conversion claim is therefore asserted rather than derived, while the measured energy/accuracy results remain independent.

specific steps
  1. self definitional [Appendix D.1 (Eq. 30) and D.2 (Eq. 32), versus Section 2.1 (Eq. 1)]
    "V^l_j(t) = V^l_j(t−1) + Σ_i w^l_{ij} · s^{l−1}_i(t) · f(t) ... The integrated membrane potential at layer l sums all incoming PSPs and adds the bias: V^l_j = Σ_i ϵ(w^l_{ij}, t_i) + b^l_j ... we conclude V^l_j = a^l_j ... The SNN neuron fires at the earliest time t∈[0,T−1] such that its membrane potential reaches the time-varying threshold ... a^l_j ≥ θ^l(t)."

    Eq. (1) defines V^l_j(t) as a running sum over spikes that have arrived by time t, and firing is the first crossing of θ. The proof replaces this running sum by the final full sum a^l_j in Eq. (30), then uses a^l_j in the threshold condition (32). Thus the theorem assumes, rather than proves, that all PSPs are integrated before the earliest firing time. The assumption is false even for positive weights: with T=16, A=7, inputs q=(3,2), weights 1, b=0, the neuron fires at t=4 with V=3=θ(4), while a=5; the decoded value A−t=3 differs from the QNN output 5. The claimed equivalence is the unproved equality V(t_fire)=a.

full rationale

The measured GLUE accuracy and energy numbers are benchmark results, not derived predictions, so those parts are not circular. The comparison baseline and energy methodology draw on the authors' prior work, but the comparison is empirical and the analog parameters come from an external chip paper; this does not make the central claim circular. The load-bearing derivation is Proposition 1. In Appendix D, the integration-phase proof sets the membrane potential at firing equal to the full QNN pre-activation, whereas Eq. (1) makes the membrane potential a causal partial sum. The firing-phase proof then uses that full value in the threshold inequality. This is a definitional substitution: V at firing = a is exactly the equivalence to be shown. A concrete positive-weight counterexample shows the reduction is not harmless. Because the conversion pipeline (Eqs. 11-13) is justified by this proposition, the theoretical claim is partly circular/unsupported as written. The empirical results are independent but do not validate the theorem.

Axiom & Free-Parameter Ledger

3 free parameters · 6 axioms · 1 invented entities

The central claim rests on a set of engineering projections and one mathematically false causal assumption. The only genuinely new free design choices are the global mask time Imax and the dead-zone radius k, plus a trainable scaling constant in the MSU. The proof of functional equivalence relies on the non-causal full-integration assumption, which is the weakest point.

free parameters (3)
  • Imax (masked firing time) = 7
    Fixed globally for all layers/heads; assumed to be the most frequent firing time of the QNN/SNN; sets the silent state to q=0 under Proposition 1. If the true mode differs, the mask is suboptimal.
  • dead-zone radius k = 1
    Selected from the SST-2 energy/accuracy ablation as the reported operating point; used for all GLUE tasks and energy comparisons.
  • γ (MSU scaling) = trainable
    Learnable scaling in Eq. 14 to reconstruct signed synaptic values from CIM output; a model parameter not derived from first principles.
axioms (6)
  • ad hoc to paper The SNN neuron has integrated all presynaptic spikes before threshold comparison (V_j = a_j at firing time)
    Assumed in Appendix D.1 Eq. 30; not true for the causal recurrence in Eq. 1, where late spikes cannot contribute to an earlier firing decision.
  • domain assumption A single global Imax=7 is the most frequent firing time for every layer/head
    Used to configure the mask; no per-layer or per-dataset estimation shown.
  • domain assumption RRAM CIM macro energy/noise properties from Ye et al. 2023 transfer to the proposed MSU
    Analog energy (2.164 fJ/bit), cell density, and noise immunity are adopted from the cited 28nm macro without MSU-specific silicon.
  • domain assumption Spatial dataflow architecture with NoC and SRAM weight reads is the correct hardware model
    Energy decomposition in §2.3; unit costs from the authors' synthesized Verilog on 22nm, not shipped.
  • domain assumption Analog noise is negligible
    Section 5 asserts noise is suppressed by 1b×1b bit-serial CIM and binary weights, but no noise model or measurement is provided.
  • standard math Integer floor/ceiling identities in symmetric quantization
    Appendix D uses the ⌈A−r⌉ identity and clipping to match QNN quantization; this is standard integer math.
invented entities (1)
  • Memristive Synapse Unit (MSU) no independent evidence
    purpose: Compute-in-memory analog crossbar to eliminate weight access energy and perform bit-serial VMM
    No fabricated chip or measured silicon; energy and area are projections from Ye et al. 2023 crossbar and 22nm synthesis.

pith-pipeline@v1.3.0-alltime-deepseek · 17846 in / 22836 out tokens · 219527 ms · 2026-08-03T06:22:15.110276+00:00 · methodology

0 comments
read the original abstract

Spiking neural networks (SNNs) promise energy-efficient inference for large language models (LLMs), yet most reported savings rely on compute-operation counts that overlook data movement. Energy characterization of representative spiking transformers on a commercial 22-nm process shows that accumulation contributes less than 3% of total energy, while spike-triggered inter-core transfers and weight reads dominate the cost. This makes time-to-first-spike (TTFS) encoding a natural choice, as it limits each neuron to at most one spike. However, standard TTFS maps the silent state, an all-zero spike train that transmits no events, to the rarely occurring smallest value, while the most common activations still spike. This raises a simple question: why reserve the only cost-free codeword for a rare value? This choice inverts a basic principle of energy-aware coding, under which the zero-event codeword should represent the most common value, rather than a rare extreme. Thus, we introduce masked time-to-first-spike encoding (M-TTFS), which uses a temporal mask to reassign the silent state to the most common activation value, and a dead-zone extension that trades a controlled amount of information for greater sparsity. Built on M-TTFS with dead-zone radius $k{=}1$, our spiking transformer Matterhorn reaches an overall spike rate of 1.64% on GLUE at an average score of 84.64, exceeding the best prior spiking transformer by 1.42 percentage points while consuming 67% less energy, with consistent gains on spiking LLaMA models from 7B to 70B parameters. Together, these results show that under hardware-faithful accounting, the energy advantage of SNNs is not a given: it is earned by encodings that align spikes with the data distribution.

Figures

Figures reproduced from arXiv: 2601.22876 by Kaiwen Tang, Qianhui Liu, Weng-Fai Wong, Yongxin Zhu, Zhanglu Yan, Zhenyu Bai, Zixuan Zhu.

Figure 1
Figure 1. Figure 1: Proportional Energy Breakdown of SOTA spiking trans￾formers. Otters (standard TTFS encoding version is chose), Sorbet (rate encoding), SpikingBERT (rate encoding), and SpikingLM (rate encoding) are evaluated with time steps T set to 15, 16, 16, and 4, and average spike rates of 5.14%, 13%, 25%, and 33%, respectively, as reported in (Bal & Sengupta, 2024; Xing et al., 2024b; Tang et al., 2025; Yan et al., 2… view at source ↗
Figure 2
Figure 2. Figure 2: M-TTFS with dead zone. M(t) = ( 0, if t ∈ Ωdead 1, otherwise (8) By adjusting the radius k, we can explicitly control the trade-off between information retention and sparsity. 3.2. QNN-to-SNN conversion To address the training challenges discussed in Section 2.1, we adopt a QNN-to-SNN conversion framework. This ap￾proach involves training a source QNN and mapping its parameters to an M-TTFS SNN to ensure f… view at source ↗
Figure 3
Figure 3. Figure 3: M-TTFS spiking transformer structure. To satisfy the functional equivalence constraints derived in Proposition 1, we incorporate the dead-zone logic directly into the QNN training loop using knowledge distillation (Liu et al., 2022). In the forward pass, pre-activations are quan￾tized and passed through a dead-zone filter centered at µ. Values falling within a radius k of µ are mapped to µ to simulate the … view at source ↗
Figure 4
Figure 4. Figure 4: Memristive Synapse Unit (MSU) architecture. 5 [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 6
Figure 6. Figure 6: quantifies the trade-off between spike movement energy and accuracy on the SST-2 dataset. Compared to the standard TTFS baseline, M-TTFS with k=0 reduces energy consumption to 4.75 mJ while maintaining an identical ac￾curacy of 92.55%. Subsequently, increasing the dead zone to k = 1 yields the optimal balance point, decreasing energy use by 59% to 2.84 mJ with a negligible accuracy drop of only 0.9%. Final… view at source ↗
Figure 7
Figure 7. Figure 7: Comparison with SOTA spiking transformer We further compare Matterhorn (k = 1) with MSU against its equivalent QNN baseline, as derived in Proposition 1. While both models achieve identical accuracy due to their mathematical equivalence, Matterhorn demonstrates supe￾rior efficiency. For the same transformer block on SST-2 dataset, the QNN consumes 14.47 mJ, whereas Matterhorn delivers a 2.35× improvement i… view at source ↗
Figure 8
Figure 8. Figure 8: Each cell contains a resistive memory device, which stores the weight as conductance, and an access transistor that acts as a selector switch. The array utilizes three sets of orthogonal lines for control and data: Word Lines (WL) control the transistor gates to select active rows; Source Lines (SL) supply the analog input voltages; and Bit Lines (BL) collect the accumulated summation current for each colu… view at source ↗
Figure 9
Figure 9. Figure 9: Schematic of the typical analog crossbar architecture. Inputs are applied to Source Lines, and accumulated currents are read from Bit Lines. Since analog computation is most efficient at lower precision, we employ a mixed-signal strategy that processes inputs in a bit-serial manner. We decompose a n-bit activation vector A⃗ into binary bit-vectors A⃗ n. The full vector-matrix multiplication (VMM) is then r… view at source ↗
Figure 10
Figure 10. Figure 10: Self attention operation using time-based accumulation. C. Floorplan of the Mixed-Signal MSU Prior work (Ye et al., 2023) demonstrated an 8Kb (64×128) array prototype with a cell size of about 0.585 µm2 . As a research test chip, the original layout allocated significant area to peripheral test circuits, resulting in low array efficiency. 12 [PITH_FULL_IMAGE:figures/full_fig_p012_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: illustrates the proposed floorplan for the MSU. Analog crossbar arrays typically result in unutilized silicon area at the macro corners. We leverage this space to embed necessary digital components, including decoders, adders, and accumulators. To ensure signal integrity, we allocate a 6 µm keep-out zone between the analog and digital domains, isolating sensitive analog signals from digital switching nois… view at source ↗
Figure 12
Figure 12. Figure 12: Comparison of spike time distributions on the SST-2 dataset for the asymmetric variant. We compare the traditional and our proposed M-TTFS method under different dead-zone parameters k ∈ {0, 1, 2, 3} (a-d). 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Spike Time 0 10 20 30 40 Frequency (%) Silent 34.9% Active (a) QQP M-TTFS(k=0) 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Spike Time 0 10 20 30 40 50 60 Frequency (%) S… view at source ↗
Figure 13
Figure 13. Figure 13: M-TTFS method under different dead-zone parameters k ∈ {0, 1} F. Full ablation study for MTTFS for all other datasets in GLUE We evaluate our model on seven datasets from the GLUE benchmark and report the spike time distribution of the M-TTFS method under different dead-zone parameters in [PITH_FULL_IMAGE:figures/full_fig_p016_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Matterhorn energy breakdown 17 [PITH_FULL_IMAGE:figures/full_fig_p017_14.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

17 extracted references · 4 linked inside Pith

  1. [6]

    doi: 10.1109/JSSC.2023.3234893. Khan, A. A., Farzaneh, H., Friebel, K. F. A., Fournier, C., Chelini, L., and Castrillon, J. Cinm (cinnamon): A compilation infrastructure for heterogeneous compute in-memory and compute near-memory paradigms. In Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Opera...

    arXiv 2023

  2. [7]

    10067540

    doi: 10.1109/ISSCC42615.2023. 10067540. Prabhakar, R., Jairath, S., and Shin, J. L. Sambanova sn10 rdu: A 7nm dataflow architecture to accelerate software 2.0. In2022 IEEE International Solid-State Circuits Con- ference (ISSCC), volume 65, pp. 350–352. IEEE,

    arXiv 2023

  3. [9]

    Energy-efficient and dequantization-free Q-LLMs: A spiking neural network approach to salient value mitiga- tion.arXiv preprint arXiv:2510.19498,

    Wang, C., Yan, Z., Zhou, Z., Chen, X., and Wong, W.- F. Energy-efficient and dequantization-free Q-LLMs: A spiking neural network approach to salient value mitiga- tion.arXiv preprint arXiv:2510.19498,

    arXiv

  4. [10]

    9 Preprint Xiao, G., Lin, J., Seznec, M., Wu, H., Demouth, J., and Han, S

    doi: 10.1109/ISSCC49657.2024.10454468. 9 Preprint Xiao, G., Lin, J., Seznec, M., Wu, H., Demouth, J., and Han, S. Smoothquant: Accurate and efficient post-training quantization for large language models. InInterna- tional conference on machine learning, pp. 38087–38099. PMLR,

    arXiv 2024

  5. [11]

    A., Xiao, S., Du, L., Li, G., and Zhang, J

    Xing, X., Gao, B., Zhang, Z., Clifton, D. A., Xiao, S., Du, L., Li, G., and Zhang, J. Spikellm: Scaling up spiking neural network to large language models via saliency- based spiking.arXiv preprint arXiv:2407.04752, 2024a. Xing, X., Zhang, Z., Ni, Z., Xiao, S., Ju, Y ., Fan, S., Wang, Y ., Zhang, J., and Li, G. Spikelm: Towards general spike-driven langua...

    Pith/arXiv arXiv

  6. [12]

    doi: 10.1109/TPAMI.2023. 3286121. Yan, Z., Bai, Z., and Wong, W.-F. Reconsidering the energy efficiency of spiking neural networks.arXiv preprint arXiv:2409.08290,

    Pith/arXiv arXiv 2023

  7. [13]

    Otters: An energy-efficient spik- ingtransformer via optical time-to-first-spike encoding

    Yan, Z., Mao, J., Liu, Q., Li, F., Pan, G., Luo, T., Zhu, B., and Wong, W.-F. Otters: An energy-efficient spik- ingtransformer via optical time-to-first-spike encoding. arXiv preprint arXiv:2509.18968, 2025a. Yan, Z., Tang, K., Zhou, J., and Wong, W.-F. Low latency conversion of artificial neural network models to rate- encoded spiking neural networks.IEE...

    arXiv 2025

  8. [14]

    Zhang, W., Hou, L., Yin, Y ., Shang, L., Chen, X., Jiang, X., and Liu, Q

    1109/JSSC.2023.3280357. Zhang, W., Hou, L., Yin, Y ., Shang, L., Chen, X., Jiang, X., and Liu, Q. Ternarybert: Distillation-aware ultra-low bit bert. InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 509–521,

    arXiv 2023

  9. [15]

    Spikingformer: Spike-driven residual learn- ing for transformer-based spiking neural network.arXiv preprint arXiv:2304.11954,

    Zhou, C., Yu, L., Zhou, Z., Ma, Z., Zhang, H., Zhou, H., and Tian, Y . Spikingformer: Spike-driven residual learn- ing for transformer-based spiking neural network.arXiv preprint arXiv:2304.11954,

    arXiv

  10. [16]

    2024.3483632

    doi: 10.1109/TC. 2024.3483632. 10 Preprint A. Analog-Centric Mixed-Signal In-Memory Computing Analog-centric in-memory computing leverages device physics to perform matrix multiplication directly within memory arrays, eliminating the latency and energy costs of data movement. The core mechanism maps multiply-accumulate (MAC) operations to circuit current ...

    arXiv 2024

  11. [17]

    G. Energy calculation We evaluate energy consumption based on a standard BERT-base architecture (Batch size B=64, Sequence length S=128, Hidden dimension H=768, FFN dimension 3072, Heads h=12). All digital computation, memory access, and data movement costs are calibrated using measurements from a commercial 22nm process, while analog computing parameters...

    2023

  12. [2018]

    112130359

    doi: 10.1109/MM.2018. 112130359. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. Bert: Pre-training of deep bidirectional transformers for lan- guage understanding. InProceedings of the 2019 confer- ence of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pp. 4...

    doi:10.1109/mm.2018 2018

  13. [2019]

    Tinybert: Distilling bert for natural language understanding

    Jiao, X., Yin, Y ., Shang, L., Jiang, X., Chen, X., Li, L., Wang, F., and Liu, Q. Tinybert: Distilling bert for natural language understanding. InFindings of the Association for Computational Linguistics: EMNLP 2020, pp. 4163– 4174,

    2020

  14. [2022]

    Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter.arXiv preprint arXiv:1910.01108,

    Sanh, V ., Debut, L., Chaumond, J., and Wolf, T. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter.arXiv preprint arXiv:1910.01108,

    Pith/arXiv arXiv 1910

  15. [2023]

    Davies, M., Srinivasa, N., Lin, T.-H., Chinya, G., Cao, Y ., Choday, S

    doi: 10.1109/TETCI.2022.3214509. Davies, M., Srinivasa, N., Lin, T.-H., Chinya, G., Cao, Y ., Choday, S. H., Dimou, G., Joshi, P., Imam, N., Jain, S., Liao, Y ., Lin, C.-K., Lines, A., Liu, R., Mathaikutty, D., McCoy, S., Paul, A., Tse, J., Venkataramanan, G., Weng, Y .-H., Wild, A., Yang, Y ., and Wang, H. Loihi: A neuromorphic manycore processor with on...

    arXiv 2022

  16. [2024]

    1016/j.nanoen.2024.109646

    doi: https://doi.org/10. 1016/j.nanoen.2024.109646. Bal, M. and Sengupta, A. Spikingbert: Distilling bert to train spiking language models using implicit differentia- tion. InProceedings of the AAAI conference on artificial intelligence, volume 38, pp. 10998–11006,

    arXiv 2024

  17. [2025]

    Dampfhoffer, M., Mesquida, T., Valentian, A., and Anghel, L

    URLhttps://arxiv.org/abs/2410.23619. Dampfhoffer, M., Mesquida, T., Valentian, A., and Anghel, L. Are snns really more energy-efficient than anns? an in-depth hardware-aware study.IEEE Transactions on Emerging Topics in Computational Intelligence, 7(3):731– 741,

    Pith/arXiv arXiv