{"id":"5809d655-90f9-4850-8c0d-99ba8c6b9c5a","arxiv_id":"2502.01330","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Sparse, 8-bit quantized S5 linear RNNs match dense model audio denoising accuracy with 2x less compute and 36% less memory, and run 42x faster with 149x lower energy on Loihi 2 than a dense FP32 model on Jetson Orin Nano.","lead":"This paper shows that a heavily pruned and quantized linear recurrent network (S5) can match dense models on streaming audio denoising while using about half the compute and a third less memory. Deployed on Intel's Loihi 2 chip, the sparse model runs far faster and more efficiently than a dense model on an edge GPU, though hardware differences contribute to the gains.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Hardware speedup claim rests on a confounded iso-accuracy baseline: Loihi's sparse W8A16 model is compared to a dense FP32 GPU model, and the deployed Loihi model's on-chip SI-SNR is never reported.","rationale":"The paper's analytical contribution—the Pareto front in Figure 4 and the released code—is a credible empirical study: sparse ReLU-fied S5 models match dense GELU S5 models at lower effective MACs and memory on N-DNS, with a similar trend on SpeechCommands V2-35, and the MAC/memory accounting in Appendix A.1 is transparent. The load-bearing weakness is not the Pareto analysis but the hardware comparison used to support the 42x/149x claim. The comparison varies three factors at once: platform (Loihi 2 vs Jetson), precision (W8A16 vs FP32), and sparsity. The paper explicitly disclaims the quantized GPU baseline, and Figure 6/Appendix A.3.3 show fixed-point conversion degrades SI-SNR. Because Table 1 does not report the deployed models' SI-SNR, the 'iso-accuracy' premise is unverified. A W8A16 dense GPU baseline and on-chip accuracy numbers would settle whether sparsity, quantization, or platform is responsible. This does not overturn the core scaling result, so conditional acceptance remains appropriate.","tokens_in":19751,"tokens_out":8181,"duration_ms":73508,"concrete_test":"Run the dense-3 S5 model on the Jetson Orin Nano with W8A16 static quantization and fixed-point inference (TensorRT or the paper's own fixed-point JAX implementation) and measure per-token latency and energy, while also reporting the SI-SNR of the deployed Loihi sparse-8 model and the dense-3 baseline on the N-DNS test set. If the quantized dense GPU is within 2-3x of Loihi, or if Loihi's on-chip SI-SNR falls below dense-3's, the headline hardware claims do not hold at iso-accuracy.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline result—42x lower latency and 149x lower energy at iso-accuracy (Abstract, Section 3.3, Table 1)—does not isolate unstructured sparsity as the source of the gain. The Loihi 2 model is sparse and quantized (W8A16), while the Jetson Orin Nano runs a dense FP32 model; the Limitations section concedes that the fixed-point JAX model 'provides no improvements in runtime or energy' and that a more optimized quantized Jetson implementation could improve the GPU numbers. In addition, Section 3.3 reports that the fixed-point and Loihi implementations 'again find a degradation in SI-SNR' relative to FP32 (see Figure 6 and Appendix A.3.3), yet Table 1 does not report SI-SNR for the deployed sparse-8 model or the dense-3 baseline. The 'iso-accuracy' qualifier is therefore asserted rather than demonstrated at the point of hardware measurement. If the deployed Loihi model is less accurate than the dense-3 baseline, or if a W8A16 dense GPU baseline closes most of the energy/latency gap, the 42x/149x numbers are not evidence for sparsity at equal accuracy.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a compression pipeline for linear RNNs (S5) that combines iterative magnitude pruning to 90% weight sparsity, ReLU-based activation sparsification, and W8A16 quantization-aware training, and deploys the result on Intel Loihi 2 for streaming audio denoising. The authors report a scaling study showing that sparse models are Pareto-optimal against dense baselines in effective MACs and memory, with sparse-8 matching dense-3 with 2x less compute and 36% less memory, and they report hardware measurements in which a sparse quantized model on Loihi 2 achieves 42x lower latency and 149x lower energy per token than a dense FP32 model on an NVIDIA Jetson Orin Nano.","tokens_in":20016,"tokens_out":9943,"duration_ms":92990,"significance":"If the central claims hold, the paper is a useful contribution: it provides a concrete recipe for sparse linear RNN training, reports measured activation densities and effective MACs rather than assuming sparsity, includes a second task in the appendix, releases code, and demonstrates a real neuromorphic deployment of an S5-style model. The Pareto-front result for sparse models is the strongest part and is supported by explicit training and pruning details. The headline hardware numbers are important but currently rest on a confounded baseline: the Loihi 2 model is sparse and quantized while the Jetson model is dense and FP32, and the paper's own limitations section concedes that the fixed-point JAX model gives no runtime or energy improvement on Jetson. As written, the hardware claims do not isolate unstructured sparsity as the cause of the reported gains.","major_comments":[{"comment":"The 42x/149x comparison is confounded: the Loihi 2 model is sparse and W8A16-quantized, while the Jetson Orin Nano runs a dense FP32 model. Footnote 1 states that the W8A16 fixed-point model in JAX provides no speedup over FP32 on the Jetson, and the Limitations section concedes that a more optimized quantized Jetson implementation could improve the GPU numbers. Therefore the reported gains are not attributable to unstructured sparsity alone; they conflate sparsity, quantization, and hardware platform. Please add a dense W8A16 (or otherwise quantized) Jetson baseline, for example via TensorRT INT8, and report its latency and energy, so that the contribution of sparsity can be separated from the contribution of the platform.","section":"Section 3.3, Table 1, footnote 1, Limitations"},{"comment":"The 'iso-accuracy' comparison is not verified at the point of hardware measurement. Table 1 states that the Jetson model is 'a smaller dense S5 model that reaches similar test performance,' but no SI-SNR is reported for the deployed sparse-8 model on Loihi 2 or for the dense-3 baseline on Jetson. Section 3.3 says the fixed-point simulation and the Loihi 2 implementation 'again find a degradation in SI-SNR' relative to FP32, and Appendix A.3.3 describes an accumulating fixed-point mismatch. Without reporting the actual on-chip SI-SNR values and the tolerance used for the iso-accuracy criterion, the 42x/149x comparison may be at unequal accuracy.","section":"Section 3.3, Figure 6, Appendix A.3.3, Table 1"},{"comment":"The Pareto comparison is between 'Dense w/ GELU' and 'Sparse w/ ReLU,' so the gains conflate weight pruning with the ReLU/activation-sparsity surgery. ReLU-fication by itself changes the activation density and can reduce effective MACs, and Figure 5 shows that ReLU-fied dense models do exhibit substantial activation sparsity. Please add the dense-ReLU family to the Pareto plot, or alternatively state explicitly that the comparison is the full pruning+ReLU pipeline against the original GELU baseline, and adjust the attribution of the efficiency gains accordingly.","section":"Figure 4, Section 2.4, Figure 5"}],"minor_comments":[{"comment":"The phrase '36% less memory iso-accuracy' is missing a preposition; it should read '36% less memory at iso-accuracy.'","section":"Abstract and Section 1"},{"comment":"The reference 'Google. Aqt: Accurate quantized training. https://github.com/charlespwd/project-title, 2024' appears to point to a placeholder URL; it should be replaced with the actual AQT repository citation.","section":"References"},{"comment":"The caption says 'Dashed horizontal like marks SI-SNR' and should say 'line.' The phrase 'iso-accuracy' in the caption should also be clarified to indicate which model is used as the accuracy reference.","section":"Figure 4 caption"},{"comment":"The notation in Eq. (3), specifically 'W{i,j|xj≠0}x{i|xi≠0}', is ambiguous because the subscripts are not defined precisely; please define the per-layer pre-activation vector and the dynamic set of nonzero entries.","section":"Section 2.4, Eq. (3)"},{"comment":"The ERK formula s_t^l = s_t (N^l + M^l) / (N^l M^l) does not appear to be normalized so that the global sparsity equals s_t; please clarify the normalization or point to the exact JaxPruner implementation used.","section":"Section 2.4, Eq. (13)"}],"recommendation":"major_revision","confidential_remarks":"The central Pareto-front claim is credible and the deployment effort is substantial, but the hardware headline needs a quantized dense GPU baseline and on-chip accuracy reporting before the 42x/149x numbers can be accepted as evidence for unstructured sparsity. These are fixable within the scope of a revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a solid empirical study. The core simulation result—that highly sparse, ReLU-fied S5 models beat dense baselines on the Pareto front of effective MACs and memory at iso-accuracy on the N-DNS audio task—is well supported. The MAC metric uses measured activation densities, the training and pruning details are explicit, and the scaling study over model widths is genuinely useful. The observation that weight pruning reduces activation sparsity is a nice, reproducible finding. Code and hardware profiling details are included. I believe the numbers on the simulated Pareto front. The hardware section is where I start to squint. The headline 42x/149x numbers compare a sparse, quantized Loihi 2 model to a dense FP32 model on a Jetson Orin Nano. That is a confounded comparison: you are changing both the model and the platform. The paper is honest about this in the Limitations section, and the footnote that the JAX fixed-point model provides no speedup on the Jetson is refreshingly candid. But the abstract's language—'tangible gains' from 'model compression'—overstates what is actually isolated. If you want to claim sparsity plus quantization gives those gains, you need a W8A16 dense baseline on the same GPU, which the paper itself admits could close much of the gap. The other soft spot is that Table 1 never reports SI-SNR for the deployed Loihi model or the dense-3 baseline. Section 3.3 says the fixed-point and Loihi implementations 'again find a degradation in SI-SNR' relative to FP32. So the 'iso-accuracy' qualifier for the hardware comparison is asserted, not demonstrated. It may well be that sparse-8 on Loihi lands slightly below dense-3 on the Jetson, and the speedups are still impressive, but they should be reported with the actual accuracy numbers. These are fixable issues, not fatal flaws. The paper deserves a serious referee. I would ask for a quantized GPU baseline, the on-chip SI-SNR for the deployed models, and error bars on the hardware measurements. The simulation-level contribution is worth publishing on its own.","headline":"Good simulation-level evidence for sparse RNN Pareto gains; hardware speedup claim needs a quantized baseline and on-chip accuracy numbers before it can be taken at face value.","tokens_in":699,"tokens_out":584,"would_cite":true,"duration_ms":21996,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Linear RNNs pruned to 90% sparsity and quantized to fixed point match dense models at 2x less compute, and run 42x faster with 149x less energy on a neuromorphic chip.","keywords":["linear recurrent neural networks","unstructured sparsity","activation sparsity","iterative magnitude pruning","quantization-aware training","fixed-point arithmetic","neuromorphic hardware","streaming audio denoising"],"falsifier":"Measure per-token latency and energy for a dense W8A16 fixed-point version of the equally accurate S5 model on the same edge GPU; if that quantized dense baseline approaches the neuromorphic chip's latency and energy numbers, then the claimed gains come mostly from hardware differences rather than from sparsity.","tokens_in":19526,"feed_emoji":"⚡","tokens_out":33725,"duration_ms":244495,"temperature":0.7,"pith_summary":"Linear recurrent networks compress a whole sequence into a fixed-size hidden state, which makes them attractive for streaming applications on power- and size-limited devices — but only if they can be made cheap enough to run there. This paper tries to establish that unstructured sparsity is that lever: train a linear RNN of the S5 type (a recurrent matrix diagonalized in the complex domain) to 90% weight sparsity with iterative magnitude pruning, force activations sparse by replacing GELU with ReLU, and widen the model to compensate. On a streaming audio denoising benchmark, the sparse family then dominates the efficiency-accuracy Pareto front, matching a dense model at 2x less compute and 36% less memory, and matching the previous state of the art with 3.2x less compute and 5.37x less memory at equal accuracy. Quantized to 8/16-bit fixed point and mapped to the Loihi 2 neuromorphic chip, the compressed model processes frames 42x faster and uses 149x less energy per token than the equally accurate dense model on an edge GPU. If these trade-offs hold, the constant-memory sequence models behind recent advances in language, audio, and genomics could also run where latency and energy budgets are tightest.","feed_headline":"Sparse fixed-point RNNs run 42x faster and use 149x less energy","feed_subtitle":"The speedups pit a pruned 8/16-bit model against a dense 32-bit floating-point model on an edge GPU.","key_machinery":"The mechanism that carries the argument is the diagonal form of the linear recurrence — $x_k = \\mathrm{diag}(\\bar{A}) \\otimes x_{k-1} + \\bar{B}^{\\mathsf{T}} u_k$ with complex diagonal recurrent weights — which makes the S5 layer friendly to pruning and to fixed-point hardware. Compression runs through three stacked techniques: iterative magnitude pruning with an Erdős–Rényi–Kernel per-layer sparsity allocation that reaches 90% final sparsity on a cubic schedule; ReLU-fication, which replaces GELU with ReLU and inserts extra ReLUs so pre-activations entering linear layers are partly zero; and quantization-aware training with static symmetric scales that converts the model to W8A16 fixed-point arithmetic (8-bit weights, 16-bit activations, 16-bit diagonal state weights). Effective MACs are computed per layer as a product of weight density and activation density, which is why the two sparsity types multiply together. On the neuromorphic chip, matrix-vector products skip zero activations and only non-zero spikes are communicated, so weight and activation sparsity turn directly into fewer operations and fewer messages; complex matrices are split into real and imaginary synaptic layers, while element-wise operations are fused into programmable neurons.","core_discovery":"On its own terms, the paper claims that sparsity is not a tax on accuracy but a way to buy efficiency: S5 linear RNNs trained with iterative magnitude pruning to 90% weight sparsity, with GELU replaced by ReLU and extra ReLUs inserted after residual connections and on the hidden state, form the full Pareto front of test SI-SNR versus effective MACs and memory across a scaling study from 0.5M to 4M parameters. Concretely, sparse-8 matches dense-3 with 2x fewer effective MACs and 36% lower memory, the largest sparse model sustains the accuracy of the previous state of the art with 3.2x less compute and 5.37x less memory, and quantization-aware training to 8-bit weights and 16-bit activations keeps fixed-point models close to floating-point accuracy while post-training quantization does not. Deploying the fixed-point sparse model on the Loihi 2 neuromorphic chip yields 35x lower latency and 1200x lower energy at single-token latency, and 42x lower latency with 149x lower energy per token for the largest chunks that still fit the 8 ms real-time budget, compared with the dense model of equal accuracy running in 32-bit floating point on a Jetson Orin Nano edge GPU. The authors report definite positive answers to whether sparse linear RNNs can be trained and whether the gains translate to hardware, and positive evidence rather than proof for the Pareto-dominance and quantization claims.","pith_inferences":["A cleaner decomposition of the hardware result would isolate sparsity from platform: run a dense W8A16 model on the same edge GPU, since the paper's own Limitations section states its quantized model gives no runtime or energy gain on that GPU, implying the measured gap would shrink.","The paper observes that pruning lowers activation sparsity and that activation sparsity falls with depth, which suggests ReLU-fication saturates as a sparsity lever; approximate top-k activations or per-layer sparsity budgets are a natural next step to extend the Pareto front.","Because the diagonal-recurrence form is shared across audio, language, and genomics state-space models, the same prune-then-quantize pipeline may transfer to those settings, where the constant-memory streaming property is equally valuable, though the paper only demonstrates the audio task.","The Pareto comparison counts effective MACs and memory, not measured energy per operation; on conventional hardware where sparse accesses are not cheaper, the 2x compute advantage would not translate into energy savings, so the strongest form of the claim is tied to event-driven architectures."],"forward_implications":["An audio denoising linear RNN trained to 90% weight sparsity matches a dense model's accuracy at 2x lower compute and 36% lower memory, so the same denoising quality can ship in smaller, cheaper, lower-power edge hardware.","Quantization-aware training to 8-bit weights and 16-bit activations preserves fixed-point accuracy while post-training quantization degrades it, making quantization-aware training the required route to integer-only deployment.","On event-driven neuromorphic hardware, the combination of weight sparsity, activation sparsity, and fixed-point arithmetic delivers 35x to 42x lower latency and 1200x to 149x lower energy per token than an equally accurate dense model on an edge GPU within the real-time streaming budget.","The efficiency-accuracy advantage of sparse models appears across a range of compute budgets and the paper reports a similar trend on a keyword-spotting task, indicating the finding is not an artifact of the denoising benchmark."],"supporting_citations":[{"why":"supplies the S5 architecture, its parameterization, and the codebase that this work prunes, ReLU-fies, and quantizes.","marker":"(Smith et al., 2023)"},{"why":"provides iterative magnitude pruning and the cubic sparsity schedule used to reach 90% weight sparsity.","marker":"(Zhu & Gupta, 2018)"},{"why":"contributes the Erdős–Rényi–Kernel per-layer sparsity allocation and the MAC-counting convention used in the Pareto analysis.","marker":"(Evci et al., 2020)"},{"why":"introduces ReLU-fication, the activation-sparsifying surgery that the paper adapts from transformers to linear RNNs.","marker":"(Mirzadeh et al., 2024)"},{"why":"established quantization-aware training for linear RNNs and the W8A16 fixed-point recipe that the hardware deployment relies on.","marker":"(Abreu et al., 2024)"},{"why":"defines the deep-noise-suppression benchmark, its data, and the 32 ms processing latency budget that drives the hardware comparison.","marker":"(Timcheck et al., 2023)"},{"why":"supplies the theoretical result that wider sparse networks outperform dense ones at fixed parameter count, which the Pareto findings are read as supporting.","marker":"(Golubeva et al., 2021)"},{"why":"is the previous state-of-the-art model at 15.2 dB SI-SNR that the sparse S5 model matches with 3.2x less compute and 5.37x less memory.","marker":"(Hao et al., 2024)"}],"fun_headline_variants":["Sparse linear RNNs hit 42x latency cut, 149x energy drop on edge","Pruned RNNs: 2x less compute, 36% less memory at same accuracy","Sparse RNNs on edge: 42x faster, 149x lower energy than dense","Unstructured sparsity makes linear RNNs 42x faster, 149x greener","Sparse fixed-point RNNs: 42x lower latency, 149x lower energy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline 42x and 149x advantages rest on comparing a sparse 8/16-bit fixed-point model on the neuromorphic chip against a dense 32-bit floating-point model on the edge GPU, and the paper's Limitations section concedes that its quantized model gives no speed or energy gain on that GPU.","fun_headline_variants_meta":{"raw":{"variants":["Sparse linear RNNs hit 42x latency cut, 149x energy drop on edge","Pruned RNNs: 2x less compute, 36% less memory at same accuracy","Sparse RNNs on edge: 42x faster, 149x lower energy than dense","Unstructured sparsity makes linear RNNs 42x faster, 149x greener","Sparse fixed-point RNNs: 42x lower latency, 149x lower energy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000735,"raw_usage":{"total_tokens":3357,"prompt_tokens":1089,"completion_tokens":2268,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":705,"completion_tokens_details":{"reasoning_tokens":2145}},"tokens_in":705,"tokens_out":2268,"duration_ms":12222,"temperature":1.0,"reasoning_tokens":2145,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T15:40:40.500407+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure per-token latency and energy for a dense W8A16 fixed-point version of the equally accurate S5 model on the same edge GPU; if that quantized dense baseline approaches the neuromorphic chip's latency and energy numbers, then the claimed gains come mostly from hardware differences rather than from sparsity.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the S5 architecture, its parameterization, and the codebase that this work prunes, ReLU-fies, and quantizes."},{"cited_title":"and Gupta, S","cited_arxiv_id":null,"evidence_quote":"provides iterative magnitude pruning and the cubic sparsity schedule used to reach 90% weight sparsity."},{"cited_title":"S., and Elsen, E","cited_arxiv_id":null,"evidence_quote":"contributes the Erdős–Rényi–Kernel per-layer sparsity allocation and the MAC-counting convention used in the Pareto analysis."},{"cited_title":"E., Heckel, K","cited_arxiv_id":null,"evidence_quote":"established quantization-aware training for linear RNNs and the W8A16 fixed-point recipe that the hardware deployment relies on."},{"cited_title":"B., Ben Dayan Rubin, D., Kupryjanow, A., Orchard, G., Pindor, L., Shea, T., and Davies, M","cited_arxiv_id":null,"evidence_quote":"defines the deep-noise-suppression benchmark, its data, and the 32 ms processing latency budget that drives the hardware comparison."},{"cited_title":"Are wider nets better given the same number of parameters? October 2021","cited_arxiv_id":null,"evidence_quote":"supplies the theoretical result that wider sparse networks outperform dense ones at fixed parameter count, which the Pareto findings are read as supporting."}],"review_version":1}