Pith. sign in

REVIEW 2 major objections 5 minor 1 cited by

Let SSMs be ConvNets: State-space Modeling with Optimal Tensor Contractions

T0 review · 2 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SSM layers are tensor contractions; choosing each block's contraction order unlocks CNN-style connector blocks and a hybrid network that beats uniform SSM stacks on keyword spotting, denoising, and streaming ASR.

desk verdict Solid tensor-contraction framework for SSMs, but the 'first fully state-space ASR' claim is contradicted by the paper's own Table 2. read the letter →

arxiv 2501.13230 v2 pith:WPWSVQJS submitted 2025-01-22 cs.LG cs.AI

Yan Ru Pei This is my paper · ORCID
classification cs.LGcs.AI
keywords state-spacemodelstensornetworkseinsumcontractionorderFFTconvolutionheterogeneousnetworkdesignkeywordspottingspeechenhancementautomaticrecognition
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Centaurus is the claim that state-space model (SSM) layers can be designed and trained the way convolutional networks are: each layer is a tensor contraction whose connective pattern — depthwise, grouped, full, or bottleneck — is a free architectural choice, and whose contraction order is chosen per layer for efficiency. In the frequency domain an SSM layer's long convolution becomes an einsum expression, so training cost is whatever the cheapest valid contraction path costs; the paper proves which paths stay feasible for the bottleneck block and derives the exact dimension condition for choosing between the two efficient ones. The payoff is heterogeneity: dense full SSM blocks in shallow layers, sparse bottleneck blocks where channels are large, following classical CNN scaling rules. The paper reports that this hybrid beats every homogeneous SSM variant on Speech Commands keyword spotting, achieves competitive PESQ scores on speech denoising at a fraction of the size of heavier real-time systems, and reaches competitive word error rates on LibriSpeech streaming ASR with a fully state-space stack that uses no nonlinear recurrence, explicit convolution, or attention. If the paper is right, SSM network design gains the per-layer freedom that made CNNs flexible, at a fraction of the usual inference cost.

What carries the argument

The carrying object is the frequency-domain einsum expression of an SSM block, treated as a tensor network in which the DFT operator is itself a contractable operand; this makes FFT placement part of the contraction-path optimization, justified by the linearity identity $F(E\cdot K)=E\cdot F(K)$. Lemma 1 does the heavy lifting: for the bottleneck block $\hat{y}_{bif}=\hat{u}_{bif}B_{ni}\hat{k}_{nf}C_{jn}$, all intermediate tensors stay at most three-dimensional if and only if the input is contracted only with its neighbours, which leaves exactly two feasible contraction patterns whose relative cost is settled by the inequality $\frac{1}{B}+\frac{1}{N} > \frac{1}{H}+\frac{1}{H'}$, with an additional shape check deciding whether to Fourier-transform the projected input or the built kernel. The block family — depthwise, depthwise-separable, grouped, full, and (pointwise) bottleneck — is read directly off the tensor-network index pattern, mirroring classical CNN blocks, and the hybrid architecture orders them by the classical rule of dense connectivity in shallow layers and sparser connectivity deeper.

What would settle it

Run the Table 5 and Table 1 comparisons at the same configurations using standard released implementations of the S4, S4D, S5, and Mamba baselines and a general-purpose FLOP profiler instead of the paper's estimation rules: if a canonical depthwise-separable network matches the hybrid's accuracy on Speech Commands at comparable measured cost, the efficiency claim fails. A complementary check: train the homogeneous depthwise denoising variant with the hybrid's exact width schedule, optimizer, and data pipeline — if it escapes the reported plateau and approaches PESQ 3.12, the claim that heterogeneity is what makes the hybrid train would be falsified.

Watch

Extended reading notes

Core claim

The central claim is that an SSM layer, written in the frequency domain, is a tensor network — an einsum expression such as $\hat{y}_{bif} = \hat{u}_{bif}B_{ni}\hat{K}_{nmf}E_{nm}C_{jn}$, where $\hat{u}$ is the input, $\hat{K}_{nmf}$ are basis kernels generated by the state matrix $A$, $E_{nm}$ weights the oscillation modes, and $B_{ni}$ and $C_{jn}$ project channels — so its computational cost is determined by the order in which the tensors are contracted. Standard depthwise-separable SSM blocks (one state evolution per input channel, as in the S4 and Mamba families) are a single point of this design space, and group, full, and bottleneck connectivities are natural alternatives read off the index pattern. The technical core is that because the Fourier transform is linear, $F(E\cdot K)=E\cdot F(K)$, the FFT itself can be moved around like an operand; Lemma 1 then restricts the bottleneck block to exactly two feasible contraction patterns — project-convolve-project, or build the full kernel and convolve once — when no intermediate tensor may exceed three dimensions, with the cheaper pattern selected by the inequality $\frac{1}{B}+\frac{1}{N} > \frac{1}{H}+\frac{1}{H'}$. Using these blocks, the paper builds the hybrid Centaurus network and reports that it outperforms its homogeneous counterparts on keyword spotting, speech denoising, and end-to-end streaming ASR, claiming it as the first competitive ASR network that is fully state-space based, with no nonlinear recurrence, no explicit convolution, and no (surrogate) attention.

Load-bearing premise

The load-bearing premise is that the efficiency comparisons are fair: the baseline SSMs are counted with the paper's own estimation rules rather than measured, and some homogeneous variants were effectively tuned out of contention by training plateaus, so inflated baseline costs would shrink the reported margins.

Editorial extensions

If this is right

  • The depthwise-separable template is not intrinsic to SSMs: group, full, and bottleneck connectivities are direct generalizations, so classical CNN design rules (dense early, sparse deep) transfer to state-space architectures.
  • Training any SSM block type, including existing S4D/S5-style ones, can be accelerated by systematically choosing contraction order and FFT placement; the bottleneck block's training time is shown to scale smoothly with batch, state count, and sequence length under the optimized path.
  • On Speech Commands the hybrid reaches 98.53% accuracy on the 10-class set with roughly a hundred times fewer inference FLOPs than the reported S4/S4D/S5 baselines, and its scaling curves beat every homogeneous variant on the 35-class set.
  • On VB-DMD speech enhancement the hybrid scores PESQ 3.12 with 0.51M parameters and 0.29 GFLOPs per second, rising to 3.25 when a small causal Conv1d is prepended to each block — above every listed real-time baseline — while the homogeneous depthwise and bottleneck variants plateau and fail to train.
  • On LibriSpeech streaming ASR the fully state-space base reaches 6.0/13.1 test WER, improving to 4.4/10.2 with a Mamba-style gated macro-block, staying competitive with streaming ASR systems built with convolutions or attention while using no attention and no explicit convolution.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the contraction machinery is as general as the paper argues, the same block family and path-selection rule should transfer to 2D SSMs, strided or dilated variants, and vision tasks — an extension the paper names but does not test, which would turn the block family into a general-purpose sequence and image toolkit.
  • The failure of homogeneous depthwise and bottleneck blocks to train on raw-waveform denoising suggests per-channel independent state evolution is a poor inductive bias for that task and that heterogeneity acts as structured capacity allocation; this is testable by training the depthwise variant with the hybrid's channel schedule and optimization recipe.
  • Combining Centaurus blocks with data-gating (input-dependent $A$, $B$, $C$, as in selective-scan models) is the paper's stated open direction; if the contraction machinery can absorb the extra sequence dimension that data-dependence introduces, the training-speed and expressivity gains would compound and could plausibly extend the approach to language modeling.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper introduces Centaurus, a class of state-space models in which SSM operations are expressed as tensor contractions in the frequency domain and the contraction order is optimized per block. It generalizes the connective structure of SSM blocks from the usual depthwise-separable form to full, grouped, and bottleneck configurations inspired by classical CNN designs. The authors report experiments on keyword spotting (SC35/SC10), speech enhancement (VB-DMD), and LibriSpeech ASR, claiming that a heterogeneous Centaurus network outperforms homogeneous SSM baselines and that, for ASR, it is the first network with competitive performance that is fully state-space based without explicit convolutions or attention.

Significance. The tensor-contraction formulation of SSM blocks and the formal derivation of feasible contraction orders (Lemma 1 and Appendix B) are clean, self-contained, and potentially useful for practitioners seeking to speed up SSM training. The hybrid block design is a reasonable exploration of architectural diversity beyond the standard depthwise-separable SSM. However, the paper's headline empirical claim for ASR is not supported by its own Table 2, and the efficiency comparisons against baseline SSMs rely on author-estimated FLOPs and possibly undertuned baselines. If the contraction-order analysis is correct and the block designs are adopted, the work could have practical value, but the current presentation overstates the empirical results.

major comments (2)
  1. [Abstract and Section 5.3, Table 2] The abstract's claim that Centaurus is "the first network with competitive performance that can be made fully state-space based, without using any nonlinear recurrence (LSTMs), explicit convolutions (CNNs), or (surrogate) attention mechanism" is contradicted by the data in Table 2. The base (full SSM) Centaurus achieves 6.0/13.1 WER on test/dev, while the streaming Conformer baseline is 4.6/9.9 and ContextNet is 4.5/10.0. The two variants that approach or beat these baselines, "with causal conv-block" (4.8/10.6) and "with Mamba macro-block" (4.4/10.2), both include a causal depthwise Conv1D layer, as described in Appendix E.3 and Appendix D, so they violate the "without explicit convolutions" condition. The only variant without an explicit convolution, "with FFN" (5.4/11.5), is roughly 17% relative worse than streaming Conformer on test-clean and still worse than the other streaming baselines. The claim therefore requires either a competitive configuration that is truly free of convolution and attention, or a quantitative definition of "competitive" that makes the FFN variant acceptable.
  2. [Appendix D and Section 5.2 (Tables 1 and 5)] The efficiency comparison against baseline SSMs is not on equal footing. The FLOPs and parameter counts for S4, S4D, S5, S6, and Mamba are estimated using the authors' own rules described in Appendix D, and the entries in Table 5 are marked with asterisks, while the Centaurus numbers are computed for the actual architecture. In Section 5.2, the homogeneous DWS and bottleneck variants "suffered from severe training plateaus" and produced no usable audio, and the paper reports unstable training for Mamba-based variants in the ASR experiments. If the baselines are undertuned or the FLOP estimates are inconsistent with standard implementations, the "orders of magnitude fewer FLOPs" claim (Section 5.1) and the PESQ/FLOP advantage in Table 1 would not hold. Please provide measured training/inference times or standard-library FLOP counts for the baselines, or state clearly the assumptions and justify them.
minor comments (5)
  1. [Listing 1 in Appendix C] In the second branch of opt_fft_conv (lines 38-39), the variable `x` is used before it is defined; it should be `u`.
  2. [Section 4.3] The inequality `1/B + 1/N > 1/H + 1/H'` is stated without an equation number; please number it and explicitly define B as the batch size.
  3. [Table 2] Several offline rows (Full Conv, Transformer, Wav2Vec2) do not report parameter counts or FLOPs; please fill these entries or state why they are omitted.
  4. [Figure 1] The tensor network diagrams are difficult to read at the printed size; consider increasing the font size and edge labeling.
  5. [Section 5.2] The word "Surprisingly" is subjective; consider rephrasing to a neutral statement about the observed training plateaus.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the contraction-order analysis is a parameter-free cost argument and the empirical claims are real experiments; remaining self-citations are not load-bearing.

full rationale

The paper's central derivation is self-contained: the tensor-network/einsum formulation of SSM blocks (Section 4.1) is a notational restatement of the convolution/impulse-response structure, and Lemma 1 (Appendix B) is a shape-based proof that intermediates stay 3-dimensional only if the input contracts with neighboring operands. The optimality condition in Section 4.3 follows from explicit FLOP inequalities (BNF(H+H') vs HH'F(B+N)) and is benchmarked in Appendix C, so it is not fitted to the experimental results. The heterogeneous-vs-homogeneous and ASR comparisons are actual trained-model evaluations, not predictions derived from fitted parameters. The FLOP estimates for baseline SSMs in Appendix D are stated counting conventions rather than values fitted to outcomes; their fairness is a correctness risk, not a circularity. The paper cites two prior works by the same author (Pei and Coenen 2024; Pei et al. 2024) for design inspiration and an hourglass backbone, but the hybrid-superiority and efficiency claims are assessed with this paper's own runs and independent benchmark numbers, so the self-citations are not load-bearing. One evidentiary weakness should be noted separately: the abstract's 'first fully state-space ASR' claim is not supported by Table 2, since the best WERs come from variants with a causal conv-block or Mamba macro-block, and Appendix D states the Mamba block contains 'a lightweight causal depthwise Conv1D layer'; the base full-SSM row (6.0/13.1) is not competitive with streaming Conformer (4.6/9.9). This is an internal-consistency/correctness concern, not a circular reduction, because the claim is not obtained by defining the target in terms of the input. Therefore no circular step is identified.

Assumptions & free parameters 4 free parameters · 5 assumptions · 3 invented entities

The central claims rest on standard linear algebra and FFT properties, plus a few hand-chosen architectural and estimation choices. The new block variants are empirically evaluated, so they are not unsupported inventions, but the manual layout choices and FLOPs assumptions introduce potential bias that the reader should keep in mind.

free parameters (4)
  • Hybrid block layout = full×2 + bottleneck×2 + S5×2 (KWS); similar for ASR/denoising
    The central empirical claim that hybrid networks outperform homogeneous ones is demonstrated using this manually chosen sequence of block types, not a systematic search. Different layouts could change the conclusion.
  • Number of sub-states M = 4
    The bottleneck/neck blocks use M=4 sub-states per state block throughout; performance may depend on this value.
  • State dimensions per layer = e.g., [4,4,16,32,64,128] in KWS hybrid
    Hand-chosen per layer to balance size and efficiency; not optimized.
  • FLOPs estimation constants = e.g., expf≈4 FLOPs, complex mult≈6 FLOPs
    These assumptions in Appendix D affect the comparison of inference FLOPs, which is central to the efficiency claims. If the true constants differ, the reported speedups over baselines change.
assumptions (5)
  • standard math The SSM impulse response can be computed as k[τ]=C A^τ B and training can be done via FFT-based convolution.
    Used throughout Sections 3-4; relies on linearity and the convolution theorem.
  • domain assumption A is complex diagonal, B and C are real projection matrices.
    Stated in Section 3.1 and justified in Appendix A as a restriction for memory and compute efficiency; deviates from prior S4D and S5 where B and C are complex.
  • standard math ZOH discretization (Eq. 2) is used.
    Standard discretization for SSMs.
  • ad hoc to paper The generalized basis-kernel system (Eq. 7) with K_nm and E_nm is a valid SSM generalization.
    Introduced in Section 3.3 as a more expressive variant; its recurrent form (Eq. 8) is a new construct not found in prior work.
  • standard math Padding to 2L in FFT yields correct linear convolution.
    Used in Listing 1; standard property of FFT-based convolution.
invented entities (3)
  • Full SSM block independent evidence
    purpose: Provides full input-output channel connectivity in an SSM layer, analogous to a full convolution.
    Defined in Section 4.1 and evaluated in KWS (Fig. 3), denoising (Table 1), and ASR (Table 2). The benchmark results on public datasets provide a falsifiable handle.
  • Bottleneck SSM block (with E_nm expansion) independent evidence
    purpose: Allows bottleneck-style connectivity with multiple sub-states per channel group.
    Defined in Sections 3.3 and 4.1 and used in the hybrid networks. Its performance is measured on SC35, VB-DMD, and Librispeech.
  • Grouped SSM block independent evidence
    purpose: Channel-grouped connectivity inspired by group convolution.
    Introduced in Figure 1 and mentioned in the experiments; the framework predicts it can be trained and used, and the paper's hyper-optimized contraction logic applies to it.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Let SSMs be ConvNets: State-space Modeling with Optimal Tensor Contractions." pith.science (2026). https://pith.science/paper/WPWSVQJS

@misc{pith2026250113230,
  author       = {Pith},
  title        = {Pith review of: Let SSMs be ConvNets: State-space Modeling with Optimal Tensor Contractions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WPWSVQJS}},
  note         = {Machine review of arXiv:2501.13230}
}
read the original abstract

We introduce Centaurus, a class of networks composed of generalized state-space model (SSM) blocks, where the SSM operations can be treated as tensor contractions during training. The optimal order of tensor contractions can then be systematically determined for every SSM block to maximize training efficiency. This allows more flexibility in designing SSM blocks beyond the depthwise-separable configuration commonly implemented. The new design choices will take inspiration from classical convolutional blocks including group convolutions, full convolutions, and bottleneck blocks. We architect the Centaurus network with a mixture of these blocks, to balance between network size and performance, as well as memory and computational efficiency during both training and inference. We show that this heterogeneous network design outperforms its homogeneous counterparts in raw audio processing tasks including keyword spotting, speech denoising, and automatic speech recognition (ASR). For ASR, Centaurus is the first network with competitive performance that can be made fully state-space based, without using any nonlinear recurrence (LSTMs), explicit convolutions (CNNs), or (surrogate) attention mechanism. The source code is available as supplementary material on https://openreview.net/forum?id=PkpNRmBZ32

Figures

Figures reproduced from arXiv: 2501.13230 by the authors.

Figure 1
Figure 1. A tensor network representation of the tensorial connection structure of the differ [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. The two feasible patterns for performing the SSM bottleneck operations. a) Follow [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. The scaling of the classification error versus the number of parameters and FLOPs per [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The training time (forward plus backward pass time) scaling with respect to the batch, [PITH_FULL_IMAGE:figures/full_fig_p017_4.png]
Figure 5
Figure 5. Figure 5: We use a basic design for our Centaurus block, where the lighter-shaded blocks are [PITH_FULL_IMAGE:figures/full_fig_p020_5.png]
Figure 6
Figure 6. Figure 6: The scaling of the multiclass classification error with respect to the number of parameters [PITH_FULL_IMAGE:figures/full_fig_p022_6.png]
Figure 7
Figure 7. Figure 7: We compare the scaling of the hybrid network compared to homogeneous network variants [PITH_FULL_IMAGE:figures/full_fig_p022_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Falcon-H1: A Family of Hybrid-Head Language Models Redefining Efficiency and Performance

    cs.CL 2025-07 conditional novelty 6.0 of 10

    Falcon-H1 reports competitive benchmark scores for a 0.5B to 34B family of parallel hybrid attention/Mamba-2 models, claiming 2x to 4x parameter efficiency versus dense transformers.

Reference graph

Works this paper leans on

41 extracted references · 20 canonical work pages · cited by 1 Pith paper

  1. [2]

    insertion points

    identify the optimal “insertion points” for FFT and iFFT operations accounting for the preference of real tensors over complex ones. We compare the training performance of the contraction-optimized SSM block as given in Listing 1 against the naive contraction order following the “natural” order of input projection, state evolution, and output projection (...

    work page

  2. [4]

    • For the full SSM block, we initialize ∆in over the i dimension, and Ajin over the n dimen- sion

    ∆n is initialized across the groups, and An is initialized within each group. • For the full SSM block, we initialize ∆in over the i dimension, and Ajin over the n dimen- sion. • For the bottleneck block, we initialize ∆n over the n dimension, and Anm over the m dimension. All of our training runs and trials are done with PyTorch with torch.compile enable...

    work page 2025

  3. [6]

    It’s raw! audio generation with state- space models

    11 Published as a conference paper at ICLR 2025 Karan Goel, Albert Gu, Chris Donahue, and Christopher R ´e. It’s raw! audio generation with state- space models. In International Conference on Machine Learning , pp. 7616–7633. PMLR,

    work page 2025

  4. [8]

    language head

    The character head is supervised with the soft logits from the WAV2VEC2 ASR LARGE LV60K 960H teacher model, and the language head is supervised via CTC loss with the tokenized transcripts. During inference, we simply extract the most probable token produced by the final layer of the “language head”. 13Note that nowhere in our solution will we actually use...

    work page 2025

  5. [9]

    Conformer: Convolution-augmented transformer for speech recognition

    Anmol Gulati, James Qin, Chung-Cheng Chiu, Niki Parmar, Yu Zhang, Jiahui Yu, Wei Han, Shibo Wang, Zhengdong Zhang, Yonghui Wu, et al. Conformer: Convolution-augmented transformer for speech recognition. arXiv preprint arXiv:2005.08100,

    arXiv 2005

  6. [11]

    Sequence-to-sequence speech recog- nition with time-depth separable convolutions

    Awni Hannun, Ann Lee, Qiantong Xu, and Ronan Collobert. Sequence-to-sequence speech recog- nition with time-depth separable convolutions. arXiv preprint arXiv:1904.02619,

    arXiv 1904

  7. [16]

    bottleneck

    The generation of the B and C matrices requires a total of 2HN parame- ters and 4HN FLOPs. • The discretization process has to occur dynamically due to the dynamicity of the ∆ pa- rameter. It does not incur additional parameters but does require additional FLOPs. This involves applying softplus to ∆, performing an element-wise multiplication with the inpu...

    work page 2025

  8. [17]

    Jamba: A hybrid transformer- mamba language model

    12 Published as a conference paper at ICLR 2025 Opher Lieber, Barak Lenz, Hofit Bata, Gal Cohen, Jhonathan Osin, Itay Dalmedigos, Erez Safahi, Shaked Meirom, Yonatan Belinkov, Shai Shalev-Shwartz, et al. Jamba: A hybrid transformer- mamba language model. arXiv preprint arXiv:2403.19887,

    arXiv 2025

Show all 41 references
  1. [18]

    Structured state space decoder for speech recognition and synthesis

    Koichi Miyazaki, Masato Murata, and Tomoki Koriyama. Structured state space decoder for speech recognition and synthesis. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pp. 1–5. IEEE,

    2023

  2. [19]

    Simba: Simplified mamba-based architecture for vision and multivariate time series

    Badri N Patro and Vijay S Agneeswaran. Simba: Simplified mamba-based architecture for vision and multivariate time series. arXiv preprint arXiv:2403.15360,

    arXiv

  3. [20]

    Building temporal kernels with orthogonal polynomials

    Yan Ru Pei and Olivier Coenen. Building temporal kernels with orthogonal polynomials. arXiv preprint arXiv:2405.12179,

  4. [21]

    Raw speech enhancement with deep state space modeling

    Yan Ru Pei, Ritik Shrivastava, and FNU Sidharth. Raw speech enhancement with deep state space modeling. arXiv preprint arXiv:2409.03377,

    arXiv

  5. [22]

    Samba: Sim- ple hybrid state space models for efficient unlimited context language modeling

    Liliang Ren, Yang Liu, Yadong Lu, Yelong Shen, Chen Liang, and Weizhu Chen. Samba: Sim- ple hybrid state space models for efficient unlimited context language modeling. arXiv preprint arXiv:2406.07522,

    arXiv

  6. [23]

    Deepfilternet: Perceptually motivated real-time speech enhancement

    Hendrik Schr¨oter, Tobias Rosenkranz, Andreas Maier, et al. Deepfilternet: Perceptually motivated real-time speech enhancement. arXiv preprint arXiv:2305.08227,

    arXiv

  7. [24]

    Augmenting conformers with structured state space models for online speech recognition

    Haozhe Shan, Albert Gu, Zhong Meng, Weiran Wang, Krzysztof Choromanski, and Tara Sainath. Augmenting conformers with structured state space models for online speech recognition. arXiv preprint arXiv:2309.08551,

    arXiv

  8. [25]

    Simplified state space layers for sequence modeling

    Jimmy TH Smith, Andrew Warrington, and Scott W Linderman. Simplified state space layers for sequence modeling. arXiv preprint arXiv:2208.04933,

    arXiv

  9. [26]

    A perceptually-motivated approach for low-complexity, real-time enhancement of fullband speech

    Jean-Marc Valin, Umut Isik, Neerad Phansalkar, Ritwik Giri, Karim Helwani, and Arvindh Kr- ishnaswamy. A perceptually-motivated approach for low-complexity, real-time enhancement of fullband speech. arXiv preprint arXiv:2008.04259,

    2008 arXiv

  10. [27]

    Speech commands: A dataset for limited-vocabulary speech recognition

    Pete Warden. Speech commands: A dataset for limited-vocabulary speech recognition. arXiv preprint arXiv:1804.03209,

    arXiv

  11. [28]

    Fully convolutional speech recognition

    13 Published as a conference paper at ICLR 2025 Neil Zeghidour, Qiantong Xu, Vitaliy Liptchinsky, Nicolas Usunier, Gabriel Synnaeve, and Ronan Collobert. Fully convolutional speech recognition. arXiv preprint arXiv:1812.06864,

    2025 arXiv

  12. [29]

    Transformer transducer: A streamable speech recognition model with transformer en- coders and rnn-t loss

    Qian Zhang, Han Lu, Hasim Sak, Anshuman Tripathi, Erik McDermott, Stephen Koo, and Shankar Kumar. Transformer transducer: A streamable speech recognition model with transformer en- coders and rnn-t loss. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech an...

    2020

  13. [30]

    Mamba in speech: Towards an alternative to self- attention

    Xiangyu Zhang, Qiquan Zhang, Hexin Liu, Tianyi Xiao, Xinyuan Qian, Beena Ahmed, Eliathamby Ambikairajah, Haizhou Li, and Julien Epps. Mamba in speech: Towards an alternative to self- attention. arXiv preprint arXiv:2405.12609,

    arXiv

  14. [31]

    Frcrn: Boosting feature representation using frequency recurrence for monaural speech enhancement

    Shengkui Zhao, Bin Ma, Karn N Watcharasupat, and Woon-Seng Gan. Frcrn: Boosting feature representation using frequency recurrence for monaural speech enhancement. In ICASSP 2022- 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pp. 9281–...

    2022

  15. [32]

    memoryless

    A C ANONICAL FORM OF A STATE -SPACE MODEL It is a generic property (though not always true) that a diagonal form exists for the state-space model, meaning that we can almost always assume A to be diagonal, at the expense of potentially requiring B and C to be complex matrices....

    2025

  16. [37]

    Over the model sizes tested, it appears that real variants outperform the complex variants when accounting for the model parameters and inference FLOPs. Furthermore, we compare against network variants with S6 layers having different state dimensions, as the num- ber of states...

    2023

  17. [38]

    The number of sub-states for the “neck” SSM block is always

    - 0.280M 13.5G* Centaurus (hybrid) 98.53 0.378M 0.134G 22 Published as a conference paper at ICLR 2025 Table 6: Resampling factor and output channel of each block of the network, which consists of an encoder performing down-samplings, an intermediate bottleneck, a decoder perf...

    2025

  18. [39]

    no weight decay

    Layers Resampling Factor Channels States Encoder Block 1 (full) 4 16 16 Block 2 (full) 4 32 4 Block 3 (neck) 2 64 128 × 4 Block 4 (neck) 2 96 128 × 4 Block 5 (pw-neck) 2 128 256 Block 6 (pw-neck) 2 256 256 Bottleneck Block 1 (pw-neck) 1 256 256 Block 2 (pw-neck) 1 256 256 Deco...

    2025

  19. [41]

    language head

    • Mamba block: This is a complex residual gated block containing three nested skip con- nections, adapted from the Mamba macro-architecture (Gu & Dao, 2023). It begins by projecting the input channels into 4 times the original dimension, and splitting the channel features into...

    2023

  20. [1922]

    Hungry hungry hippos: Towards language modeling with state space models

    Daniel Y Fu, Tri Dao, Khaled K Saab, Armin W Thomas, Atri Rudra, and Christopher R ´e. Hungry hungry hippos: Towards language modeling with state space models. arXiv preprint arXiv:2212.14052,

    arXiv

  21. [1994]

    Contextnet: Improving convolutional neural networks for automatic speech recognition with global context

    Wei Han, Zhengdong Zhang, Yu Zhang, Jiahui Yu, Chung-Cheng Chiu, James Qin, Anmol Gulati, Ruoming Pang, and Yonghui Wu. Contextnet: Improving convolutional neural networks for automatic speech recognition with global context. arXiv preprint arXiv:2005.03191,

    2005 arXiv

  22. [2009]

    Quartznet: Deep automatic speech recog- nition with 1d time-channel separable convolutions

    Samuel Kriman, Stanislav Beliaev, Boris Ginsburg, Jocelyn Huang, Oleksii Kuchaiev, Vitaly Lavrukhin, Ryan Leary, Jason Li, and Yang Zhang. Quartznet: Deep automatic speech recog- nition with 1d time-channel separable convolutions. In ICASSP 2020-2020 IEEE International Confere...

    2020

  23. [2015]

    An investigation of incorporating mamba for speech enhance- ment

    Rong Chao, Wen-Huang Cheng, Moreno La Quatra, Sabato Marco Siniscalchi, Chao-Han Huck Yang, Szu-Wei Fu, and Yu Tsao. An investigation of incorporating mamba for speech enhance- ment. arXiv preprint arXiv:2405.06573,

  24. [2016]

    Parallelization of an ubiquitous sequential computation

    Franz A Heinsen. Parallelization of an ubiquitous sequential computation. arXiv preprint arXiv:2311.06281,

    arXiv

  25. [2017]

    Gateloop: Fully data-controlled linear recurrence for sequence modeling

    Tobias Katsch. Gateloop: Fully data-controlled linear recurrence for sequence modeling. arXiv preprint arXiv:2311.01927,

    arXiv

  26. [2018]

    Transformers are ssms: Generalized models and efficient algorithms through structured state space duality

    Tri Dao and Albert Gu. Transformers are ssms: Generalized models and efficient algorithms through structured state space duality. arXiv preprint arXiv:2405.21060,

    arXiv

  27. [2019]

    Liquid structural state-space models

    Ramin Hasani, Mathias Lechner, Tsun-Hsuan Wang, Makram Chahine, Alexander Amini, and Daniela Rus. Liquid structural state-space models. arXiv preprint arXiv:2209.12951,

    arXiv

  28. [2020]

    Efficiently modeling long sequences with structured state spaces

    Albert Gu, Karan Goel, and Christopher R ´e. Efficiently modeling long sequences with structured state spaces. arXiv preprint arXiv:2111.00396,

    arXiv

  29. [2021]

    Mamba: Linear-time sequence modeling with selective state spaces

    Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752,

    arXiv

  30. [2022]

    Zamba: A compact 7b ssm hybrid model

    Paolo Glorioso, Quentin Anthony, Yury Tokpanov, James Whittington, Jonathan Pilault, Adam Ibrahim, and Beren Millidge. Zamba: A compact 7b ssm hybrid model. arXiv preprint arXiv:2405.16712,

    arXiv

  31. [2023]

    Mobilenets: Efficient convolutional neural networks for mobile vision applications

    Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861,

    arXiv

  32. [2024]

    Real time speech enhancement in the wave- form domain

    Alexandre Defossez, Gabriel Synnaeve, and Yossi Adi. Real time speech enhancement in the wave- form domain. arXiv preprint arXiv:2006.12847,

    2006 arXiv

  33. [2048]

    contracted away

    Figure 4: The training time (forward plus backward pass time) scaling with respect to the batch, state, and length dimensions. Note that the x-axis is in log scale. 17 Published as a conference paper at ICLR 2025 D E STIMATION OF PARAMETERS AND FLOP S It is very tempting to tr...

    2025

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.