Pith. sign in

REVIEW 15 cited by

State Space Model for New-Generation Network Alternative to Transformers: A Survey

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2404.09516 v1 pith:KJVUHNGL submitted 2024-04-15 cs.LG cs.AIcs.CLcs.CVcs.MM

Xiao Wang , Shiao Wang , Yuhe Ding , Yuehang Li , Wentao Wu , Yao Rong , Weizhe Kong , Ju Huang
show 8 more authors
Shihao Li Haoxiang Yang Ziwen Wang Bo Jiang Chenglong Li Yaowei Wang Yonghong Tian Jin Tang
This is my paper · ORCID
classification cs.LGcs.AIcs.CLcs.CVcs.MM
keywords modelgivemodelsspacestatevariousanalysisarchitecture
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

In the post-deep learning era, the Transformer architecture has demonstrated its powerful performance across pre-trained big models and various downstream tasks. However, the enormous computational demands of this architecture have deterred many researchers. To further reduce the complexity of attention models, numerous efforts have been made to design more efficient methods. Among them, the State Space Model (SSM), as a possible replacement for the self-attention based Transformer model, has drawn more and more attention in recent years. In this paper, we give the first comprehensive review of these works and also provide experimental comparisons and analysis to better demonstrate the features and advantages of SSM. Specifically, we first give a detailed description of principles to help the readers quickly capture the key ideas of SSM. After that, we dive into the reviews of existing SSMs and their various applications, including natural language processing, computer vision, graph, multi-modal and multi-media, point cloud/event stream, time series data, and other domains. In addition, we give statistical comparisons and analysis of these models and hope it helps the readers to understand the effectiveness of different structures on various tasks. Then, we propose possible research points in this direction to better promote the development of the theoretical model and application of SSM. More related works will be continuously updated on the following GitHub: https://github.com/Event-AHU/Mamba_State_Space_Model_Paper_List.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 15 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Geometric Hyena Networks for Large-scale Equivariant Learning

    cs.LG 2025-05 conditional novelty 8.0 of 10

    Geometric Hyena is an equivariant long-convolutional architecture that captures global geometric context with sub-quadratic complexity and outperforms equivariant transformer baselines on several RNA and protein predi...

  2. OGG-FR: Orthogonal Gradient Gaming and Frequency Rectification for Unmanned Aerial Vehicle Infrared Image Super-Resolution

    cs.CV 2026-08 conditional novelty 6.0 of 10

    OGG-FR is a plug-and-play training update that separates redundant and innovative parts of the FFT loss gradient and gates the innovative part by a confidence score, improving UAV infrared super-resolution in most tes...

  3. Cortical-SSM: A Deep State Space Model for Motor Imagery Decoding from EEG Signals

    cs.CV 2025-10 conditional novelty 6.0 of 10

    Cortical-SSM, a dual state-space architecture with wavelet-based frequency features, reports state-of-the-art motor-imagery decoding accuracy on OpenBMI, Stieger2021, and a clinical ECoG-ALS dataset.

  4. A Deep State-Space Model Compression Method using Upper Bound on Output Error

    eess.SY 2025-10 conditional novelty 6.0 of 10

    A new upper bound ties end-to-end deep state-space model compression error to layerwise H2 approximation errors, enabling a gradient-based method that cuts ~80% of parameters with a modest accuracy drop.

  5. GPSMamba: A Global Phase and Spectral Prompt-guided Mamba for Infrared Image Super-Resolution

    cs.CV 2025-07 conditional novelty 6.0 of 10

    A Mamba-based model with an injected frequency-domain prompt and a phase-spectrum loss improves infrared image super-resolution by roughly 0.1 dB PSNR over earlier state-of-the-art results.

  6. Block-Biased Mamba for Long-Range Sequence Processing

    cs.LG 2025-05 conditional novelty 6.0 of 10

    B2S6, a block-biased variant of Mamba's S6 unit, restores universal approximation, softens input-dependent forgetting, and beats S4/S4D on Long-Range Arena while keeping Mamba-like language perplexity.

  7. Mixture of Sparse Attention: Content-Based Learnable Sparse Attention via Expert-Choice Routing

    cs.LG 2025-05 conditional novelty 6.0 of 10

    MoSA, an expert-choice style sparse attention that selects per-head top-k tokens, outperforms dense transformers on C4 language modeling under matched FLOPs with up to 27% perplexity improvement and reduces wall-clock...

  8. GraphMinNet: Learning Dependencies in Graphs with Light Complexity Minimal Architecture

    cs.LG 2025-02 reject novelty 6.0 of 10

    A gated recurrent graph architecture that applies minGRU-style gating to node embeddings with Laplacian positional encoding, reporting state-of-the-art on several benchmarks and claiming linear-time long-range depende...

  9. EDMB: Edge Detector with Mamba

    cs.CV 2025-01 conditional novelty 6.0 of 10

    A Mamba-based edge detector achieves SOTA results on BSDS500 and produces multi-granularity edges on single-label datasets using an ELBO-supervised Gaussian decoder.

  10. Activating Associative Disease-Aware Vision Token Memory for LLM-Based X-ray Report Generation

    eess.IV 2025-01 conditional novelty 6.0 of 10

    AM-MRG combines disease-region extraction with two Hopfield memory retrievers to improve LLM-generated chest X-ray reports on IU X-ray, MIMIC-CXR, and Chexpert Plus.

  11. TrackingMiM: Efficient Mamba-in-Mamba Serialization for Real-time UAV Object Tracking

    cs.CV 2025-07 conditional novelty 5.0 of 10

    A nested Mamba-in-Mamba architecture with template-first spatial scans, temporal serialization scans, and retrieval-augmented query attention reports top average precision and success on five UAV tracking benchmarks.

  12. Mamba-FETrack V2: Revisiting State Space Model for Frame-Event based Visual Object Tracking

    cs.CV 2025-06 conditional novelty 5.0 of 10

    Mamba-FETrack V2 fuses RGB and event streams inside a Vision Mamba backbone, achieving 53.8% success rate on FELT V2 with 30M parameters and 29 FPS.

  13. FMaMIL: Frequency-Driven Mamba Multi-Instance Learning for Weakly Supervised Lesion Segmentation in Medical Images

    cs.CV 2025-06 conditional novelty 4.0 of 10

    FMaMIL combines Mamba-based multiple instance learning with learnable frequency-domain encoding and CAM-guided pseudo-label refinement to segment lesions from image-level labels only.

  14. A Study on the Performance of U-Net Modifications in Retroperitoneal Tumor Segmentation

    eess.IV 2025-02 conditional novelty 3.0 of 10

    ViLU-Net, a U-Net built with Vision LSTM blocks, reports the best segmentation scores on a new retroperitoneal tumor CT dataset and on FLARE22, but the comparison lacks error bars and omits the closest prior architecture.

  15. Exploring Graph Mamba: A Comprehensive Survey on State-Space Models for Graph Learning

    cs.LG 2024-12 conditional novelty 2.0 of 10

    A survey of Graph Mamba, the adaptation of state-space models (Mamba, S4, S6) to graph learning, synthesizing roughly 30 recent papers into a taxonomy of architectures, applications, benchmarks, and open challenges.

Pith tools