Pith. sign in

REVIEW 17 cited by

Mamba-360: Survey of State Space Models as Transformer Alternative for Long Sequence Modelling: Methods, Applications, and Challenges

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2404.16112 v1 pith:KNUREQJP submitted 2024-04-24 cs.LG cs.AIcs.CVcs.MMeess.IV

Badri Narayana Patro , Vijay Srinivas Agneeswaran This is my paper · ORCID
classification cs.LGcs.AIcs.CVcs.MMeess.IV
keywords longsequencemodelingssmsstatearchitecturesdatasetsincluding
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
abstract

Sequence modeling is a crucial area across various domains, including Natural Language Processing (NLP), speech recognition, time series forecasting, music generation, and bioinformatics. Recurrent Neural Networks (RNNs) and Long Short Term Memory Networks (LSTMs) have historically dominated sequence modeling tasks like Machine Translation, Named Entity Recognition (NER), etc. However, the advancement of transformers has led to a shift in this paradigm, given their superior performance. Yet, transformers suffer from $O(N^2)$ attention complexity and challenges in handling inductive bias. Several variations have been proposed to address these issues which use spectral networks or convolutions and have performed well on a range of tasks. However, they still have difficulty in dealing with long sequences. State Space Models(SSMs) have emerged as promising alternatives for sequence modeling paradigms in this context, especially with the advent of S4 and its variants, such as S4nd, Hippo, Hyena, Diagnol State Spaces (DSS), Gated State Spaces (GSS), Linear Recurrent Unit (LRU), Liquid-S4, Mamba, etc. In this survey, we categorize the foundational SSMs based on three paradigms namely, Gating architectures, Structural architectures, and Recurrent architectures. This survey also highlights diverse applications of SSMs across domains such as vision, video, audio, speech, language (especially long sequence modeling), medical (including genomics), chemical (like drug design), recommendation systems, and time series analysis, including tabular data. Moreover, we consolidate the performance of SSMs on benchmark datasets like Long Range Arena (LRA), WikiText, Glue, Pile, ImageNet, Kinetics-400, sstv2, as well as video datasets such as Breakfast, COIN, LVU, and various time series datasets. The project page for Mamba-360 work is available on this webpage.\url{https://github.com/badripatro/mamba360}.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 17 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Cortical-SSM: A Deep State Space Model for Motor Imagery Decoding from EEG Signals

    cs.CV 2025-10 conditional novelty 6.0 of 10

    Cortical-SSM, a dual state-space architecture with wavelet-based frequency features, reports state-of-the-art motor-imagery decoding accuracy on OpenBMI, Stieger2021, and a clinical ECoG-ALS dataset.

  2. Focus Through Motion: RGB-Event Collaborative Token Sparsification for Efficient Object Detection

    cs.CV 2025-09 conditional novelty 6.0 of 10

    FocusMamba uses event-camera activity to adaptively prune uninformative tokens in both RGB and event streams, improving detection accuracy and cutting FLOPs.

  3. Systolic Array-based Accelerator for Structured State-Space Models

    cs.LG 2025-07 reject novelty 6.0 of 10

    A specialized systolic-array accelerator with a reconfigurable processing element and diagonal dataflow claims 2000x inference speedup over GPUs for S4 and Liquid-S4 state-space models.

  4. MeshMamba: State Space Models for Articulated 3D Mesh Generation and Reconstruction

    cs.CV 2025-07 conditional novelty 6.0 of 10

    MeshMamba applies Mamba state space models to dense 3D articulated mesh generation and single-image human mesh recovery, reaching over 10,000 vertices with competitive accuracy and faster inference than transformers.

  5. Block-Biased Mamba for Long-Range Sequence Processing

    cs.LG 2025-05 conditional novelty 6.0 of 10

    B2S6, a block-biased variant of Mamba's S6 unit, restores universal approximation, softens input-dependent forgetting, and beats S4/S4D on Long-Range Arena while keeping Mamba-like language perplexity.

  6. XAMBA: Enabling Efficient State Space Models on Resource-Constrained Neural Processing Units

    cs.LG 2025-02 conditional novelty 6.0 of 10

    XAMBA makes Mamba and Mamba-2 state-space models run up to 4.8x faster on an Intel AI PC NPU by replacing CumSum and ReduceSum with matrix operations and activations with piecewise linear approximations.

  7. SMamba: Sparse Mamba for Event-based Object Detection

    cs.CV 2025-01 conditional novelty 6.0 of 10

    SMamba prunes uninformative event tokens using a spatiotemporal continuity score, then scans the survivors with Mamba blocks to match or beat prior event detectors at lower compute.

  8. BlueGlass: A Framework for Composite AI Safety

    cs.AI 2025-07 conditional novelty 5.0 of 10

    BlueGlass provides composite AI safety infrastructure; its case studies on object-detection VLMs reveal dataset trade-offs, a decoder-layer phase transition in probe accuracy, and SAE-discovered concepts including spu...

  9. Vision Mamba in Remote Sensing: A Comprehensive Survey of Techniques, Applications and Outlook

    cs.CV 2025-05 conditional novelty 5.0 of 10

    A structured review of roughly 120 Mamba-based remote sensing papers that proposes taxonomies for scan strategies and architectural integrations, and claims Mamba-based models often outperform CNN and Transformer base...

  10. XYScanNet: A State Space Model for Single Image Deblurring

    cs.CV 2024-12 conditional novelty 5.0 of 10

    An SSM deblurring network that scans image features in interleaved intra-slice and inter-slice routes achieves state-of-the-art perceptual metrics on GoPro, HIDE, RWBI, and RealBlur datasets.

  11. A Deep State Space Model for Rainfall-Runoff Simulations

    cs.LG 2025-01 conditional novelty 4.0 of 10

    Frequency-tuned state space models (S4D-FT) give slightly better median NSE and KGE than LSTM for rainfall-runoff prediction at 531 US watersheds.

  12. A Survey on Large Language Model Acceleration based on KV Cache Management

    cs.AI 2024-12 conditional novelty 4.0 of 10

    A survey that classifies KV cache management techniques for faster LLM inference into token-level, model-level, and system-level categories, with benchmark resources.

  13. A Comparative Study on Dynamic Graph Embedding based on Mamba and Transformers

    cs.LG 2024-12 conditional novelty 4.0 of 10

    Mamba-based dynamic graph embedding models achieve comparable or better link prediction than transformer-based models on five benchmarks, with linear instead of quadratic scaling.

  14. TransXSSM: A Hybrid Transformer State Space Model with Unified Rotary Position Embedding

    cs.CL 2025-06 conditional novelty 3.0 of 10

    TransXSSM applies Rotary Position Embedding to both attention and state-space layers and reports speedups and accuracy gains over Transformer, Mamba, and Jamba baselines at 320M and 1.3B scale.

  15. W4S4: WaLRUS Meets S4 for Long-Range Sequence Modeling

    cs.LG 2025-06 conditional novelty 3.0 of 10

    W4S4 initializes S4 state space models with WaLRUS wavelet frames and reports better delay reconstruction and classification accuracy than HiPPO-based S4, with frozen (A,B).

  16. Exploring Graph Mamba: A Comprehensive Survey on State-Space Models for Graph Learning

    cs.LG 2024-12 conditional novelty 2.0 of 10

    A survey of Graph Mamba, the adaptation of state-space models (Mamba, S4, S6) to graph learning, synthesizing roughly 30 recent papers into a taxonomy of architectures, applications, benchmarks, and open challenges.

  17. A Survey on Mamba Architecture for Vision Applications

    cs.CV 2025-02 conditional novelty 1.0 of 10

    A survey of Mamba-based vision models that summarizes scanning mechanisms, key architectures, and benchmark results, contributing no new experimental findings.

Pith tools