Pith. sign in

REVIEW 7 cited by

Mamba-360: Survey of State Space Models as Transformer Alternative for Long Sequence Modelling: Methods, Applications, and Challenges

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2404.16112 v1 pith:KNUREQJP submitted 2024-04-24 cs.LG cs.AIcs.CVcs.MMeess.IV

Badri Narayana Patro , Vijay Srinivas Agneeswaran This is my paper
classification cs.LGcs.AIcs.CVcs.MMeess.IV
keywords longsequencemodelingssmsstatearchitecturesdatasetsincluding
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
abstract

Sequence modeling is a crucial area across various domains, including Natural Language Processing (NLP), speech recognition, time series forecasting, music generation, and bioinformatics. Recurrent Neural Networks (RNNs) and Long Short Term Memory Networks (LSTMs) have historically dominated sequence modeling tasks like Machine Translation, Named Entity Recognition (NER), etc. However, the advancement of transformers has led to a shift in this paradigm, given their superior performance. Yet, transformers suffer from $O(N^2)$ attention complexity and challenges in handling inductive bias. Several variations have been proposed to address these issues which use spectral networks or convolutions and have performed well on a range of tasks. However, they still have difficulty in dealing with long sequences. State Space Models(SSMs) have emerged as promising alternatives for sequence modeling paradigms in this context, especially with the advent of S4 and its variants, such as S4nd, Hippo, Hyena, Diagnol State Spaces (DSS), Gated State Spaces (GSS), Linear Recurrent Unit (LRU), Liquid-S4, Mamba, etc. In this survey, we categorize the foundational SSMs based on three paradigms namely, Gating architectures, Structural architectures, and Recurrent architectures. This survey also highlights diverse applications of SSMs across domains such as vision, video, audio, speech, language (especially long sequence modeling), medical (including genomics), chemical (like drug design), recommendation systems, and time series analysis, including tabular data. Moreover, we consolidate the performance of SSMs on benchmark datasets like Long Range Arena (LRA), WikiText, Glue, Pile, ImageNet, Kinetics-400, sstv2, as well as video datasets such as Breakfast, COIN, LVU, and various time series datasets. The project page for Mamba-360 work is available on this webpage.\url{https://github.com/badripatro/mamba360}.

Discussion (0). Sign in to comment.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Cortical-SSM: A Deep State Space Model for Motor Imagery Decoding from EEG Signals

    cs.CV 2025-10 conditional novelty 6.0 of 10

    Cortical-SSM, a dual state-space architecture with wavelet-based frequency features, reports state-of-the-art motor-imagery decoding accuracy on OpenBMI, Stieger2021, and a clinical ECoG-ALS dataset.

  2. Focus Through Motion: RGB-Event Collaborative Token Sparsification for Efficient Object Detection

    cs.CV 2025-09 conditional novelty 6.0 of 10

    FocusMamba uses event-camera activity to adaptively prune uninformative tokens in both RGB and event streams, improving detection accuracy and cutting FLOPs.

  3. Systolic Array-based Accelerator for Structured State-Space Models

    cs.LG 2025-07 reject novelty 6.0 of 10

    A specialized systolic-array accelerator with a reconfigurable processing element and diagonal dataflow claims 2000x inference speedup over GPUs for S4 and Liquid-S4 state-space models.

  4. MeshMamba: State Space Models for Articulated 3D Mesh Generation and Reconstruction

    cs.CV 2025-07 conditional novelty 6.0 of 10

    MeshMamba applies Mamba state space models to dense 3D articulated mesh generation and single-image human mesh recovery, reaching over 10,000 vertices with competitive accuracy and faster inference than transformers.

  5. BlueGlass: A Framework for Composite AI Safety

    cs.AI 2025-07 conditional novelty 5.0 of 10

    BlueGlass provides composite AI safety infrastructure; its case studies on object-detection VLMs reveal dataset trade-offs, a decoder-layer phase transition in probe accuracy, and SAE-discovered concepts including spu...

  6. TransXSSM: A Hybrid Transformer State Space Model with Unified Rotary Position Embedding

    cs.CL 2025-06 conditional novelty 3.0 of 10

    TransXSSM applies Rotary Position Embedding to both attention and state-space layers and reports speedups and accuracy gains over Transformer, Mamba, and Jamba baselines at 320M and 1.3B scale.

  7. W4S4: WaLRUS Meets S4 for Long-Range Sequence Modeling

    cs.LG 2025-06 conditional novelty 3.0 of 10

    W4S4 initializes S4 state space models with WaLRUS wavelet frames and reports better delay reconstruction and classification accuracy than HiPPO-based S4, with frozen (A,B).

Pith tools