REVIEW 6 cited by
State Space Model for New-Generation Network Alternative to Transformers: A Survey
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
In the post-deep learning era, the Transformer architecture has demonstrated its powerful performance across pre-trained big models and various downstream tasks. However, the enormous computational demands of this architecture have deterred many researchers. To further reduce the complexity of attention models, numerous efforts have been made to design more efficient methods. Among them, the State Space Model (SSM), as a possible replacement for the self-attention based Transformer model, has drawn more and more attention in recent years. In this paper, we give the first comprehensive review of these works and also provide experimental comparisons and analysis to better demonstrate the features and advantages of SSM. Specifically, we first give a detailed description of principles to help the readers quickly capture the key ideas of SSM. After that, we dive into the reviews of existing SSMs and their various applications, including natural language processing, computer vision, graph, multi-modal and multi-media, point cloud/event stream, time series data, and other domains. In addition, we give statistical comparisons and analysis of these models and hope it helps the readers to understand the effectiveness of different structures on various tasks. Then, we propose possible research points in this direction to better promote the development of the theoretical model and application of SSM. More related works will be continuously updated on the following GitHub: https://github.com/Event-AHU/Mamba_State_Space_Model_Paper_List.
Forward citations
Cited by 6 Pith papers
-
Geometric Hyena Networks for Large-scale Equivariant Learning
Geometric Hyena is an equivariant long-convolutional architecture that captures global geometric context with sub-quadratic complexity and outperforms equivariant transformer baselines on several RNA and protein predi...
-
Cortical-SSM: A Deep State Space Model for Motor Imagery Decoding from EEG Signals
Cortical-SSM, a dual state-space architecture with wavelet-based frequency features, reports state-of-the-art motor-imagery decoding accuracy on OpenBMI, Stieger2021, and a clinical ECoG-ALS dataset.
-
A Deep State-Space Model Compression Method using Upper Bound on Output Error
A new upper bound ties end-to-end deep state-space model compression error to layerwise H2 approximation errors, enabling a gradient-based method that cuts ~80% of parameters with a modest accuracy drop.
-
TrackingMiM: Efficient Mamba-in-Mamba Serialization for Real-time UAV Object Tracking
A nested Mamba-in-Mamba architecture with template-first spatial scans, temporal serialization scans, and retrieval-augmented query attention reports top average precision and success on five UAV tracking benchmarks.
-
Mamba-FETrack V2: Revisiting State Space Model for Frame-Event based Visual Object Tracking
Mamba-FETrack V2 fuses RGB and event streams inside a Vision Mamba backbone, achieving 53.8% success rate on FELT V2 with 30M parameters and 29 FPS.
-
FMaMIL: Frequency-Driven Mamba Multi-Instance Learning for Weakly Supervised Lesion Segmentation in Medical Images
FMaMIL combines Mamba-based multiple instance learning with learnable frequency-domain encoding and CAM-guided pseudo-label refinement to segment lesions from image-level labels only.
Discussion (0). Sign in to comment.