REVIEW 17 cited by
MambaVision: A Hybrid Mamba-Transformer Vision Backbone
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
MambaVision: A Hybrid Mamba-Transformer Vision Backbone
read the original abstract
We propose a novel hybrid Mamba-Transformer backbone, MambaVision, specifically tailored for vision applications. Our core contribution includes redesigning the Mamba formulation to enhance its capability for efficient modeling of visual features. Through a comprehensive ablation study, we demonstrate the feasibility of integrating Vision Transformers (ViT) with Mamba. Our results show that equipping the Mamba architecture with self-attention blocks in the final layers greatly improves its capacity to capture long-range spatial dependencies. Based on these findings, we introduce a family of MambaVision models with a hierarchical architecture to meet various design criteria. For classification on the ImageNet-1K dataset, MambaVision variants achieve state-of-the-art (SOTA) performance in terms of both Top-1 accuracy and throughput. In downstream tasks such as object detection, instance segmentation, and semantic segmentation on MS COCO and ADE20K datasets, MambaVision outperforms comparably sized backbones while demonstrating favorable performance. Code: https://github.com/NVlabs/MambaVision
Forward citations
Cited by 17 Pith papers
-
Exemplar-Free Continual Learning for State Space Models
Inf-SSM constrains the infinite-horizon evolution of SSMs via Grassmannian geometry and an efficient O(n^2) Sylvester solver to enable exemplar-free continual learning with reduced forgetting.
-
FractalMamba++: Scaling Vision Mamba Across Resolutions via Hilbert Fractal Geometry
FractalMamba++ scales Vision Mamba across resolutions by using Hilbert fractal serialization, hierarchy-based skip connections, and fractal-aware 2D rotary position encoding.
-
Sign Language Question Answering: A New Task, Benchmark, and Baseline for Sign Language Understanding
Sign Language QA benchmarks are introduced from PHOENIX14T and CSL-Daily via template-generated questions, and a question-conditioned baseline outperforms video-language and cascaded baselines.
-
Zamba2-VL Technical Report
Zamba2-VL is a family of 1.2B–7B hybrid Mamba2-transformer vision-language models that match leading transformer VLMs on image, reasoning, OCR, grounding and counting benchmarks while delivering roughly 10x lower time...
-
A Controlled Benchmark of Visual State-Space Backbones with Domain-Shift and Boundary Analysis for Remote-Sensing Segmentation
Controlled tests on LoveDA and ISPRS Potsdam show visual SSM encoders deliver favorable speed-accuracy trade-offs but suffer most from boundary errors under domain shift, indicating that robustness and boundary-aware ...
-
HAMSA: Scanning-Free Vision State Space Models via SpectralPulseNet
HAMSA achieves 85.7% ImageNet-1K top-1 accuracy as a spectral-domain SSM with 2.2x faster inference and lower memory than transformers or scanning-based SSMs.
-
Catch-Only-One: Non-Transferable Examples for Model-Specific Authorization
Non-transferable examples add perturbations in an authorized model's low-sensitivity subspace, keeping that model's accuracy nearly intact while collapsing the accuracy of all other tested models.
-
How Can Mamba Learn In Context with Outliers and Generalize Provably?
A simplified one-layer Mamba provably learns in-context binary classification tolerating outlier fractions approaching 1, whereas a linear Transformer can only tolerate α < 1/2.
-
Less is More: Efficient Black-box Attribution via Minimal Interpretable Subset Selection
LiMA reformulates attribution as submodular subset selection and uses bidirectional greedy search to identify minimal important regions, reporting 36.3% better insertion and 39.6% better deletion scores than prior met...
-
An approach with Visual and Tabular Mamba to multimodal medical data using Mixed Fusion
Mamba-based visual and tabular models with mixed fusion achieve competitive balanced accuracy and higher recall than prior methods on skin and oral cancer datasets while enabling SHAP interpretability.
-
MOSAIC: Efficient Mixture-of-Agent Scheduling via Adaptive Aggregation and Inference Concurrency
MOSAIC uses an Integer Linear Program scheduler for expert placement and prompt assignment plus adaptive aggregation to achieve 1.7-2.3x end-to-end speedup on 4-GPU MoA workloads while keeping accuracy within 0.1pp.
-
Fusion Complexity Inversion: Why Simpler Cross View Modules Outperform SSMs and Cross View Attention Transformers for Pasture Biomass Regression
On scarce dual-view pasture data, a simple two-layer gated depthwise convolution fusion achieves R²=0.903, beating cross-view attention transformers (0.833), bidirectional SSMs (0.819), and Mamba (0.793), while backbo...
-
ENA: Efficient N-dimensional Attention
ENA combines linear recurrence with hardware-friendly sliding tile attention to model images and videos efficiently, claiming Transformer-level accuracy at roughly 70% attention sparsity.
-
A Controlled Visual-Backbone Benchmark for Multimodal Short-Term Solar Irradiance Forecasting
Under one fixed multimodal 10-min irradiance forecaster, VMamba-S and Swin-B nearly tie best Folsom RMSE (~65.4 W/m²) over smart persistence, while NREL’s tiny matched split favors persistence.
-
Efficient PEFT Methods with Adaptive Checkpointing for Vision Models and VLMs on Resource Constrained Consumer-GPUs
Compares PEFT methods (LoRA, QLoRA, BitFit etc.) plus a new adaptive checkpointing strategy on ViT/Mamba vision models and VLMs, showing 20-30% energy cuts and 43-79% memory reduction at small accuracy cost on CIFAR-100/DTD.
-
VCMamba: Bridging Convolutions with Multi-Directional Mamba for Efficient Visual Representation
VCMamba reports that using convolutional feed-forward blocks for the first three stages followed by multi-directional Mamba blocks in the final stage yields 82.6% ImageNet-1K and 47.1 ADE20K mIoU at 31.5M parameters, ...
-
A Survey of Mamba
The paper consolidates existing research on Mamba models, their architecture variants, adaptations to different data modalities, and applications across domains.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.