Pith. sign in

REVIEW 5 cited by

Sigma: Siamese Mamba Network for Multi-Modal Semantic Segmentation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2404.04256 v3 pith:WVZIH7QF submitted 2024-04-05 cs.CV

Zifu Wan , Pingping Zhang , Yuhao Wang , Silong Yong , Simon Stepputtis , Katia Sycara , Yaqi Xie This is my paper
classification cs.CV
keywords multi-modalsegmentationsemanticfieldsmambareceptivesiamesesigma
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Multi-modal semantic segmentation significantly enhances AI agents' perception and scene understanding, especially under adverse conditions like low-light or overexposed environments. Leveraging additional modalities (X-modality) like thermal and depth alongside traditional RGB provides complementary information, enabling more robust and reliable prediction. In this work, we introduce Sigma, a Siamese Mamba network for multi-modal semantic segmentation utilizing the advanced Mamba. Unlike conventional methods that rely on CNNs, with their limited local receptive fields, or Vision Transformers (ViTs), which offer global receptive fields at the cost of quadratic complexity, our model achieves global receptive fields with linear complexity. By employing a Siamese encoder and innovating a Mamba-based fusion mechanism, we effectively select essential information from different modalities. A decoder is then developed to enhance the channel-wise modeling ability of the model. Our proposed method is rigorously evaluated on both RGB-Thermal and RGB-Depth semantic segmentation tasks, demonstrating its superiority and marking the first successful application of State Space Models (SSMs) in multi-modal perception tasks. Code is available at https://github.com/zifuwan/Sigma.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Multi-Modal Object Re-Identification with Prompt-S6 and Semantic-Aware Knowledge Guidance

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Prompt-S6 plus semantic token pruning and progressive tri-modal fusion improves multi-spectral object ReID accuracy and efficiency on four benchmarks.

  2. Focus Through Motion: RGB-Event Collaborative Token Sparsification for Efficient Object Detection

    cs.CV 2025-09 conditional novelty 6.0 of 10

    FocusMamba uses event-camera activity to adaptively prune uninformative tokens in both RGB and event streams, improving detection accuracy and cutting FLOPs.

  3. UIS-Mamba: Exploring Mamba for Underwater Instance Segmentation via Dynamic Tree Scan and Hidden State Weaken

    cs.CV 2025-08 conditional novelty 6.0 of 10

    UIS-Mamba applies a Mamba state space backbone with dynamic tree scanning and background-hidden-state suppression to achieve SOTA underwater instance segmentation.

  4. MoMa: Modulating Mamba for Adapting Image Foundation Models to Video Recognition

    cs.CV 2025-06 conditional novelty 5.0 of 10

    MoMa adapts frozen CLIP to video by injecting Mamba-computed scale and bias into each layer, improving accuracy and efficiency on multiple action recognition benchmarks.

  5. Cross-modal State Space Modeling for Real-time RGB-thermal Wild Scene Semantic Segmentation

    cs.CV 2025-06 conditional novelty 4.0 of 10

    CM-SSM reports 74.6% mIoU on the CART RGB-thermal benchmark and 85.9% on PST900, using 12.59M parameters and running at 114 FPS on an RTX 4090.

Pith tools