Pith. sign in

REVIEW 3 major objections 5 minor 87 references

HiddenObject: Modality-Agnostic Fusion for Multimodal Hidden Object Detection

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read HiddenObject claims a single Mamba-based fusion network outperforms prior RGB-thermal and RGB-depth detectors on four of five benchmarks.

desk verdict A plausible Mamba fusion variant that is undermined by overclaimed SOTA numbers and missing significance tests; worth a careful revision, not a desk reject. read the letter →

arxiv 2508.21135 v2 pith:OIINRUIH submitted 2025-08-28 cs.CV cs.AI

Harris Song , Tuan-Anh Vu , Sanjith Menon , Sriram Narasimhan , M. Khalid Jawed This is my paper
classification cs.CVcs.AI
keywords hiddenobjectdetectioncamouflagedmultimodalfusionMambastatespacemodelsRGB-thermalsegmentationRGB-depthselectivescan
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that one fusion architecture, HiddenObject, can detect hidden or partially concealed objects equally well whether the second sensor is thermal or depth, without redesigning the network per modality. The architecture pairs a shared-weight Mamba encoder with a multiscale fusion module and a decoder that lets the higher-level RGB feature control how the hidden state is read out. On the benchmarks tested, the paper reports the highest mean IoU on MFNet (61.5), PST900 (88.5), and SUN RGB-D (52.1), and best camouflaged-object-detection scores on ACOD-12K (S_alpha 0.865, F_beta 0.807, E_phi 0.965). The reason to care is practical: if true, one model and one training recipe cover RGB-thermal and RGB-depth concealed-object detection, improving on dedicated per-modality designs.

What carries the argument

The central mechanism is the multimodal selective-scan fusion block (MMFF) plus a decoder-side assignment of the state-space readout. In a selective state-space scan, the output is y = C h, and which matrix C is chosen determines what gets extracted from the hidden state. The decoder computes A, B, and Delta from the lower-level feature, feeds the lower-level embedding into the scan, but computes C from the higher-level (RGB-anchored) feature, so the readout is conditioned on the richer modality. In MMFF, two modality features are linearly projected, scanned through SS2D in four directions, and the concatenated sequence is also scanned in reverse, flipped back, added, and re-split, then scal

What would settle it

Re-run every baseline in Tables 2-4 under HiddenObject's exact protocol: same optimizer, learning rate, epochs, batch size, 640x480 input, and same train/val splits, then check whether HiddenObject still holds the top mIoU, S_alpha, F_beta, and E_phi values. Separately, drop the thermal or depth channel at inference and measure the performance drop to test whether the claimed modality-agnostic fusion actually preserves detection when a sensor fails.

Watch

Extended reading notes

Core claim

In the paper's own terms, the discovery is that a Mamba (selective state-space) backbone can serve as a modality-agnostic fusion engine for hidden object detection. The network encodes RGB and an X-modality (thermal or depth) with two weight-shared Vision Mamba branches, then fuses at four scales through a Multimodal Feature Fusion module whose cross-scan and reverse-scan sequence exchange lets each modality read the other's context. Its channel-aware decoder performs SS2D scans where the A, B, and Delta matrices come from lower-level features while matrix C, the readout from the hidden state, comes from the higher-level feature, giving the stronger modality control over the output. Reported

Load-bearing premise

The reported superiority assumes that published baseline numbers from different backbones, input resolutions, and training recipes can be compared directly with a single new run under the authors' setup; if those baselines were re-run under the same protocol and scored differently, the claimed margins would change.

Editorial extensions

If this is right

  • One training protocol covers RGB-thermal, RGB-depth, and any future X-modality, so deploying the model in agriculture or robotics with different sensors does not require architecture changes.
  • The multiscale fusion plus Mamba linear-complexity design suggests the gains do not rely on a heavy transformer decoder; the VMamba-S backbone keeps the model comparatively light.
  • The decoder's C-from-higher-feature readout provides a natural handle for missing-modality scenarios: when one sensor drops out, the remaining modality can still drive the hidden-state readout.
  • The ACOD-12K results bring camouflaged-object-detection metrics into multimodal benchmarks, allowing structural measures such as S_alpha and E_phi rather than only mIoU.
  • Because the same code sets new bests across datasets, existing unimodal detectors that fail under occlusion and camouflage could be upgraded by swapping in the fusion mechanism without task-specific redesign.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A fair test of the claimed margins would re-run each baseline in Tables 2-4 under HiddenObject's exact training protocol; if those numbers shift, the architecture claim may survive while the precise margins change.
  • The paper's modality-agnostic claim predicts that NIR, event-camera, or any aligned extra modality could be fed through the same X-modality branch with no architectural change, and that this should generalize; this is directly testable.
  • The decoder's choice to compute C from the higher-level feature rather than from the fused feature is not ablated; isolating that choice would show where the gain actually comes from.
  • Because the thermal and depth experiments share weights, the model could be trained jointly on RGB-T and RGB-D data in one pass; the paper does not test this combined setting.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The manuscript proposes HiddenObject, a multimodal fusion framework built on VMamba SS2D modules. It uses a dual-stream encoder with shared weights for RGB and X-modality, an MMFF module that scans a concatenated RGB-X sequence and adds a reverse-scan flip-back branch, and a channel-aware decoder in which the SS2D matrix C is computed from higher-level or RGB features while A, B, and Δ come from lower-level features. The method is evaluated on RGB-T semantic segmentation (MFNet, PST900), RGB-D segmentation (NYU Depth V2, SUN RGB-D), and RGB-D concealed-object detection (ACOD-12K). The authors report the highest mIoU on MFNet and PST900 and claim significant improvements over all existing methods on ACOD-12K.

Significance. If the comparisons were properly controlled, the MMFF arrangement and the decoder's asymmetric use of the RGB feature would be a useful baseline for modality-agnostic hidden object detection, especially given the limited number of Mamba-based multimodal fusion studies on concealed-object benchmarks. The paper also benefits from a broad benchmark suite spanning thermal and depth modalities. However, the current evidence does not establish the stated SOTA claims: baseline numbers are taken from heterogeneous settings, the ACOD-12K 'significant' claim conflicts with the paper's own table and discussion, and no ablation isolates the proposed components. The architecture may well be competitive, but the validation as presented is insufficient.

major comments (3)
  1. [Abstract, §4.2, Table 4] The claim that 'our model significantly outperforms existing methods across all metrics' on ACOD-12K is not supported by Table 4. Compared with RISNet, Ours is 0.865 vs 0.87 in Sα (lower after rounding), 0.807 vs 0.80 in Fβ (+0.007), and 0.965 vs 0.97 in Eφ (tied within rounding). No error bars or significance tests are reported, and the Discussion in §4.2 admits that 'the observed fluctuations are not statistically significant in most cases.' This is an internal inconsistency in the paper's strongest claim; it must be replaced by a defensible statement and supported by multi-seed runs or statistical tests.
  2. [§4.1, Tables 2-3] The SOTA and 'highest' claims rely on direct comparison with published baseline numbers obtained under different backbones, input resolutions, and training protocols. For example, Table 3 mixes ResNet-152, MiT-B3/B4, ViT-B, MobileNet-V2, and VMamba-S at resolutions from 480×480 to 640×640. Crucially, the text says Ours 'achieves the highest mIoU of 52.1 on SUN RGB-D,' but Table 3 lists Sigma (VMamba-S) at 52.4, so this is factually wrong. Likewise, on NYU, Ours is third (56.8) behind Sigma (57.0) and CMNeXt (56.9), not merely 'slightly behind CMNeXt.' The authors need to re-run baselines under their own protocol or clearly qualify the comparisons as cross-paper.
  3. [§3.3, §3.4, Tables 2-4] No ablation isolates the novelty. The closest comparison, Sigma (VMamba-S), differs from Ours in multiple ways: fusion arrangement, decoder C computation, scaling parameters, shuffling, and upsampling. Without removing or replacing the proposed MMFF and decoder components one at a time, the reported gains cannot be attributed to the paper's contribution. A minimal ablation should include: (a) Sigma's fusion in the same encoder, (b) Ours without the reverse-scan flip-back addition, and (c) decoder C computed from the lower-level or RGB feature instead of the higher-level feature. This is necessary even to support 'competitive' rather than 'SOTA.'
minor comments (5)
  1. [Figure 6] The caption says 'PSD900' but should be 'PST900.'
  2. [Figure 1] The caption uses 'ACOD-K12' while the rest of the paper uses 'ACOD-12K.'
  3. [§4.1 Implementation Detail] The text says 'All methods were trained and evaluated on a cluster equipped with NVIDIA RTX 3090 GPUs.' This cannot be true for baseline scores copied from other papers; please clarify which numbers were re-run and which were taken from publications.
  4. [Table 3] The table would benefit from a column noting the evaluation protocol and training data, and the 'In. Size' column should be consistently reported for every method.
  5. [§4.2 Discussion] The Discussion assigns all metric differences to 'not statistically significant' without reporting variances, confidence intervals, or seeds. Either provide the underlying statistics or remove the statistical phrasing.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is an empirical architecture/benchmark study; the reported numbers come from direct evaluation on external datasets and no prediction reduces to a fitted parameter or self-citation chain.

full rationale

This is an empirical architecture paper, not a derivation. The claimed contributions — a Mamba-based fusion mechanism and a channel-aware decoder — are described as architectural designs, and the evidence is direct benchmark performance on external datasets (MFNet, PST900, NYU Depth V2, SUN RGB-D, ACOD-12K). I checked the seven circularity patterns. (1) Self-definitional: no quantity is defined in terms of the target result; the decoder's use of the RGB feature for matrix C is an architectural choice, not a definition that forces the reported scores. (2) Fitted input called prediction: no parameter is fit to a subset and then reported as a prediction; all metrics are measured on test sets. (3) Self-citation load-bearing: the paper cites prior work by one of the authors ([2], [3]) only to motivate why RGB-only detection is hard; these citations do not support the architecture's validity or the benchmark claims. (4) Uniqueness imported from authors: no uniqueness theorem is invoked. (5) Ansatz smuggled in via citation: the design is said to be 'Inspired by CMX and Sigma', but these are external prior works and the paper explicitly lists what is distinct about the fusion module; no core premise is adopted merely by self-citation. (6) Renaming known result: the paper does not rename an existing pattern; it presents a new network and compares it with baselines. The main weaknesses — baseline comparability across different backbones/resolutions and the 'significantly outperforms' wording on ACOD-12K despite near-tied numbers — are correctness/evidentiary concerns, not circularity. There is no equation, fitted parameter, or self-citation chain that makes the reported results equivalent to the inputs by construction. Therefore the appropriate circularity score is 0.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The central claim rests on standard Mamba math, pretrained weights, benchmark comparability, and several unablated design choices. There are no invented physical or theoretical entities. The main unexamined degrees of freedom are the training hyperparameters and the specific scan/attention arrangements.

free parameters (3)
  • Learning rate, weight decay, epochs, batch size = 6e-5, 0.01, 500, 8
    Copied from CMX/Sigma rather than derived; no sensitivity analysis is provided, so the reported results are conditional on these choices.
  • Input resolution = 640x480
    Fixed for all experiments; differs from several baselines (480x640, 530x730) in Table 3, making the comparisons indirect.
  • MMFF scaling parameters and channel widths
    Learned or fixed by network width choices; no ablation justifies the specific arrangement claimed as novel.
assumptions (6)
  • standard math ZOH discretization and first-order Taylor approximation of B are valid for the SS2D modules
    Section 3.1, Eqs. (2)-(5), inherited from Mamba; not re-derived by this paper.
  • domain assumption ImageNet-1K pretrained VMamba weights, shared between the RGB and X-modality streams, produce useful features for thermal and depth inputs
    Section 3.2 uses shared weights for Irgb and Ix; no per-modality adaptation or experiment isolates this assumption.
  • domain assumption Published baseline numbers from different papers are directly comparable to the authors' single runs
    Section 4.1 and Tables 2-4 compare across different backbones, input resolutions, and training protocols without re-running baselines.
  • ad hoc to paper Computing decoder SS2D matrix C from the RGB feature while A, B, and delta come from lower-level features improves detection
    Section 3.4 and Figure 4; the choice is asserted, not ablated.
  • ad hoc to paper Reversed scan of the concatenated RGB-X sequence in MMFF, followed by flip-back and addition, is beneficial
    Section 3.3 and Figure 5; no comparison against forward-only or other scan strategies is provided.
  • domain assumption ACOD-12K COD metrics are appropriate for this task and comparable to prior reported values
    Section 4.1 Dataset Overview; the paper switches from semantic segmentation metrics to COD metrics for ACOD, which is a protocol choice.

how reviews work

0 comments
Cite this review

Pith. "Pith review of HiddenObject: Modality-Agnostic Fusion for Multimodal Hidden Object Detection." pith.science (2026). https://pith.science/paper/OIINRUIH

@misc{pith2026250821135,
  author       = {Pith},
  title        = {Pith review of: HiddenObject: Modality-Agnostic Fusion for Multimodal Hidden Object Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OIINRUIH}},
  note         = {Machine review of arXiv:2508.21135}
}
read the original abstract

Detecting hidden or partially concealed objects remains a fundamental challenge in multimodal environments, where factors like occlusion, camouflage, and lighting variations significantly hinder performance. Traditional RGB-based detection methods often fail under such adverse conditions, motivating the need for more robust, modality-agnostic approaches. In this work, we present HiddenObject, a fusion framework that integrates RGB, thermal, and depth data using a Mamba-based fusion mechanism. Our method captures complementary signals across modalities, enabling enhanced detection of obscured or camouflaged targets. Specifically, the proposed approach identifies modality-specific features and fuses them in a unified representation that generalizes well across challenging scenarios. We validate HiddenObject across multiple benchmark datasets, demonstrating state-of-the-art or competitive performance compared to existing methods. These results highlight the efficacy of our fusion design and expose key limitations in current unimodal and na\"ive fusion strategies. More broadly, our findings suggest that Mamba-based fusion architectures can significantly advance the field of multimodal object detection, especially under visually degraded or complex conditions.

Figures

Figures reproduced from arXiv: 2508.21135 by the authors.

Figure 1
Figure 1. Our proposed method effectively addresses severe [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Comparison of different multimodal fusion methods. (a) Training a feature extractor using only RGB images. (b) Sharing a trainable feature extractor between RGB images and other modalities. (c) Single-scale feature fusion within the model. (d) Multiscale feature fusion (Our method). However, fusing multiple modalities into a cohesive detection framework introduces considerable technical chal￾lenges, such as modality… view at source ↗
Figure 3
Figure 3. Architecture overview of our proposed HiddenObject network, which integrates RGB and additional modality inputs for robust hidden object detection. demonstrating the advantage of mid-level feature fusion. Subsequent efforts developed CNN-based attention mecha￾nisms to enhance cross-modal synergy [39], [40], [41]. More recent transformer-based fusion strategies [42], [43], [44], [45] exploit broader complementary int… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The architecture of Vision Mamba Encoder and Decoder. For the SS2D module in the decoder, the matrices A, B, and ∆ are computed from the X-modality, the input of the selective scan is the X-modality embedding, and the matrix C is calculated using the RGB feature. 3.2 M…
Figure 5
Figure 5. Figure 5: Multimodal Feature Fusion (MMFF) Module. [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Qualitative comparison on MFNet (Left) and PSD900 (Right) Datasets. [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Qualitative comparison on ACOD-12K Dataset. [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

87 extracted references · 76 canonical work pages

  1. [1]

    Object detection in 20 years: A survey,

    Z. Zou, K. Chen, Z. Shi, Y. Guo, and J. Ye, “Object detection in 20 years: A survey,”Proceedings of the IEEE, vol. 111, no. 3, pp. 257–276,

  2. [2]

    Leveraging open-vocabulary diffusion to camouflaged instance segmentation,

    T.-A. Vu, D. T. Nguyen, Q. Guo, B.-S. Hua, N. M. Chung, I. W. Tsang, and S.-K. Yeung, “Leveraging open-vocabulary diffusion to camouflaged instance segmentation,” 2023. [Online]. Available: https://arxiv.org/abs/2312.17505 1

    work page arXiv 2023

  3. [3]

    Transcues: Boundary and reflection-empowered pyramid vision transformer for semantic transparent object segmentation,

    T.-A. Vu, N. T. Hai, Z. Zheng, B.-S. Hua, Q. Guo, I. Tsang, and S.-K. Yeung, “Transcues: Boundary and reflection-empowered pyramid vision transformer for semantic transparent object segmentation,” 2024. [Online]. Available: https://openreview.net/ forum?id=e9bEoxNiTJ 1

    2024

  4. [4]

    Thermal imaging system and its real-time applications: a survey,

    M. Rai, T. Maity, and R. Yadav, “Thermal imaging system and its real-time applications: a survey,”Journal of Engineering Technology, vol. 6, no. 2, pp. 290–303, 2017. 1

    2017

  5. [5]

    A survey on rgb-d datasets,

    A. Lopes, R. Souza, and H. Pedrini, “A survey on rgb-d datasets,” Computer Vision and Image Understanding, vol. 222, p. 103489, 2022. 1

    2022

  6. [6]

    Seeing through fog without seeing fog: Deep multimodal sensor fusion in unseen adverse weather,

    M. Bijelic, T. Gruber, F. Mannan, F. Kraus, W. Ritter, K. Dietmayer, and F. Heide, “Seeing through fog without seeing fog: Deep multimodal sensor fusion in unseen adverse weather,” inThe IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June

  7. [7]

    Multimodal dataset for localization, mapping and crop monitoring in citrus tree farms,

    H. Teng, Y. Wang, X. Song, and K. Karydis, “Multimodal dataset for localization, mapping and crop monitoring in citrus tree farms,” in Advances in Visual Computing. Cham: Springer Nature Switzerland, 2023, pp. 571–582. 1

    2023

  8. [8]

    Depth- aware concealed crop detection in dense agricultural scenes,

    L. Wang, J. Yang, Y. Zhang, F. Wang, and F. Zheng, “Depth- aware concealed crop detection in dense agricultural scenes,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2024, pp. 17 201–17 211. 1, 6, 7, 8

    2024

Show all 87 references
  1. [9]

    Citdet: A benchmark dataset for citrus fruit detection,

    J. James, H. Manching, M. Mattia, K. Bowman, A. Hulse-Kemp, and W. Beksi, “Citdet: A benchmark dataset for citrus fruit detection,” IEEE Robotics and Automation Letters, vol. PP , pp. 1–8, 12 2024. 1

    2024

  2. [10]

    Multi-modal fusion trans- former for end-to-end autonomous driving,

    A. Prakash, K. Chitta, and A. Geiger, “Multi-modal fusion trans- former for end-to-end autonomous driving,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 7077–7087. 1

    2021

  3. [11]

    Multi-level fusion for robust rgbt tracking via enhanced thermal representation,

    Z. Tang, T. Xu, X.-J. Wu, and J. Kittler, “Multi-level fusion for robust rgbt tracking via enhanced thermal representation,”ACM Trans- actions on Multimedia Computing, Communications and Applications, vol. 20, no. 10, pp. 1–24, 2024. 1

    2024

  4. [12]

    Mfnet: Towards real-time semantic segmentation for autonomous vehicles with multi-spectral scenes,

    Q. Ha, K. Watanabe, T. Karasawa, Y. Ushiku, and T. Harada, “Mfnet: Towards real-time semantic segmentation for autonomous vehicles with multi-spectral scenes,” in2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2017, pp. 5108–

    2017

  5. [13]

    Rtfnet: Rgb-thermal fusion network for semantic segmentation of urban scenes,

    Y. Sun, W. Zuo, and M. Liu, “Rtfnet: Rgb-thermal fusion network for semantic segmentation of urban scenes,”IEEE Robotics and Automation Letters, vol. 4, no. 3, pp. 2576–2583, 2019. 2, 5, 6

    2019

  6. [14]

    Pst900: Rgb-thermal calibration, dataset and segmentation network,

    S. S. Shivakumar, N. Rodrigues, A. Zhou, I. D. Miller, V . Kumar, and C. J. Taylor, “Pst900: Rgb-thermal calibration, dataset and segmentation network,” 2019. 2, 5, 6

    2019

  7. [15]

    Fruity: A multi-modal dataset for fruit recognition and 6D-Pose estimation in precision agriculture,

    M. Abdulsalam, Z. Chekakta, N. Aouf, and M. Hogan, “Fruity: A multi-modal dataset for fruit recognition and 6D-Pose estimation in precision agriculture,” in2023 31st Mediterranean Conference on Control and Automation (MED). IEEE, Jun. 2023, pp. 144–149. 2

    2023

  8. [16]

    Firestereo: Forest infrared stereo dataset for uas depth perception in visually degraded environments,

    D. Dhrafani, Y. Liu, A. Jong, U. Shin, Y. He, T. Harp, Y. Hu, J. Oh, and S. Scherer, “Firestereo: Forest infrared stereo dataset for uas depth perception in visually degraded environments,”IEEE Robotics and Automation Letters, vol. 10, no. 4, pp. 3302–3309, 2025. 2

    2025

  9. [17]

    Caltech aerial rgb-thermal dataset in the wild,

    C. Lee, M. Anderson, N. Ranganathan, X. Zuo, K. Do, G. Gkioxari, and S.-J. Chung, “Caltech aerial rgb-thermal dataset in the wild,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 236–256. 2

    2024

  10. [18]

    End-to-end object detection with transformers,

    N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, “End-to-end object detection with transformers,” in European conference on computer vision. Springer, 2020, pp. 213–229. 2

    2020

  11. [19]

    Deformable {detr}: Deformable transformers for end-to-end object detection,

    X. Zhu, W. Su, L. Lu, B. Li, X. Wang, and J. Dai, “Deformable {detr}: Deformable transformers for end-to-end object detection,” inInternational Conference on Learning Representations, 2021. 2

    2021

  12. [20]

    Conditional detr for fast training convergence,

    D. Meng, X. Chen, Z. Fan, G. Zeng, H. Li, Y. Yuan, L. Sun, and J. Wang, “Conditional detr for fast training convergence,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 3651–3660. 2

    2021

  13. [21]

    Efficient detr: Improv- ing end-to-end object detector with dense prior,

    Z. Yao, J. Ai, B. Li, and C. Zhang, “Efficient detr: Improv- ing end-to-end object detector with dense prior,”arXiv preprint arXiv:2104.01318, 2021. 2

    2021 arXiv

  14. [22]

    DAB-DETR: Dynamic anchor boxes are better queries for DETR,

    S. Liu, F. Li, H. Zhang, X. Yang, X. Qi, H. Su, J. Zhu, and L. Zhang, “DAB-DETR: Dynamic anchor boxes are better queries for DETR,” inInternational Conference on Learning Representations, 2022. 2

    2022

  15. [23]

    Dn- detr: Accelerate detr training by introducing query denoising,

    F. Li, H. Zhang, S. Liu, J. Guo, L. M. Ni, and L. Zhang, “Dn- detr: Accelerate detr training by introducing query denoising,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 13 619–13 627. 2

    2022

  16. [24]

    DINO: DETR with improved denoising anchor boxes for end-to-end object detection,

    H. Zhang, F. Li, S. Liu, L. Zhang, H. Su, J. Zhu, L. Ni, and H.-Y. Shum, “DINO: DETR with improved denoising anchor boxes for end-to-end object detection,” inThe Eleventh International Conference on Learning Representations, 2023. 2

    2023

  17. [25]

    Detrs beat yolos on real-time object detection,

    Y. Zhao, W. Lv, S. Xu, J. Wei, G. Wang, Q. Dang, Y. Liu, and J. Chen, “Detrs beat yolos on real-time object detection,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 16 965–16 974. 2 10

    2024

  18. [26]

    Rt-detrv2: Improved baseline with bag-of-freebies for real-time detection transformer,

    W. Lv, Y. Zhao, Q. Chang, K. Huang, G. Wang, and Y. Liu, “Rt-detrv2: Improved baseline with bag-of-freebies for real-time detection transformer,” 2024. [Online]. Available: https://arxiv.org/abs/2407.17140 2

    2024 arXiv

  19. [27]

    Mamba: Linear-time sequence modeling with selective state spaces,

    A. Gu and T. Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” inFirst Conference on Language Modeling,

  20. [28]

    Vision mamba: efficient visual representation learning with bidirectional state space model,

    L. Zhu, B. Liao, Q. Zhang, X. Wang, W. Liu, and X. Wang, “Vision mamba: efficient visual representation learning with bidirectional state space model,” inProceedings of the 41st International Conference on Machine Learning, ser. ICML’24, 2024. 2

    2024

  21. [29]

    Training data-efficient image transformers & distillation through attention,

    H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles, and H. J´egou, “Training data-efficient image transformers & distillation through attention,” inInternational Conference on Machine Learning. PMLR, 2021, pp. 10 347–10 357. 2

    2021

  22. [30]

    Mamba yolo: A simple baseline for object detection with state space model,

    Z. Wang, C. Li, H. Xu, and X. Zhu, “Mamba yolo: A simple baseline for object detection with state space model,” inThe 39th Annual AAAI Conference on Artificial Intelligence, 2025. 2

    2025

  23. [31]

    Fusion-mamba for cross-modality object detection,

    W. Dong, H. Zhu, S. Lin, X. Luo, Y. Shen, X. Liu, J. Zhang, G. Guo, and B. Zhang, “Fusion-mamba for cross-modality object detection,” arXiv preprint arXiv:2402.16853, 2024. 2

    2024 arXiv

  24. [32]

    Coupled mamba: Enhanced multimodal fusion with coupled state space model,

    W. Li, H. Zhou, J. Yu, Z. Song, and W. Yang, “Coupled mamba: Enhanced multimodal fusion with coupled state space model,” in The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. 2

    2024

  25. [33]

    Sigma: Siamese mamba network for multi-modal semantic segmentation,

    Z. Wan, P . Zhang, Y. Wang, S. Yong, S. Stepputtis, K. Sycara, and Y. Xie, “Sigma: Siamese mamba network for multi-modal semantic segmentation,” in2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). IEEE, 2025, pp. 1734–1744. 2, 4, 5, 6, 8

    2025

  26. [34]

    Faster r-cnn: Towards real- time object detection with region proposal networks,

    S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real- time object detection with region proposal networks,”Advances in Neural Information Processing Systems, vol. 28, 2015. 2

    2015

  27. [35]

    You only look once: Unified, real-time object detection,

    J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 779–788. 2

    2016

  28. [36]

    Yolo9000: Better, faster, stronger,

    J. Redmon and A. Farhadi, “Yolo9000: Better, faster, stronger,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 7263–7271. 2

    2017

  29. [37]

    Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detec- tors,

    C.-Y. Wang, A. Bochkovskiy, and H.-Y. M. Liao, “Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detec- tors,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 7464–7475. 2

    2023

  30. [38]

    Fully convolutional region proposal networks for multispectral person detection,

    D. K ¨onig, M. Adam, C. Jarvers, G. Layher, H. Neumann, and M. Teutsch, “Fully convolutional region proposal networks for multispectral person detection,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2017, pp. 49–

    2017

  31. [39]

    Cross-modality attentive feature fusion for object detection in multispectral remote sensing imagery,

    F. Qingyun and W. Zhaokui, “Cross-modality attentive feature fusion for object detection in multispectral remote sensing imagery,” Pattern Recognition, vol. 130, p. 108786, 2022. 3

    2022

  32. [40]

    Adopting the yolov4 architecture for low-latency multispectral pedestrian detection in autonomous driving,

    K. Roszyk, M. R. Nowicki, and P . Skrzypczy ´nski, “Adopting the yolov4 architecture for low-latency multispectral pedestrian detection in autonomous driving,”Sensors, vol. 22, no. 3, p. 1082,

  33. [41]

    Multimodal object detection by channel switching and spatial attention,

    Y. Cao, J. Bin, J. Hamari, E. Blasch, and Z. Liu, “Multimodal object detection by channel switching and spatial attention,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 403–411. 3

    2023

  34. [42]

    Cross-modality fusion transformer for multispectral object detection,

    F. Qingyun, H. Dapeng, and W. Zhaokui, “Cross-modality fusion transformer for multispectral object detection,”arXiv preprint arXiv:2111.00273, 2021. 3

    2021 arXiv

  35. [43]

    Lraf-net: Long-range attention fusion network for visible-infrared object detection,

    H. Fu, S. Wang, P . Duan, C. Xiao, R. Dian, S. Li, and Z. Li, “Lraf-net: Long-range attention fusion network for visible-infrared object detection,”IEEE Transactions on Neural Networks and Learning Systems, 2023. 3

    2023

  36. [44]

    Multi-modal feature pyramid transformer for rgb-infrared object detection,

    Y. Zhu, X. Sun, M. Wang, and H. Huang, “Multi-modal feature pyramid transformer for rgb-infrared object detection,”IEEE Transactions on Intelligent Transportation Systems, 2023. 3

    2023

  37. [45]

    Icafusion: Iterative cross-attention guided feature fusion for multispectral object detection,

    J. Shen, Y. Chen, Y. Liu, X. Zuo, H. Fan, and W. Yang, “Icafusion: Iterative cross-attention guided feature fusion for multispectral object detection,”Pattern Recognition, vol. 145, p. 109913, 2024. 3

    2024

  38. [46]

    Improving multispectral pedes- trian detection by addressing modality imbalance problems,

    K. Zhou, L. Chen, and X. Cao, “Improving multispectral pedes- trian detection by addressing modality imbalance problems,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVIII 16. Springer, 2020, pp. 787–803. 3

    2020

  39. [47]

    Baanet: Learning bi-directional adaptive attention gates for multispectral pedestrian detection,

    X. Yang, Y. Qian, H. Zhu, C. Wang, and M. Yang, “Baanet: Learning bi-directional adaptive attention gates for multispectral pedestrian detection,” in2022 International Conference on Robotics and Automation (ICRA). IEEE, 2022, pp. 2920–2926. 3

    2022

  40. [48]

    Multispectral fusion for object detection with cyclic fuse-and-refine blocks,

    H. Zhang, E. Fromont, S. Lef `evre, and B. Avignon, “Multispectral fusion for object detection with cyclic fuse-and-refine blocks,” in 2020 IEEE International Conference on Image Processing (ICIP). IEEE, 2020, pp. 276–280. 3

    2020

  41. [49]

    Guided attentive feature fusion for multispectral pedestrian detection,

    ——, “Guided attentive feature fusion for multispectral pedestrian detection,” inProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2021, pp. 72–80. 3

    2021

  42. [50]

    Weakly aligned feature fusion for multimodal object detection,

    L. Zhang, Z. Liu, X. Zhu, Z. Song, X. Yang, Z. Lei, and H. Qiao, “Weakly aligned feature fusion for multimodal object detection,” IEEE Transactions on Neural Networks and Learning Systems, 2021. 3

    2021

  43. [51]

    Uncertainty-guided cross- modal learning for robust multispectral pedestrian detection,

    J. U. Kim, S. Park, and Y. M. Ro, “Uncertainty-guided cross- modal learning for robust multispectral pedestrian detection,”IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 3, pp. 1510–1523, 2021. 3

    2021

  44. [52]

    Confidence-aware fusion using dempster-shafer theory for multispectral pedestrian detection,

    Q. Li, C. Zhang, Q. Hu, H. Fu, and P . Zhu, “Confidence-aware fusion using dempster-shafer theory for multispectral pedestrian detection,”IEEE Transactions on Multimedia, 2022. 3

    2022

  45. [53]

    Stabilizing multispectral pedestrian detection with evidential hybrid fusion,

    Q. Li, C. Zhang, Q. Hu, P . Zhu, H. Fu, and L. Chen, “Stabilizing multispectral pedestrian detection with evidential hybrid fusion,” IEEE Transactions on Circuits and Systems for Video Technology, 2023. 3

    2023

  46. [54]

    Mlpd: Multi-label pedestrian detector in multispectral domain,

    J. Kim, H. Kim, T. Kim, N. Kim, and Y. Choi, “Mlpd: Multi-label pedestrian detector in multispectral domain,”IEEE Robotics and Automation Letters, vol. 6, no. 4, pp. 7846–7853, 2021. 3

    2021

  47. [55]

    Damsdet: Dynamic adaptive multispectral detection transformer with competitive query selection and adaptive feature fusion,

    J. Guo, C. Gao, F. Liu, D. Meng, and X. Gao, “Damsdet: Dynamic adaptive multispectral detection transformer with competitive query selection and adaptive feature fusion,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 464–481. 3

    2024

  48. [56]

    E2E-MFD: Towards end-to-end synchronous multimodal fusion detection,

    J. Zhang, M. Cao, W. Xie, J. Lei, D. Li, W. Huang, Y. Li, and X. Yang, “E2E-MFD: Towards end-to-end synchronous multimodal fusion detection,”Advances in Neural Information Processing Systems, vol. 37, pp. 52 296–52 322, 2024. 3

    2024

  49. [57]

    Efficiently modeling long sequences with structured state spaces,

    A. Gu, K. Goel, and C. Re, “Efficiently modeling long sequences with structured state spaces,” inInternational Conference on Learning Representations, 2022. 3

    2022

  50. [58]

    Combining recurrent, convolutional, and continuous-time models with linear state space layers,

    A. Gu, I. Johnson, K. Goel, K. Saab, T. Dao, A. Rudra, and C. R ´e, “Combining recurrent, convolutional, and continuous-time models with linear state space layers,”Advances in neural information processing systems, vol. 34, pp. 572–585, 2021. 3

    2021

  51. [59]

    Simplified state space layers for sequence modeling,

    J. T. Smith, A. Warrington, and S. Linderman, “Simplified state space layers for sequence modeling,” inThe Eleventh International Conference on Learning Representations, 2023. 3

    2023

  52. [60]

    Cmx: Cross-modal fusion for rgb-x semantic segmentation with transformers,

    J. Zhang, H. Liu, K. Yang, X. Hu, R. Liu, and R. Stiefelhagen, “Cmx: Cross-modal fusion for rgb-x semantic segmentation with transformers,”IEEE Transactions on Intelligent Transportation Systems, vol. 24, no. 12, pp. 14 679–14 694, 2023. 4, 6, 8

    2023

  53. [61]

    GeminiFusion: Efficient pixel-wise multimodal fusion for vision transformer,

    D. Jia, J. Guo, K. Han, H. Wu, C. Zhang, C. Xu, and X. Chen, “GeminiFusion: Efficient pixel-wise multimodal fusion for vision transformer,” inProceedings of the 41st International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 235. PMLR, 21...

    2024

  54. [62]

    VMamba: Visual state space model,

    Y. Liu, Y. Tian, Y. Zhao, H. Yu, L. Xie, Y. Wang, Q. Ye, J. Jiao, and Y. Liu, “VMamba: Visual state space model,” inThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. 4

    2024

  55. [63]

    Shapeconv: Shape-aware convolutional layer for indoor rgb-d semantic segmentation,

    J. Cao, H. Leng, D. Lischinski, D. Cohen-Or, C. Tu, and Y. Li, “Shapeconv: Shape-aware convolutional layer for indoor rgb-d semantic segmentation,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 7088–7097. 4

    2021

  56. [64]

    Spatial information guided convolution for real-time rgbd semantic segmentation,

    L.-Z. Chen, Z. Lin, Z. Wang, Y.-L. Yang, and M.-M. Cheng, “Spatial information guided convolution for real-time rgbd semantic segmentation,”IEEE Transactions on Image Processing, vol. 30, pp. 2313–2324, 2021. 4

    2021

  57. [65]

    Acnet: Attention-based network to exploit complementary features for rgbd semantic segmentation,

    X. Hu, K. Yang, L. Fei, and K. Wang, “Acnet: Attention-based network to exploit complementary features for rgbd semantic segmentation,” in2019 IEEE International Conference on Image Processing (ICIP). IEEE, 2019, pp. 1440–1444. 4

    2019

  58. [66]

    Bi-directional cross-modality feature propagation with separation- and-aggregation gate for rgb-d semantic segmentation,

    X. Chen, K.-Y. Lin, J. Wang, W. Wu, C. Qian, H. Li, and G. Zeng, “Bi-directional cross-modality feature propagation with separation- and-aggregation gate for rgb-d semantic segmentation,” inEuropean Conference on Computer Vision. Springer, 2020, pp. 561–577. 4

    2020

  59. [67]

    Indoor segmentation and support inference from rgbd images,

    P . K. Nathan Silberman, Derek Hoiem and R. Fergus, “Indoor segmentation and support inference from rgbd images,” inECCV,

  60. [68]

    Sun rgb-d: A rgb-d scene understanding benchmark suite,

    S. Song, S. P . Lichtenberg, and J. Xiao, “Sun rgb-d: A rgb-d scene understanding benchmark suite,” in2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 567–576. 6, 7

    2015

  61. [69]

    Gmnet: Graded- feature multilabel-learning network for rgb-thermal urban scene semantic segmentation,

    W. Zhou, J. Liu, J. Lei, L. Yu, and J.-N. Hwang, “Gmnet: Graded- feature multilabel-learning network for rgb-thermal urban scene semantic segmentation,”IEEE Transactions on Image Processing, vol. 30, pp. 7790–7802, 2021. 5, 6

    2021

  62. [70]

    Ccaffmnet: Dual-spectral semantic segmentation network with channel-coordinate attention feature fusion module,

    S. Yi, J. Li, X. Liu, and X. Yuan, “Ccaffmnet: Dual-spectral semantic segmentation network with channel-coordinate attention feature fusion module,”Neurocomputing, vol. 482, pp. 236–251, 2022. 5, 6

    2022

  63. [71]

    Context-aware interaction network for rgb-t semantic segmentation,

    Y. Lv, Z. Liu, and G. Li, “Context-aware interaction network for rgb-t semantic segmentation,”IEEE Transactions on Multimedia, pp. 1–13, 2023. 5, 6, 8

    2023

  64. [72]

    Rgb-t semantic segmentation with location, activation, and sharpening,

    G. Li, Y. Wang, Z. Liu, X. Zhang, and D. Zeng, “Rgb-t semantic segmentation with location, activation, and sharpening,”IEEE Transactions on Circuits and Systems for Video Technology, vol. 33, no. 3, pp. 1223–1235, 2022. 5, 6

    2022

  65. [73]

    Multimodal token fusion for vision transformers,

    Y. Wang, X. Chen, L. Cao, W. Huang, F. Sun, and Y. Wang, “Multimodal token fusion for vision transformers,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 12 186–12 195. 5, 8

    2022

  66. [74]

    Multimae: Multi-modal multi-task masked autoencoders,

    R. Bachmann, D. Mizrahi, A. Atanov, and A. Zamir, “Multimae: Multi-modal multi-task masked autoencoders,” inEuropean Confer- ence on Computer Vision. Springer, 2022, pp. 348–367. 5, 8

    2022

  67. [75]

    Deep multimodal fusion by channel exchanging,

    Y. Wang, W. Huang, F. Sun, T. Xu, Y. Rong, and J. Huang, “Deep multimodal fusion by channel exchanging,”Advances in neural information processing systems, vol. 33, pp. 4835–4845, 2020. 5, 8

    2020

  68. [76]

    Pixel difference convolutional network for rgb-d semantic segmentation,

    J. Yang, L. Bai, Y. Sun, C. Tian, M. Mao, and G. Wang, “Pixel difference convolutional network for rgb-d semantic segmentation,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 34, no. 3, pp. 1481–1492, 2023. 5, 8

    2023

  69. [77]

    Delivering arbitrary-modal semantic segmen- tation,

    J. Zhang, R. Liu, H. Shi, K. Yang, S. Reiß, K. Peng, H. Fu, K. Wang, and R. Stiefelhagen, “Delivering arbitrary-modal semantic segmen- tation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 1136–1147. 5, 8

    2023

  70. [78]

    Depth- aided camouflaged object detection,

    Q. Wang, J. Yang, X. Yu, F. Wang, P . Chen, and F. Zheng, “Depth- aided camouflaged object detection,” inProceedings of the 31st ACM International Conference on Multimedia, 2023, pp. 3297–3306. 5, 8

    2023

  71. [79]

    Source-free depth for object pop-out,

    Z. Wu, D. P . Paudel, D.-P . Fan, J. Wang, S. Wang, C. Demonceaux, R. Timofte, and L. Van Gool, “Source-free depth for object pop-out,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 1032–1042. 5, 8

    2023

  72. [80]

    High-resolution iterative feedback network for camouflaged object detection,

    X. Hu, S. Wang, X. Qin, H. Dai, W. Ren, D. Luo, Y. Tai, and L. Shao, “High-resolution iterative feedback network for camouflaged object detection,” inProceedings of the AAAI Conference on Artificial Intelligence, 2023, pp. 881–889. 5, 8

    2023

  73. [81]

    Feature shrinkage pyramid for camouflaged object detection with transformers,

    Z. Huang, H. Dai, T.-Z. Xiang, S. Wang, H.-X. Chen, J. Qin, and H. Xiong, “Feature shrinkage pyramid for camouflaged object detection with transformers,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 5557–

    2023

  74. [82]

    Hidanet: Rgb-d salient object detection via hierarchical depth awareness,

    Z. Wu, G. Allibert, F. Meriaudeau, C. Ma, and C. Demonceaux, “Hidanet: Rgb-d salient object detection via hierarchical depth awareness,”IEEE Transactions on Image Processing, vol. 32, pp. 2160– 2173, 2023. 6, 8

    2023

  75. [83]

    Object segmentation by mining cross-modal semantics,

    Z. Wu, J. Wang, Z. Zhou, Z. An, Q. Jiang, C. Demonceaux, G. Sun, and R. Timofte, “Object segmentation by mining cross-modal semantics,” inProceedings of the 31st ACM International Conference on Multimedia, 2023, pp. 3455–3464. 6, 8

    2023

  76. [84]

    Fusionmamba: Dynamic feature enhancement for multimodal image fusion with mamba,

    X. Xie, Y. Cui, T. Tan, X. Zheng, and Z. Yu, “Fusionmamba: Dynamic feature enhancement for multimodal image fusion with mamba,” Visual Intelligence, vol. 2, no. 1, p. 37, 2024. 6, 8

    2024

  77. [85]

    Structure- measure: A new way to evaluate foreground maps,

    D.-P . Fan, M.-M. Cheng, Y. Liu, T. Li, and A. Borji, “Structure- measure: A new way to evaluate foreground maps,” inProceedings of the IEEE International Conference on Computer Vision (ICCV), 2017, pp. 4548–4557. 6

    2017

  78. [86]

    Enhanced-alignment measure for binary foreground map evaluation,

    D.-P . Fan, C. Gong, Y. Cao, B. Ren, M.-M. Cheng, and A. Borji, “Enhanced-alignment measure for binary foreground map evaluation,” inProceedings of the International Joint Conference on Artificial Intelligence (IJCAI), 2018, pp. 698–704. [Online]. Available: https://arxiv.org/...

    2018 arXiv

  79. [87]

    Saliency detection via graph-based manifold ranking,

    M.-M. Cheng, N. J. Mitra, X. Huang, P . H. S. Torr, and S.-M. Hu, “Saliency detection via graph-based manifold ranking,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2013, pp. 3166–3173. [Online]. Available: https://ieeexplore.ieee.o...

    2013

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.