Pith. sign in

REVIEW 1 major objections 1 minor 63 references

Dual sparse attention mechanisms in transformers improve multispectral object detection by limiting interactions to high-similarity cross-modal features.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-07-01 00:37 UTC pith:F2CMCF3A

load-bearing objection DSAFormer adds dual sparse cross-attention to trim redundant token mixing in multispectral detection and reports gains on four datasets, but the pruning of low-similarity pairs rests on an assumption that needs direct checks. the 1 major comments →

arxiv 2606.31015 v1 pith:F2CMCF3A submitted 2026-06-30 cs.CV

Dual Sparse Aggregation Transformer for Multispectral Object Detection

Wencong Wu , Xiuwei Zhang , Hanlin Yin , Hongxi Zhang , Yanning Zhang This is my paper
classification cs.CV
keywords multispectral object detectiontransformersparse attentioncross-modal fusionfeature refinementmultimodal detection
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes DSAFormer to reduce redundant token interactions in transformer-based multispectral detection. It introduces spatial and channel sparse cross-attention that retains only high query-key similarity scores, along with multi-scale refinement and a learnable fusion block. The goal is to better exploit complementary information from different spectral modalities while filtering irrelevant areas. Results indicate improved detection accuracy compared to prior methods across four public datasets.

Core claim

The Dual Sparse Aggregation Transformer uses a Dual Sparse Transformer with Spatial Sparse Multi-Head Cross-Attention and Channel Sparse Multi-Head Cross-Attention to selectively capture cross-modal relationships, combined with a Multi-Scale Feature Refinement Layer and Learnable Addition Fusion Block, achieving better detection performance than state-of-the-art methods on the MFAD, FLIR, M3FD, and LLVIP datasets.

What carries the argument

Dual Sparse Transformer (DSFormer) incorporating Spatial Sparse Multi-Head Cross-Attention (SSMHCA) and Channel Sparse Multi-Head Cross-Attention (CSMHCA) that reserve only high query-key similarity scores to eliminate irrelevant cross-modal interactions.

Load-bearing premise

Keeping only the highest query-key similarity scores in spatial and channel attention still captures all the complementary information needed from the different input modalities.

What would settle it

A controlled test on one of the four datasets where objects detected only via low-similarity cross-modal cues are systematically missed after applying the sparse selection.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

1 major / 1 minor

Summary. The manuscript introduces DSAFormer for multispectral object detection, consisting of DSFormer (incorporating SSMHCA to sparsely retain high spatial query-key similarities, CSMHCA for channel-level sparse attention, and MSFRL for hierarchical feature aggregation) plus LAFB for learnable multimodal fusion. It claims this eliminates redundant cross-modal interactions from irrelevant areas and yields superior detection performance versus state-of-the-art methods on the MFAD, FLIR, M³FD, and LLVIP datasets, with source code to be released.

Significance. If the empirical gains are confirmed and the sparse selection is shown not to discard critical modality-specific cues, the dual sparse aggregation strategy could improve efficiency in transformer-based multimodal detection by pruning low-similarity interactions. The explicit commitment to release source code is a clear strength for reproducibility.

major comments (1)
  1. [Abstract and method description of SSMHCA/CSMHCA] The performance attribution to SSMHCA and CSMHCA rests on the assumption (stated in the abstract and method description) that low query-key similarity scores contain only irrelevant information and can be safely discarded without losing complementary cross-modal signals (e.g., thermal edges absent in visible imagery). No supporting evidence—such as similarity histograms contrasting positive vs. negative regions, ablation retaining bottom-k scores, or direct comparison of full vs. sparse attention—is provided to test this assumption, which is load-bearing for the central claim that the sparse mechanisms drive the reported gains over LAFB/MSFRL alone.
minor comments (1)
  1. [Abstract] The abstract asserts 'extensive experimental results' demonstrating superiority but supplies no quantitative metrics, baselines, dataset statistics, or error bars; a brief summary table or key numbers should be added for immediate readability.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive feedback. We address the concern regarding the assumption underlying SSMHCA and CSMHCA below.

read point-by-point responses

  1. Referee: [Abstract and method description of SSMHCA/CSMHCA] The performance attribution to SSMHCA and CSMHCA rests on the assumption (stated in the abstract and method description) that low query-key similarity scores contain only irrelevant information and can be safely discarded without losing complementary cross-modal signals (e.g., thermal edges absent in visible imagery). No supporting evidence—such as similarity histograms contrasting positive vs. negative regions, ablation retaining bottom-k scores, or direct comparison of full vs. sparse attention—is provided to test this assumption, which is load-bearing for the central claim that the sparse mechanisms drive the reported gains over LAFB/MSFRL alone.

    Authors: We agree that the manuscript would benefit from direct evidence testing the assumption that low-similarity interactions are irrelevant. While the current ablations demonstrate performance gains when SSMHCA and CSMHCA are included, they do not isolate the effect of discarding low-similarity scores versus retaining them. In the revised version we will add (1) a direct comparison of full cross-attention versus the proposed sparse versions within the same backbone, and (2) visualizations of query-key similarity score distributions on positive versus negative regions to support the design choice. revision: yes

Circularity Check

0 steps flagged

No significant circularity; claims are purely empirical

full rationale

The manuscript describes an architecture (DSAFormer with SSMHCA, CSMHCA, MSFRL, LAFB) and reports mAP gains on four public datasets. No equations, derivations, fitted parameters renamed as predictions, or load-bearing self-citations appear in the text. The performance claim rests on external experimental benchmarks rather than any internal reduction to inputs or prior author work. This is the standard non-circular outcome for an empirical architecture paper.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

No mathematical model, free parameters, or axioms are described in the abstract; the contribution consists of new architectural modules whose correctness is asserted via empirical results.

pith-pipeline@v0.9.1-grok · 5831 in / 1161 out tokens · 51999 ms · 2026-07-01T00:37:56.039850+00:00 · methodology

0 comments
read the original abstract

Transformer-based approaches have obtained excellent performance in multispectral object detection tasks due to their ability to model long-range dependencies and capture complementary information. However, previous transformer-based multispectral detection methods tend to use all available tokens for similarity calculation, which results in redundant information interaction from irrelevant areas, leading to degraded detection performance. To overcome this challenge, we propose a novel Dual Sparse Aggregation Transformer (DSAFormer) for multispectral object detection, which consists of a Dual Sparse Transformer (DSFormer) and a Learnable Addition Fusion Block (LAFB). Specifically, the DSFormer is designed to exploit and boost cross-modal complementary information, thereby improving detection performance. It incorporates three key components: A Spatial Sparse Multi-Head Cross-Attention (SSMHCA) mechanism selectively captures cross-modal relationships at the spatial level by reserving only the high query-key similarity scores, eliminating irrelevant interactions. A Channel Sparse Multi-Head Cross-Attention (CSMHCA) mechanism performs similar sparse calculations at the channel level to enhance feature representation and filter out low matching query-key. A Multi-Scale Feature Refinement Layer (MSFRL) is developed to aggregate hierarchical features and suppress redundant information. To effectively fuse multimodal features, the LAFB is introduced to aggregate intramodal and intermodal feature information by feature reweighting. Extensive experimental results have demonstrated that our proposed DSAFormer achieves better detection performance against state-of-the-art methods on four public datasets, including the MFAD, FLIR, M$^3$FD, and LLVIP. The source code of our DSAFormer will be released at https://github.com/WenCongWu/DSAFormer.

Figures

Figures reproduced from arXiv: 2606.31015 by Hanlin Yin, Hongxi Zhang, Wencong Wu, Xiuwei Zhang, Yanning Zhang.

Figure 1
Figure 1. Figure 1: Left image: existing transformer-based multispectral detection methods perform token interactions in relevant and irrelevant areas between VIS and IR modalities, resulting in redundant information for similarity calculation. Right image: the information exchange between irrelevant regions across modalities generates low weights in the attention map, and the Softmax function will increase the value of these… view at source ↗
Figure 2
Figure 2. Figure 2: The proposed multispectral object detection framework, where the DSAFormer denotes our designed Dual Sparse [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The architecture of the proposed Dual Sparse Aggre [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: The structure of the proposed Spatial Sparse Transformer (SSFormer). [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: The structure of the proposed Multi-Scale Feature [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: The detection samples of multispectral object detection [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: The qualitative comparison of multispectral object detection methods on the FLIR dataset. The red triangles mark [PITH_FULL_IMAGE:figures/full_fig_p011_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: The failed detection case of our DSAFormer on a pair [PITH_FULL_IMAGE:figures/full_fig_p011_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: The visual comparison of multispectral object detection models on the M [PITH_FULL_IMAGE:figures/full_fig_p012_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: The visualization of multi-head dense and sparse attention maps and binary mask matrix for a pair of VIS-IR images [PITH_FULL_IMAGE:figures/full_fig_p012_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: The visual comparison of MLP and MSFRL in [PITH_FULL_IMAGE:figures/full_fig_p012_11.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

63 extracted references · 63 canonical work pages · 1 internal anchor

  1. [1]

    Faster R-CNN: towards real- time object detection with region proposal networks,

    S. Ren, K. He, R. B. Girshick, and J. Sun, “Faster R-CNN: towards real- time object detection with region proposal networks,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, pp. 1137– 1149, 2017

    work page 2017

  2. [2]

    Yolov5 by ultralytics,

    G. Jocher, “Yolov5 by ultralytics,” 2020, url https://github.com/ ultralytics/yolov5

    work page 2020

  3. [3]

    Ultralytics yolo,

    G. Jocher, A. Chaurasia, and J. Qiu, “Ultralytics yolo,” 2023, url https: //github.com/ultralytics/ultralytics

    work page 2023

  4. [4]

    Yolov10: Real-time end-to-end object detection,

    A. Wang, H. Chen, L. Liu, K. Chen, Z. Lin, J. Han, and G. Ding, “Yolov10: Real-time end-to-end object detection,” inAdvances in Neural Information Processing Systems, 2024

    work page 2024

  5. [5]

    Deformable cross-attention transformer for weakly aligned RGB-T pedestrian detection,

    Y . Hu, X. Chen, S. Wang, L. Liu, H. Shi, L. Fan, J. Tian, and J. Liang, “Deformable cross-attention transformer for weakly aligned RGB-T pedestrian detection,”IEEE Transactions on Multimedia, vol. 27, pp. 4400–4411, 2025

    work page 2025

  6. [6]

    Dacfusion: Dual asymmetric cross-attention guided feature fusion for multispectral object detection,

    J. Qian, B. Qiao, Y . Zhang, T. Liu, S. Wang, G. Wu, and D. Han, “Dacfusion: Dual asymmetric cross-attention guided feature fusion for multispectral object detection,”Neurocomputing, vol. 635, p. 129913, 2025

    work page 2025

  7. [7]

    Ei2det: Edge-guided illumination-aware interactive learning for visible-infrared object detec- tion,

    K. Hu, Y . He, Y . Li, J. Zhao, S. Chen, and Y . Kang, “Ei2det: Edge-guided illumination-aware interactive learning for visible-infrared object detec- tion,”IEEE Transactions on Circuits and Systems for Video Technology, vol. 35, no. 7, pp. 7101–7115, 2025

    work page 2025

  8. [8]

    Guided attentive feature fusion for multispectral pedestrian detection,

    H. Zhang, ´E. Fromont, S. Lef `evre, and B. Avignon, “Guided attentive feature fusion for multispectral pedestrian detection,” inIEEE Winter Conference on Applications of Computer Vision, 2021, pp. 72–80

    work page 2021

  9. [9]

    Vehicle detection based on adaptive multimodal feature fusion and cross-modal vehicle index using RGB-T images,

    Y . Wu, X. Guan, B. Zhao, L. Ni, and M. Huang, “Vehicle detection based on adaptive multimodal feature fusion and cross-modal vehicle index using RGB-T images,”IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 16, pp. 8166–8177, 2023

    work page 2023

  10. [10]

    Cross-modal oriented object detection of UA V aerial images based on image feature,

    H. Wang, C. Wang, Q. Fu, D. Zhang, R. Kou, Y . Yu, and J. Song, “Cross-modal oriented object detection of UA V aerial images based on image feature,”IEEE Transactions on Geoscience and Remote Sensing, vol. 62, pp. 1–21, 2024

    work page 2024

  11. [11]

    Cross-Modality Fusion Transformer for Multispectral Object Detection

    Q. Fang, D. Han, and Z. Wang, “Cross-modality fusion transformer for multispectral object detection,”arXiv preprint arXiv:2111.00273, 2021

    work page Pith review arXiv 2021

  12. [12]

    Crossformer: Cross-guided attention for multi-modal object detection,

    S. Lee, J. Park, and J. Park, “Crossformer: Cross-guided attention for multi-modal object detection,”Pattern Recognition Letters, vol. 179, pp. 144–150, 2024

    work page 2024

  13. [13]

    Rich feature hierarchies for accurate object detection and semantic segmentation,

    R. B. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich feature hierarchies for accurate object detection and semantic segmentation,” in IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 580–587

    work page 2014

  14. [14]

    Fast R-CNN,

    R. B. Girshick, “Fast R-CNN,” inIEEE International Conference on Computer Vision, 2015, pp. 1440–1448

    work page 2015

  15. [15]

    Mask R-CNN,

    K. He, G. Gkioxari, P. Doll ´ar, and R. B. Girshick, “Mask R-CNN,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 2, pp. 386–397, 2020

    work page 2020

  16. [16]

    Improving multispectral pedestrian detection by addressing modality imbalance problems,

    K. Zhou, L. Chen, and X. Cao, “Improving multispectral pedestrian detection by addressing modality imbalance problems,” inEuropean Conference on Computer Vision, vol. 12363, 2020, pp. 787–803

    work page 2020

  17. [17]

    Cross-modality interactive attention network for multispectral pedestrian detection,

    L. Zhang, Z. Liu, S. Zhang, X. Yang, H. Qiao, K. Huang, and A. Hus- sain, “Cross-modality interactive attention network for multispectral pedestrian detection,”Information Fusion, vol. 50, pp. 20–29, 2019

    work page 2019

  18. [18]

    Multimodal object detection by channel switching and spatial attention,

    Y . Cao, J. Bin, J. Hamari, E. Blasch, and Z. Liu, “Multimodal object detection by channel switching and spatial attention,” inIEEE Confer- ence on Computer Vision and Pattern Recognition Workshops, 2023, pp. 403–411

    work page 2023

  19. [19]

    Superyolo: Super resolution assisted object detection in multimodal remote sensing im- agery,

    J. Zhang, J. Lei, W. Xie, Z. Fang, Y . Li, and Q. Du, “Superyolo: Super resolution assisted object detection in multimodal remote sensing im- agery,”IEEE Transactions on Geoscience and Remote Sensing, vol. 61, pp. 1–15, 2023

    work page 2023

  20. [20]

    Deep cross-modal representation learning and distillation for illumination-invariant pedestrian detection,

    T. Liu, K. Lam, R. Zhao, and G. Qiu, “Deep cross-modal representation learning and distillation for illumination-invariant pedestrian detection,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 1, pp. 315–329, 2022

    work page 2022

  21. [21]

    Cross-modality com- plementary information fusion for multispectral pedestrian detection,

    C. Yan, H. Zhang, X. Li, Y . Yang, and D. Yuan, “Cross-modality com- plementary information fusion for multispectral pedestrian detection,” Neural Computing and Applications, vol. 35, no. 14, pp. 10 361–10 386, 2023

    work page 2023

  22. [22]

    Cross-modality attentive feature fusion for object detection in multispectral remote sensing imagery,

    Q. Fang and Z. Wang, “Cross-modality attentive feature fusion for object detection in multispectral remote sensing imagery,”Pattern Recognition, vol. 130, p. 108786, 2022

    work page 2022

  23. [23]

    Illumination-guided RGBT object detection with inter- and intra-modality fusion,

    Y . Zhang, H. Yu, Y . He, X. Wang, and W. Yang, “Illumination-guided RGBT object detection with inter- and intra-modality fusion,”IEEE Transactions on Instrumentation and Measurement, vol. 72, pp. 1–13, 2023

    work page 2023

  24. [24]

    Weakly aligned feature fusion for multimodal object detection,

    L. Zhang, Z. Liu, X. Zhu, Z. Song, X. Yang, Z. Lei, and H. Qiao, “Weakly aligned feature fusion for multimodal object detection,”IEEE Transactions on Neural Networks and Learning Systems, vol. 36, no. 3, pp. 4145–4159, 2025

    work page 2025

  25. [25]

    Fd2- net: Frequency-driven feature decomposition network for infrared-visible object detection,

    K. Li, D. Wang, Z. Hu, S. Li, W. Ni, L. Zhao, and Q. Wang, “Fd2- net: Frequency-driven feature decomposition network for infrared-visible object detection,” inAAAI Conference on Artificial Intelligence, 2025, pp. 4797–4805

    work page 2025

  26. [26]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” inAdvances in Neural Information Processing Systems, 2017, pp. 5998–6008

    work page 2017

  27. [27]

    An image is worth 16x16 words: Trans- MANUSCRIPT SUBMITTED TO IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 16 formers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Trans- MANUSCRIPT SUBMITTED TO IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 16 formers for image recognition at scale,” inInternational Conference ...

    work page 2021

  28. [28]

    Swin transformer: Hierarchical vision transformer using shifted windows,

    Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” inIEEE International Conference on Computer Vision, 2021, pp. 9992–10 002

    work page 2021

  29. [29]

    Swin transformer V2: scaling up capacity and resolution,

    Z. Liu, H. Hu, Y . Lin, Z. Yao, Z. Xie, Y . Wei, J. Ning, Y . Cao, Z. Zhang, L. Dong, F. Wei, and B. Guo, “Swin transformer V2: scaling up capacity and resolution,” inIEEE Conference on Computer Vision and Pattern Recognition, 2022, pp. 11 999–12 009

    work page 2022

  30. [30]

    YOLO-MS: multispectral object detection via feature interaction and self-attention guided fusion,

    Y . Xie, L. Zhang, X. Yu, and W. Xie, “YOLO-MS: multispectral object detection via feature interaction and self-attention guided fusion,”IEEE Transactions on Cognitive and Developmental Systems, vol. 15, no. 4, pp. 2132–2143, 2023

    work page 2023

  31. [31]

    Multi-scale aggregation transformers for multispectral object detection,

    S. You, X. Xie, Y . Feng, C. Mei, and Y . Ji, “Multi-scale aggregation transformers for multispectral object detection,”IEEE Signal Processing Letters, vol. 30, pp. 1172–1176, 2023

    work page 2023

  32. [32]

    Differential feature awareness network within antagonistic learning for infrared-visible object detection,

    R. Zhang, L. Li, Q. Zhang, J. Zhang, L. Xu, B. Zhang, and B. Wang, “Differential feature awareness network within antagonistic learning for infrared-visible object detection,”IEEE Transactions on Circuits and Systems for Video Technology, vol. 34, no. 8, pp. 6735–6748, 2024

    work page 2024

  33. [33]

    Mmi-det: Exploring multi-modal integration for visible and infrared object detection,

    Y . Zeng, T. Liang, Y . Jin, and Y . Li, “Mmi-det: Exploring multi-modal integration for visible and infrared object detection,”IEEE Transactions on Circuits and Systems for Video Technology, vol. 34, no. 11, pp. 11 198–11 213, 2024

    work page 2024

  34. [34]

    Seadate: Remedy dual-attention transformer with semantic alignment via contrast learning for multimodal object detection,

    S. Dong, W. Xie, D. Yang, Y . Li, J. Zhang, J. Tian, and J. Lei, “Seadate: Remedy dual-attention transformer with semantic alignment via contrast learning for multimodal object detection,”IEEE Transactions on Circuits and Systems for Video Technology, vol. 35, no. 5, pp. 4713–4726, 2025

    work page 2025

  35. [35]

    Icafusion: Iterative cross-attention guided feature fusion for multispectral object detection,

    J. Shen, Y . Chen, Y . Liu, X. Zuo, H. Fan, and W. Yang, “Icafusion: Iterative cross-attention guided feature fusion for multispectral object detection,”Pattern Recognition, vol. 145, p. 109913, 2024

    work page 2024

  36. [36]

    Multidimensional fusion net- work for multispectral object detection,

    F. Yang, B. Liang, W. Li, and J. Zhang, “Multidimensional fusion net- work for multispectral object detection,”IEEE Transactions on Circuits and Systems for Video Technology, vol. 35, no. 1, pp. 547–560, 2025

    work page 2025

  37. [37]

    Sparsevit: Revis- iting activation sparsity for efficient high-resolution vision transformer,

    X. Chen, Z. Liu, H. Tang, L. Yi, H. Zhao, and S. Han, “Sparsevit: Revis- iting activation sparsity for efficient high-resolution vision transformer,” inIEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 2061–2070

    work page 2023

  38. [38]

    Deformable DETR: deformable transformers for end-to-end object detection,

    X. Zhu, W. Su, L. Lu, B. Li, X. Wang, and J. Dai, “Deformable DETR: deformable transformers for end-to-end object detection,” in International Conference on Learning Representations, 2021

    work page 2021

  39. [39]

    Dynamicvit: Efficient vision transformers with dynamic token sparsification,

    Y . Rao, W. Zhao, B. Liu, J. Lu, J. Zhou, and C. Hsieh, “Dynamicvit: Efficient vision transformers with dynamic token sparsification,” in Advances in Neural Information Processing Systems, 2021, pp. 13 937– 13 949

    work page 2021

  40. [40]

    Multi-modal feature pyramid transformer for rgb-infrared object detection,

    Y . Zhu, X. Sun, M. Wang, and H. Huang, “Multi-modal feature pyramid transformer for rgb-infrared object detection,”IEEE Transactions on Intelligent Transportation Systems, vol. 24, no. 9, pp. 9984–9995, 2023

    work page 2023

  41. [41]

    Run, don’t walk: Chasing higher FLOPS for faster neural networks,

    J. Chen, S. Kao, H. He, W. Zhuo, S. Wen, C. Lee, and S. G. Chan, “Run, don’t walk: Chasing higher FLOPS for faster neural networks,” inIEEE Conference on Computer Vision and Pattern Recognition, 2023, pp. 12 021–12 031

    work page 2023

  42. [42]

    MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications

    A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “Mobilenets: Efficient convo- lutional neural networks for mobile vision applications,”arXiv preprint arXiv:1704.04861, 2017

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  43. [43]

    A tutorial on the cross-entropy method,

    P. de Boer, D. P. Kroese, S. Mannor, and R. Y . Rubinstein, “A tutorial on the cross-entropy method,”Annals Of Operations Research, vol. 134, no. 1, pp. 19–67, 2005

    work page 2005

  44. [44]

    Generalized intersection over union: A metric and a loss for bounding box regression,

    H. Rezatofighi, N. Tsoi, J. Gwak, A. Sadeghian, I. D. Reid, and S. Savarese, “Generalized intersection over union: A metric and a loss for bounding box regression,” inIEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 658–666

    work page 2019

  45. [45]

    Multispectral fusion for object detection with cyclic fuse-and-refine blocks,

    H. Zhang, ´E. Fromont, S. Lef `evre, and B. Avignon, “Multispectral fusion for object detection with cyclic fuse-and-refine blocks,” inIEEE International Conference on Image Processing, 2020, pp. 276–280

    work page 2020

  46. [46]

    Target- aware dual adversarial learning and a multi-scenario multi-modality benchmark to fuse infrared and visible for object detection,

    J. Liu, X. Fan, Z. Huang, G. Wu, R. Liu, W. Zhong, and Z. Luo, “Target- aware dual adversarial learning and a multi-scenario multi-modality benchmark to fuse infrared and visible for object detection,” inIEEE Conference on Computer Vision and Pattern Recognition, 2022, pp. 5792–5801

    work page 2022

  47. [47]

    LLVIP: A visible-infrared paired dataset for low-light vision,

    X. Jia, C. Zhu, M. Li, W. Tang, and W. Zhou, “LLVIP: A visible-infrared paired dataset for low-light vision,” inIEEE International Conference on Computer Vision Workshops, 2021, pp. 3489–3497

    work page 2021

  48. [48]

    Ex- plicit attention-enhanced fusion for rgb-thermal perception tasks,

    M. Liang, J. Hu, C. Bao, H. Feng, F. Deng, and T. L. Lam, “Ex- plicit attention-enhanced fusion for rgb-thermal perception tasks,”IEEE Robotics and Automation Letters, vol. 8, no. 7, pp. 4060–4067, 2023

    work page 2023

  49. [49]

    Multimodal object detection via probabilistic ensembling,

    Y . Chen, J. Shi, Z. Ye, C. Mertz, D. Ramanan, and S. Kong, “Multimodal object detection via probabilistic ensembling,” inEuropean Conference on Computer Vision, vol. 13669, 2022, pp. 139–158

    work page 2022

  50. [50]

    Fusion-mamba for cross-modality object detection,

    W. Dong, H. Zhu, S. Lin, X. Luo, Y . Shen, G. Guo, and B. Zhang, “Fusion-mamba for cross-modality object detection,”IEEE Transactions on Multimedia, vol. 27, pp. 7392–7406, 2025

    work page 2025

  51. [51]

    Removal then selection: A coarse-to-fine fusion perspective for rgb-infrared object detection,

    T. Zhao, M. Yuan, F. Jiang, N. Wang, and X. Wei, “Removal then selection: A coarse-to-fine fusion perspective for rgb-infrared object detection,”IEEE Transactions on Intelligent Transportation Systems, vol. 27, no. 2, pp. 2504–2519, 2026

    work page 2026

  52. [52]

    Dual-perspective alignment learning for multimodal remote sensing object detection,

    Y . Liu, W. Guo, C. Yao, and L. Zhang, “Dual-perspective alignment learning for multimodal remote sensing object detection,”IEEE Trans- actions on Geoscience and Remote Sensing, vol. 63, pp. 1–15, 2025

    work page 2025

  53. [53]

    Dwsf-net: A dynamic wavelet-based spatial-frequency fusion network for multispectral object detection,

    F. Yang, W. Li, L. Li, M. Yang, and J. Zhang, “Dwsf-net: A dynamic wavelet-based spatial-frequency fusion network for multispectral object detection,”IEEE Transactions on Multimedia, pp. 1–13, 2026

    work page 2026

  54. [54]

    Didfuse: Deep image decomposition for infrared and visible image fusion,

    Z. Zhao, S. Xu, C. Zhang, J. Liu, J. Zhang, and P. Li, “Didfuse: Deep image decomposition for infrared and visible image fusion,” in International Joint Conference on Artificial Intelligence, 2020, pp. 970– 976

    work page 2020

  55. [55]

    Sdnet: A versatile squeeze-and-decomposition network for real-time image fusion,

    H. Zhang and J. Ma, “Sdnet: A versatile squeeze-and-decomposition network for real-time image fusion,”International Journal of Computer Vision, vol. 129, no. 10, pp. 2761–2785, 2021

    work page 2021

  56. [56]

    Detfusion: A detection-driven infrared and visible image fusion network,

    Y . Sun, B. Cao, P. Zhu, and Q. Hu, “Detfusion: A detection-driven infrared and visible image fusion network,” inACM International Conference on Multimedia, 2022, pp. 4003–4011

    work page 2022

  57. [57]

    Superfusion: A ver- satile image registration and fusion network with semantic awareness,

    L. Tang, Y . Deng, Y . Ma, J. Huang, and J. Ma, “Superfusion: A ver- satile image registration and fusion network with semantic awareness,” IEEE/CAA Journal of Automatica Sinica, vol. 9, no. 12, pp. 2121–2137, 2022

    work page 2022

  58. [58]

    Rfnet: Unsupervised network for mutually reinforcing multi-modal image registration and fusion,

    H. Xu, J. Ma, J. Yuan, Z. Le, and W. Liu, “Rfnet: Unsupervised network for mutually reinforcing multi-modal image registration and fusion,” in IEEE Conference on Computer Vision and Pattern Recognition, 2022, pp. 19 647–19 656

    work page 2022

  59. [59]

    Cddfuse: Correlation-driven dual-branch feature decomposition for multi-modality image fusion,

    Z. Zhao, H. Bai, J. Zhang, Y . Zhang, S. Xu, Z. Lin, R. Timofte, and L. V . Gool, “Cddfuse: Correlation-driven dual-branch feature decomposition for multi-modality image fusion,” inIEEE Conference on Computer Vision and Pattern Recognition, 2023, pp. 5906–5916

    work page 2023

  60. [60]

    Learning a graph neural network with cross modality interaction for image fusion,

    J. Li, J. Chen, J. Liu, and H. Ma, “Learning a graph neural network with cross modality interaction for image fusion,” inACM International Conference on Multimedia, 2023, pp. 4471–4479

    work page 2023

  61. [61]

    Multispectral state-space feature fusion: Bridging shared and cross-parametric inter- actions for object detection,

    J. Shen, H. Zhan, S. Dong, X. Zuo, W. Yang, and H. Ling, “Multispectral state-space feature fusion: Bridging shared and cross-parametric inter- actions for object detection,”Information Fusion, vol. 127, p. 103895, 2026

    work page 2026

  62. [62]

    Ccldet: A cross-modality and cross-domain low-light detector,

    X. Shang, N. Li, D. Li, J. Lv, W. Zhao, R. Zhang, and J. Xu, “Ccldet: A cross-modality and cross-domain low-light detector,”IEEE Transactions on Intelligent Transportation Systems, vol. 26, no. 3, pp. 3284–3294, 2025

    work page 2025

  63. [63]

    Unirgb-ir: A unified framework for visible-infrared semantic tasks via adapter tuning,

    M. Yuan, B. Cui, T. Zhao, J. Wang, S. Fu, X. Yang, and X. Wei, “Unirgb-ir: A unified framework for visible-infrared semantic tasks via adapter tuning,” inACM International Conference on Multimedia, 2025, p. 2409–2418. Wencong Wuis pursuing the Ph.D. degree at the School of Computer Science, Northwestern Poly- technical University, Xi’an, China. Before, he...

    work page 2025